hypothesis driven development week 1

How to Implement Hypothesis-Driven Development

Remember back to the time when we were in high school science class. Our teachers had a framework for helping us learn – an experimental approach based on the best available evidence at hand. We were asked to make observations about the world around us, then attempt to form an explanation or hypothesis to explain what we had observed. We then tested this hypothesis by predicting an outcome based on our theory that would be achieved in a controlled experiment – if the outcome was achieved, we had proven our theory to be correct.

We could then apply this learning to inform and test other hypotheses by constructing more sophisticated experiments, and tuning, evolving or abandoning any hypothesis as we made further observations from the results we achieved.

Experimentation is the foundation of the scientific method, which is a systematic means of exploring the world around us. Although some experiments take place in laboratories, it is possible to perform an experiment anywhere, at any time, even in software development.

Practicing  Hypothesis-Driven Development  is thinking about the development of new ideas, products and services – even organizational change – as a series of experiments to determine whether an expected outcome will be achieved. The process is iterated upon until a desirable outcome is obtained or the idea is determined to be not viable.

We need to change our mindset to view our proposed solution to a problem statement as a hypothesis, especially in new product or service development – the market we are targeting, how a business model will work, how code will execute and even how the customer will use it.

We do not do projects anymore, only experiments. Customer discovery and Lean Startup strategies are designed to test assumptions about customers. Quality Assurance is testing system behavior against defined specifications. The experimental principle also applies in Test-Driven Development – we write the test first, then use the test to validate that our code is correct, and succeed if the code passes the test. Ultimately, product or service development is a process to test a hypothesis about system behaviour in the environment or market it is developed for.

The key outcome of an experimental approach is measurable evidence and learning.

Learning is the information we have gained from conducting the experiment. Did what we expect to occur actually happen? If not, what did and how does that inform what we should do next?

In order to learn we need use the scientific method for investigating phenomena, acquiring new knowledge, and correcting and integrating previous knowledge back into our thinking.

As the software development industry continues to mature, we now have an opportunity to leverage improved capabilities such as Continuous Design and Delivery to maximize our potential to learn quickly what works and what does not. By taking an experimental approach to information discovery, we can more rapidly test our solutions against the problems we have identified in the products or services we are attempting to build. With the goal to optimize our effectiveness of solving the right problems, over simply becoming a feature factory by continually building solutions.

The steps of the scientific method are to:

• Make observations
• Formulate a hypothesis
• Design an experiment to test the hypothesis
• State the indicators to evaluate if the experiment has succeeded
• Conduct the experiment
• Evaluate the results of the experiment
• Accept or reject the hypothesis
• If necessary, make and test a new hypothesis

Using an experimentation approach to software development

We need to challenge the concept of having fixed requirements for a product or service. Requirements are valuable when teams execute a well known or understood phase of an initiative, and can leverage well understood practices to achieve the outcome. However, when you are in an exploratory, complex and uncertain phase you need hypotheses.

Handing teams a set of business requirements reinforces an order-taking approach and mindset that is flawed.

Business does the thinking and ‘knows’ what is right. The purpose of the development team is to implement what they are told. But when operating in an area of uncertainty and complexity, all the members of the development team should be encouraged to think and share insights on the problem and potential solutions. A team simply taking orders from a business owner is not utilizing the full potential, experience and competency that a cross-functional multi-disciplined team offers.

Framing hypotheses

The traditional user story framework is focused on capturing requirements for what we want to build and for whom, to enable the user to receive a specific benefit from the system.

As A…. <role>

I Want… <goal/desire>

So That… <receive benefit>

Behaviour Driven Development (BDD) and Feature Injection  aims to improve the original framework by supporting communication and collaboration between developers, tester and non-technical participants in a software project.

In Order To… <receive benefit>

As A… <role>

When viewing work as an experiment, the traditional story framework is insufficient. As in our high school science experiment, we need to define the steps we will take to achieve the desired outcome. We then need to state the specific indicators (or signals) we expect to observe that provide evidence that our hypothesis is valid. These need to be stated before conducting the test to reduce biased interpretations of the results. 

If we observe signals that indicate our hypothesis is correct, we can be more confident that we are on the right path and can alter the user story framework to reflect this.

Therefore, a user story structure to support Hypothesis-Driven Development would be;

We believe < this capability >

What functionality we will develop to test our hypothesis? By defining a ‘test’ capability of the product or service that we are attempting to build, we identify the functionality and hypothesis we want to test.

Will result in < this outcome >

What is the expected outcome of our experiment? What is the specific result we expect to achieve by building the ‘test’ capability?

We will know we have succeeded when < we see a measurable signal >

What signals will indicate that the capability we have built is effective? What key metrics (qualitative or quantitative) we will measure to provide evidence that our experiment has succeeded and give us enough confidence to move to the next stage.

The threshold you use for statistically significance will depend on your understanding of the business and context you are operating within. Not every company has the user sample size of Amazon or Google to run statistically significant experiments in a short period of time. Limits and controls need to be defined by your organization to determine acceptable evidence thresholds that will allow the team to advance to the next step.

For example if you are building a rocket ship you may want your experiments to have a high threshold for statistical significance. If you are deciding between two different flows intended to help increase user sign up you may be happy to tolerate a lower significance threshold.

The final step is to clearly and visibly state any assumptions made about our hypothesis, to create a feedback loop for the team to provide further input, debate and understanding of the circumstance under which we are performing the test. Are they valid and make sense from a technical and business perspective?

Hypotheses when aligned to your MVP can provide a testing mechanism for your product or service vision. They can test the most uncertain areas of your product or service, in order to gain information and improve confidence.

Examples of Hypothesis-Driven Development user stories are;

Business story

We Believe That increasing the size of hotel images on the booking page

Will Result In improved customer engagement and conversion

We Will Know We Have Succeeded When we see a 5% increase in customers who review hotel images who then proceed to book in 48 hours.

It is imperative to have effective monitoring and evaluation tools in place when using an experimental approach to software development in order to measure the impact of our efforts and provide a feedback loop to the team. Otherwise we are essentially blind to the outcomes of our efforts.

In agile software development we define working software as the primary measure of progress.

By combining Continuous Delivery and Hypothesis-Driven Development we can now define working software and validated learning as the primary measures of progress.

Ideally we should not say we are done until we have measured the value of what is being delivered – in other words, gathered data to validate our hypothesis.

Examples of how to gather data is performing A/B Testing to test a hypothesis and measure to change in customer behaviour. Alternative testings options can be customer surveys, paper prototypes, user and/or guerrilla testing.

One example of a company we have worked with that uses Hypothesis-Driven Development is  lastminute.com . The team formulated a hypothesis that customers are only willing to pay a max price for a hotel based on the time of day they book. Tom Klein, CEO and President of Sabre Holdings shared  the story  of how they improved conversion by 400% within a week.

Combining practices such as Hypothesis-Driven Development and Continuous Delivery accelerates experimentation and amplifies validated learning. This gives us the opportunity to accelerate the rate at which we innovate while relentlessly reducing cost, leaving our competitors in the dust. Ideally we can achieve the ideal of one piece flow: atomic changes that enable us to identify causal relationships between the changes we make to our products and services, and their impact on key metrics.

As Kent Beck said, “Test-Driven Development is a great excuse to think about the problem before you think about the solution”. Hypothesis-Driven Development is a great opportunity to test what you think the problem is, before you work on the solution.

How can you achieve faster growth?

• Work together
• Product development
• Ways of working

Have you read my two bestsellers, Unlearn and Lean Enterprise? If not, please do. If you have, please write a review!

• Read my story
• Get in touch

• Oval Copy 2 Blog

How to Implement Hypothesis-Driven Development

• Facebook__x28_alt_x29_ Copy

Remember back to the time when we were in high school science class. Our teachers had a framework for helping us learn – an experimental approach based on the best available evidence at hand. We were asked to make observations about the world around us, then attempt to form an explanation or hypothesis to explain what we had observed. We then tested this hypothesis by predicting an outcome based on our theory that would be achieved in a controlled experiment – if the outcome was achieved, we had proven our theory to be correct.

We could then apply this learning to inform and test other hypotheses by constructing more sophisticated experiments, and tuning, evolving, or abandoning any hypothesis as we made further observations from the results we achieved.

Experimentation is the foundation of the scientific method, which is a systematic means of exploring the world around us. Although some experiments take place in laboratories, it is possible to perform an experiment anywhere, at any time, even in software development.

Practicing Hypothesis-Driven Development [1] is thinking about the development of new ideas, products, and services – even organizational change – as a series of experiments to determine whether an expected outcome will be achieved. The process is iterated upon until a desirable outcome is obtained or the idea is determined to be not viable.

We need to change our mindset to view our proposed solution to a problem statement as a hypothesis, especially in new product or service development – the market we are targeting, how a business model will work, how code will execute and even how the customer will use it.

We do not do projects anymore, only experiments. Customer discovery and Lean Startup strategies are designed to test assumptions about customers. Quality Assurance is testing system behavior against defined specifications. The experimental principle also applies in Test-Driven Development – we write the test first, then use the test to validate that our code is correct, and succeed if the code passes the test. Ultimately, product or service development is a process to test a hypothesis about system behavior in the environment or market it is developed for.

The key outcome of an experimental approach is measurable evidence and learning. Learning is the information we have gained from conducting the experiment. Did what we expect to occur actually happen? If not, what did and how does that inform what we should do next?

In order to learn we need to use the scientific method for investigating phenomena, acquiring new knowledge, and correcting and integrating previous knowledge back into our thinking.

As the software development industry continues to mature, we now have an opportunity to leverage improved capabilities such as Continuous Design and Delivery to maximize our potential to learn quickly what works and what does not. By taking an experimental approach to information discovery, we can more rapidly test our solutions against the problems we have identified in the products or services we are attempting to build. With the goal to optimize our effectiveness of solving the right problems, over simply becoming a feature factory by continually building solutions.

The steps of the scientific method are to:

• Make observations
• Formulate a hypothesis
• Design an experiment to test the hypothesis
• State the indicators to evaluate if the experiment has succeeded
• Conduct the experiment
• Evaluate the results of the experiment
• Accept or reject the hypothesis
• If necessary, make and test a new hypothesis

Using an experimentation approach to software development

We need to challenge the concept of having fixed requirements for a product or service. Requirements are valuable when teams execute a well known or understood phase of an initiative and can leverage well-understood practices to achieve the outcome. However, when you are in an exploratory, complex and uncertain phase you need hypotheses. Handing teams a set of business requirements reinforces an order-taking approach and mindset that is flawed. Business does the thinking and ‘knows’ what is right. The purpose of the development team is to implement what they are told. But when operating in an area of uncertainty and complexity, all the members of the development team should be encouraged to think and share insights on the problem and potential solutions. A team simply taking orders from a business owner is not utilizing the full potential, experience and competency that a cross-functional multi-disciplined team offers.

Framing Hypotheses

The traditional user story framework is focused on capturing requirements for what we want to build and for whom, to enable the user to receive a specific benefit from the system.

As A…. <role>

I Want… <goal/desire>

So That… <receive benefit>

Behaviour Driven Development (BDD) and Feature Injection aims to improve the original framework by supporting communication and collaboration between developers, tester and non-technical participants in a software project.

In Order To… <receive benefit>

As A… <role>

When viewing work as an experiment, the traditional story framework is insufficient. As in our high school science experiment, we need to define the steps we will take to achieve the desired outcome. We then need to state the specific indicators (or signals) we expect to observe that provide evidence that our hypothesis is valid. These need to be stated before conducting the test to reduce the bias of interpretation of results.

If we observe signals that indicate our hypothesis is correct, we can be more confident that we are on the right path and can alter the user story framework to reflect this.

Therefore, a user story structure to support Hypothesis-Driven Development would be;

We believe < this capability >

What functionality we will develop to test our hypothesis? By defining a ‘test’ capability of the product or service that we are attempting to build, we identify the functionality and hypothesis we want to test.

Will result in < this outcome >

What is the expected outcome of our experiment? What is the specific result we expect to achieve by building the ‘test’ capability?

We will have confidence to proceed when < we see a measurable signal >

What signals will indicate that the capability we have built is effective? What key metrics (qualitative or quantitative) we will measure to provide evidence that our experiment has succeeded and give us enough confidence to move to the next stage.

The threshold you use for statistical significance will depend on your understanding of the business and context you are operating within. Not every company has the user sample size of Amazon or Google to run statistically significant experiments in a short period of time. Limits and controls need to be defined by your organization to determine acceptable evidence thresholds that will allow the team to advance to the next step.

For example, if you are building a rocket ship you may want your experiments to have a high threshold for statistical significance. If you are deciding between two different flows intended to help increase user sign up you may be happy to tolerate a lower significance threshold.

The final step is to clearly and visibly state any assumptions made about our hypothesis, to create a feedback loop for the team to provide further input, debate, and understanding of the circumstance under which we are performing the test. Are they valid and make sense from a technical and business perspective?

Hypotheses, when aligned to your MVP, can provide a testing mechanism for your product or service vision. They can test the most uncertain areas of your product or service, in order to gain information and improve confidence.

Examples of Hypothesis-Driven Development user stories are;

Business story.

We Believe That increasing the size of hotel images on the booking page Will Result In improved customer engagement and conversion We Will Have Confidence To Proceed When  we see a 5% increase in customers who review hotel images who then proceed to book in 48 hours.

It is imperative to have effective monitoring and evaluation tools in place when using an experimental approach to software development in order to measure the impact of our efforts and provide a feedback loop to the team. Otherwise, we are essentially blind to the outcomes of our efforts.

In agile software development, we define working software as the primary measure of progress. By combining Continuous Delivery and Hypothesis-Driven Development we can now define working software and validated learning as the primary measures of progress.

Ideally, we should not say we are done until we have measured the value of what is being delivered – in other words, gathered data to validate our hypothesis.

Examples of how to gather data is performing A/B Testing to test a hypothesis and measure to change in customer behavior. Alternative testings options can be customer surveys, paper prototypes, user and/or guerilla testing.

One example of a company we have worked with that uses Hypothesis-Driven Development is lastminute.com . The team formulated a hypothesis that customers are only willing to pay a max price for a hotel based on the time of day they book. Tom Klein, CEO and President of Sabre Holdings shared the story  of how they improved conversion by 400% within a week.

Combining practices such as Hypothesis-Driven Development and Continuous Delivery accelerates experimentation and amplifies validated learning. This gives us the opportunity to accelerate the rate at which we innovate while relentlessly reducing costs, leaving our competitors in the dust. Ideally, we can achieve the ideal of one-piece flow: atomic changes that enable us to identify causal relationships between the changes we make to our products and services, and their impact on key metrics.

As Kent Beck said, “Test-Driven Development is a great excuse to think about the problem before you think about the solution”. Hypothesis-Driven Development is a great opportunity to test what you think the problem is before you work on the solution.

We also run a  workshop to help teams implement Hypothesis-Driven Development . Get in touch to run it at your company. 

[1]  Hypothesis-Driven Development  By Jeffrey L. Taylor

More strategy insights

Creating new markets, negotiation made simple with dr john lowry, how high performance organizations innovate at scale, read my newsletter.

Insights in every edition. News you can use. No spam, ever. Read the latest edition

We've just sent you your first email. Go check it out!

• Explore Insights
• Nobody Studios
• LinkedIn Learning: High Performance Organizations

Advisory boards aren’t only for executives. Join the LogRocket Content Advisory Board today →

• Product Management
• Solve User-Reported Issues
• Find Issues Faster
• Optimize Conversion and Adoption

What is hypothesis-driven development?

Uncertainty is one of the biggest challenges of modern product development. Most often, there are more question marks than answers available.

This fact forces us to work in an environment of ambiguity and unpredictability.

Instead of combatting this, we should embrace the circumstances and use tools and solutions that excel in ambiguity. One of these tools is a hypothesis-driven approach to development.

Hypothesis-driven development in a nutshell

As the name suggests, hypothesis-driven development is an approach that focuses development efforts around, you guessed it, hypotheses.

To make this example more tangible, let’s compare it to two other common development approaches: feature-driven and outcome-driven.

In feature-driven development, we prioritize our work and effort based on specific features we planned and decided on upfront. The underlying goal here is predictability.

In outcome-driven development, the priorities are dictated not by specific features but by broader outcomes we want to achieve. This approach helps us maximize the value generated.

When it comes to hypothesis-driven development, the development effort is focused first and foremost on validating the most pressing hypotheses the team has. The goal is to maximize learning speed over all else.

Benefits of hypothesis-driven development

There are numerous benefits of a hypothesis-driven approach to development, but the main ones include:

Continuous learning

Mvp mindset, data-driven decision-making.

Hypothesis-driven development maximizes the amount of knowledge the team acquires with each release.

After all, if all you do is test hypotheses, each test must bring you some insight:

Hypothesis-driven development centers the whole prioritization and development process around learning.

Instead of designing specific features or focusing on big, multi-release outcomes, a hypothesis-driven approach forces you to focus on minimum viable solutions ( MVPs ).

After all, the primary thing you are aiming for is hypothesis validation. It often doesn’t require scalability, perfect user experience, and fully-fledged features.

Over 200k developers and product managers use LogRocket to create better digital experiences

By definition, hypothesis-driven development forces you to truly focus on MVPs and avoid overcomplicating.

In hypothesis-driven development, each release focuses on testing a particular assumption. That test then brings you new data points, which help you formulate and prioritize next hypotheses.

That’s truly a data-driven development loop that leaves little room for HiPPOs (the highest-paid person in the room’s opinion).

Guide to hypothesis-driven development

Let’s take a look at what hypothesis-driven development looks like in practice. On a high level, it consists of four steps:

• Formulate a list of hypotheses and assumptions
• Prioritize the list
• Design an MVP
• Test and repeat

1. Formulate hypotheses

The first step is to list all hypotheses you are interested in.

Everything you wish to know about your users and market, as well as things you believe you know but don’t have tangible evidence to support, is a form of a hypothesis.

At this stage, I’m not a big fan of robust hypotheses such as, “We believe that if <we do something> then <something will happen> because <some user action>.”

To have such robust hypotheses, you need a solid enough understanding of your users, and if you do have it, then odds are you don’t need hypothesis-driven development anymore.

Instead, I prefer simpler statements that are closer to assumptions than hypotheses, such as:

• “Our users will love the feature X”
• “The option to do X is very important for student segment”
• “Exam preparation is an important and underserved need that our users have”

2. Prioritize

The next step in hypothesis-driven development is to prioritize all assumptions and hypotheses you have. This will create your product backlog:

There are various prioritization frameworks and approaches out there, so choose whichever you prefer. I personally prioritize assumptions based on two main criteria:

• How much will we gain if we positively validate the hypothesis?
• How much will we learn during the validation process?

Your priorities, however, might differ depending on your current context.

3. Design an MVP

Hypothesis-driven development is centered around the idea of MVPs — that is, the smallest possible releases that will help you gather enough information to validate whether a given hypothesis is true.

User experience, maintainability, and product excellence are secondary.

4. Test and repeat

The last step is to launch the MVP and validate whether the actual impact and consequent user behavior validate or invalidate the initial hypothesis.

The success isn’t measured by whether the hypothesis turned out to be accurate, but by how many new insights and learnings you captured during the process.

Based on the experiment, revisit your current list of assumptions, and, if needed, adjust the priority list.

Challenges of hypothesis-driven development

Although hypothesis-driven development comes with great benefits, it’s not all wine and roses.

Let’s take a look at a few core challenges that come with a hypothesis-focused approach.

Lack of robust product experience

Focusing on validating hypotheses and underlying MVP mindset comes at a cost. Robust product experience and great UX often require polishes, optimizations, and iterations, which go against speed-focused hypothesis-driven development.

You can’t optimize for both learning and quality simultaneously.

Unfocused direction

Although hypothesis-driven development is great for gathering initial learnings, eventually, you need to start developing a focused and sustainable long-term product strategy. That’s where outcome-driven development shines.

There’s an infinite amount of explorations you can do, but at some point, you must flip the switch and narrow down your focus around particular outcomes.

Over-emphasis on MVPs

Teams that embrace a hypothesis-driven approach often fall into the trap of an “MVP only” approach. However, shipping an actual prototype is not the only way to validate an assumption or hypothesis.

You can utilize tools such as user interviews, usability tests, market research, or willingness to pay (WTP) experiments to validate most of your doubts.

There’s a thin line between being MVP-focused in development and overusing MVPs as a validation tool.

When to use hypothesis-driven development

As you’ve most likely noticed, a hypothesis-driven development isn’t a multi-tool solution that can be used in every context.

On the contrary, its challenges make it an unsuitable development strategy for many companies.

As a rule of thumb, hypothesis-driven development works best in early-stage products with a high dose of ambiguity. Focusing on hypotheses helps bring enough clarity for the product team to understand where even to focus:

But once you discover your product-market fit and have a solid idea for your long-term strategy, it’s often better to shift into more outcome-focused development. You should still optimize for learning, but it should no longer be the primary focus of your development effort.

While at it, you might also consider feature-driven development as a next step. However, that works only under particular circumstances where predictability is more important than the impact itself — for example, B2B companies delivering custom solutions for their clients or products focused on compliance.

Hypothesis-driven development can be a powerful learning-maximization tool. Its focus on MVP, continuous learning process, and inherent data-driven approach to decision-making are great tools for reducing uncertainty and discovering a path forward in ambiguous settings.

Honestly, the whole process doesn’t differ much from other development processes. The primary difference is that backlog and priories focus on hypotheses rather than features or outcomes.

Start by listing your assumptions, prioritizing them as you would any other backlog, and working your way top-to-bottom by shipping MVPs and adjusting priorities as you learn more about your market and users.

However, since hypothesis-driven development often lacks long-term cohesiveness, focus, and sustainable product experience, it’s rarely a good long-term approach to product development.

I tend to stick to outcome-driven and feature-driven approaches most of the time and resort to hypothesis-driven development if the ambiguity in a particular area is so hard that it becomes challenging to plan sensibly.

Featured image source: IconScout

LogRocket generates product insights that lead to meaningful action

Get your teams on the same page — try LogRocket today.

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Reddit (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)
• #product strategy

Stop guessing about your digital experience with LogRocket

Recent posts:.

Leader Spotlight: Developing horizon-spotting skills, with Jennifer Musser Metz

Jennifer Musser Metz, VP Product Management — Core Platforms & Expansion at Paramount+, talks about the importance of horizon spotting.

The evolving role of AI product managers

An AI product manager plays a crucial role in translating complex project requirements into products that align with user needs.

Leader Spotlight: Engaging customers authentically, with Dianna Lyngholm

Dianna Lyngholm, Director of Creative Services + CX at FUN.com, talks about strategies for creating an authentic relationship with customers.

The digital marketing funnel: Stages and strategies

The digital marketing funnel is a visual representation of the customer’s journey as it moves through online channels.

Leave a Reply Cancel reply

Search form

• Open Educational Resources
• Coursera for UVA
• Faculty Resources
• Student Resources

Hypothesis-Driven Development

To deliver agile outcomes, you have to do more than implement an agile process- you have to create focus around what matters to your user and rigorously test how well what you’re doing is delivering on that focus. Driving to testable ideas (hypotheses) and maximizing the results of your experimentation is at the heart of a high-functioning practice of agile. This course shows you how to facilitate alignment and create a culture of experimentation across your product pipeline.

You’ll understand how to answer these four big questions: 1. How do we drive our agility with analytics? 2. How do we create compelling propositions for our user?
 3. How do we achieve excellent usability?
 4. How do we release fast without creating disasters?
 As a Project Management Institute (PMI®) Registered Education Provider, the University of Virginia Darden School of Business has been approved by PMI to issue 20 professional development units (PDUs) for this course, which focuses on core competencies recognized by PMI. (Provider #2122) This course is supported by the Batten Institute at UVA’s Darden School of Business. The Batten Institute’s mission is to improve the world through entrepreneurship and innovation: www.batteninstitute.org .

Cookie Notice

This site uses cookies for performance, analytics, personalization and advertising purposes.

For more information about how we use cookies please see our Cookie Policy .

Cookie Policy   |   Privacy Policy

Manage Consent Preferences

Essential/Strictly Necessary Cookies

These cookies are essential in order to enable you to move around the website and use its features, such as accessing secure areas of the website.

Analytical/ Performance Cookies

These are analytics cookies that allow us to collect information about how visitors use a website, for instance which pages visitors go to most often, and if they get error messages from web pages. This helps us to improve the way the website works and allows us to test different ideas on the site.

Functional/ Preference Cookies

These cookies allow our website to properly function and in particular will allow you to use its more personal features.

Targeting/ Advertising Cookies

These cookies are used by third parties to build a profile of your interests and show you relevant adverts on other sites. You should check the relevant third party website for more information and how to opt out, as described below.

• Starburst vs OSS Trino

By Use Cases

• Open Data Lakehouse
• Artificial Intelligence
• ELT Data Processing
• Data Applications
• Data Migrations
• Data Products

By Industry

• Financial Services
• Healthcare & Life Sciences
• Retail & CPG
• All Industries
• Meet our Customers
• Customer Experience
• Starburst Data Rebels
• Documentation
• Technical overview
• Starburst Galaxy
• Starburst Enterprise
• Upcoming Events
• Data Universe
• Icehouse Center
• Data Fundamentals
• Starburst Orbit
• Become a Partner
• Partner Login
• Security & Trust

Fully managed in the cloud

Self-managed anywhere

Hypothesis-Driven Development

Hypothesis-driven development (hdd), also known as hypothesis-driven product development, is an approach used in software development and product management..

HDD involves creating hypotheses about user behavior, needs, or desired outcomes, and then designing and implementing experiments to validate or invalidate those hypotheses.

Related blogs

BCG study: Data costs & architectural complexity reach a tipping point

Data-driven innovation: If you want to innovate with data, this is what you should do!

Artificial intelligence life cycle

Data Products For Dummies

Unlock the value in your data

Why use a hypothesis-driven approach?

With hypothesis-driven development, instead of making assumptions and building products or features based on those assumptions, teams should formulate hypotheses and conduct experiments to gather data and insights.

This method assists with making informed decisions and reduces the overall risk of building products that do not meet user needs or solve their problems.

How do you implement hypothesis-driven development

At a high level, here’s a general approach to implementing HDD:

• Identify the problem or opportunity: Begin by identifying the problem or opportunity that you want to address with your product or feature.
• Create a hypothesis: Clearly define a hypothesis that describes a specific user behavior, need, or outcome you believe will occur if you implement the solution.
• Design an experiment: Determine the best way to test your hypothesis. This could involve creating a prototype, conducting user interviews, A/B testing, or other forms of user research.
• Implement the experiment: Execute the experiment by building the necessary components or conducting the research activities.
• Collect and analyze data: Gather data from the experiment and analyze the results to determine if the hypothesis is supported or not.
• If the hypothesis is supported, you can move forward with further development.
• If the hypothesis is not supported, you may need to pivot, refine the hypothesis, or explore alternative solutions.
• Rinse and repeat: Continuously repeat the process, iterating and refining your hypotheses and experiments to guide the development of your product or feature.

Hypothesis-driven development emphasizes a data-driven and iterative approach to product development, allowing teams to make more informed decisions, validate assumptions, and ultimately deliver products that better meet user needs.

A single point of access to all your data

Stay in the know - sign up for our newsletter.

• Resource Library
• Events and Webinars
• Open-source Trino

Quick Links

Get in touch.

• Customer Support

© Starburst Data, Inc. Starburst and Starburst Data are registered trademarks of Starburst Data, Inc. All rights reserved. Presto®, the Presto logo, Delta Lake, and the Delta Lake logo are trademarks of LF Projects, LLC

Privacy Policy   |   Legal Terms   |   Cookie Notice

Start Free with Starburst Galaxy

Up to $500 in usage credits included

• Query your data lake fast with Starburst's best-in-class MPP SQL query engine
• Get up and running in less than 5 minutes

For more deployment options:

Please fill in all required fields and ensure you are using a valid email address.

By clicking Create Account , you agree to Starburst Galaxy's terms of service and privacy policy .

An Explanation of Hypothesis-Driven Development

In this Scrum Tapas video, PST Martin Hinshelwood delves into the Lean idea of Hypothesis Driven Development and explains how it works when it comes to delivering value.

What did you think about this content?

The 6 Steps that We Use for Hypothesis-Driven Development

One of the greatest fears of product managers is to create an app that flopped because it's based on untested assumptions. After successfully launching more than 20 products, we're convinced that we've found the right approach for hypothesis-driven development.

In this guide, I'll show you how we validated the hypotheses to ensure that the apps met the users' expectations and needs.

What is hypothesis-driven development?

Hypothesis-driven development is a prototype methodology that allows product designers to develop, test, and rebuild a product until it’s acceptable by the users. It is an iterative measure that explores assumptions defined during the project and attempts to validate it with users’ feedbacks.

What you have assumed during the initial stage of development may not be valid for the users. Even if they are backed by historical data, user behaviors can be affected by specific audiences and other factors. Hypothesis-driven development removes these uncertainties as the project progresses. 

Why we use hypothesis-driven development

For us, the hypothesis-driven approach provides a structured way to consolidate ideas and build hypotheses based on objective criteria. It’s also less costly to test the prototype before production.

Using this approach has reliably allowed us to identify what, how, and in which order should the testing be done. It gives us a deep understanding of how we prioritise the features, how it’s connected to the business goals and desired user outcomes.

We’re also able to track and compare the desired and real outcomes of developing the features. 

The process of Prototype Development that we use

Our success in building apps that are well-accepted by users is based on the Lean UX definition of hypothesis. We believe that the business outcome will be achieved if the user’s outcome is fulfilled for the particular feature. 

Here’s the process flow:

How Might We technique → Dot voting (based on estimated/assumptive impact) → converting into a hypothesis → define testing methodology (research method + success/fail criteria) → impact effort scale for prioritizing → test, learn, repeat.

Once the hypothesis is proven right, the feature is escalated into the development track for UI design and development. 

Step 1: List Down Questions And Assumptions

Whether it’s the initial stage of the project or after the launch, there are always uncertainties or ideas to further improve the existing product. In order to move forward, you’ll need to turn the ideas into structured hypotheses where they can be tested prior to production.  

To start with, jot the ideas or assumptions down on paper or a sticky note. 

Then, you’ll want to widen the scope of the questions and assumptions into possible solutions. The How Might We (HMW) technique is handy in rephrasing the statements into questions that facilitate brainstorming.

For example, if you have a social media app with a low number of users, asking, “How might we increase the number of users for the app?” makes brainstorming easier. 

Step 2: Dot Vote to Prioritize Questions and Assumptions

Once you’ve got a list of questions, it’s time to decide which are potentially more impactful for the product. The Dot Vote method, where team members are given dots to place on the questions, helps prioritize the questions and assumptions. 

Our team uses this method when we’re faced with many ideas and need to eliminate some of them. We started by grouping similar ideas and use 3-5 dots to vote. At the end of the process, we’ll have the preliminary data on the possible impact and our team’s interest in developing certain features. 

This method allows us to prioritize the statements derived from the HMW technique and we’re only converting the top ones. 

Step 3: Develop Hypotheses from Questions

The questions lead to a brainstorming session where the answers become hypotheses for the product. The hypothesis is meant to create a framework that allows the questions and solutions to be defined clearly for validation.

Our team followed a specific format in forming hypotheses. We structured the statement as follow:

We believe we will achieve [ business outcome], 

If [ the persona],

Solve their need in  [ user outcome] using [feature]. ‍

Here’s a hypothesis we’ve created:

We believe we will achieve DAU=100 if Mike (our proto persona) solve their need in recording and sharing videos instantaneously using our camera and cloud storage .

Step 4: Test the Hypothesis with an Experiment

It’s crucial to validate each of the assumptions made on the product features. Based on the hypotheses, experiments in the form of interviews, surveys, usability testing, and so forth are created to determine if the assumptions are aligned with reality. 

Each of the methods provides some level of confidence. Therefore, you don’t want to be 100% reliant on a particular method as it’s based on a sample of users.

It’s important to choose a research method that allows validation to be done with minimal effort. Even though hypotheses validation provides a degree of confidence, not all assumptions can be tested and there could be a margin of error in data obtained as the test is conducted on a sample of people. 

The experiments are designed in such a way that feedback can be compared with the predicted outcome. Only validated hypotheses are brought forward for development.

Testing all the hypotheses can be tedious. To be more efficient, you can use the impact effort scale. This method allows you to focus on hypotheses that are potentially high value and easy to validate. 

You can also work on hypotheses that deliver high impact but require high effort. Ignore those that require high impact but low impact and keep hypotheses with low impact and effort into the backlog. 

At Uptech, we assign each hypothesis with clear testing criteria. We rank the hypothesis with a binary ‘task success’ and subjective ‘effort on task’ where the latter is scored from 1 to 10. 

While we’re conducting the test, we also collect qualitative data such as the users' feedback. We have a habit of segregation the feedback into pros, cons and neutral with color-coded stickers.  (red - cons, green -pros, blue- neutral).

The best practice is to test each hypothesis at least on 5 users. 

Step 5  Learn, Build (and Repeat)

The hypothesis-driven approach is not a single-ended process. Often, you’ll find that some of the hypotheses are proven to be false. Rather than be disheartened, you should use the data gathered to finetune the hypothesis and design a better experiment in the next phase.

Treat the entire cycle as a learning process where you’ll better understand the product and the customers. 

We’ve found the process helpful when developing an MVP for Carbon Club, an environmental startup in the UK. The app allows users to donate to charity based on the carbon-footprint produced. 

In order to calculate the carbon footprint, we’re weighing the options of

• Connecting the app to the users’ bank account to monitor the carbon footprint based on purchases made.
• Allowing users to take quizzes on their lifestyles.

Upon validation, we’ve found that all of the users opted for the second option as they are concerned about linking an unknown app to their banking account. 

The result makes us shelves the first assumption we’ve made during pre-Sprint research. It also saves our client $50,000, and a few months of work as connecting the app to the bank account requires a huge effort. 

Step 6: Implement Product and Maintain

Once you’ve got the confidence that the remaining hypotheses are validated, it’s time to develop the product. However, testing must be continued even after the product is launched. 

You should be on your toes as customers’ demands, market trends, local economics, and other conditions may require some features to evolve. 

Our takeaways for hypothesis-driven development

If there’s anything that you could pick from our experience, it’s these 5 points.

1. Should every idea go straight into the backlog? No, unless they are validated with substantial evidence. 

2. While it’s hard to define business outcomes with specific metrics and desired values, you should do it anyway. Try to be as specific as possible, and avoid general terms. Give your best effort and adjust as you receive new data.  

3. Get all product teams involved as the best ideas are born from collaboration.

4. Start with a plan consists of 2 main parameters, i.e., criteria of success and research methods. Besides qualitative insights, you need to set objective criteria to determine if a test is successful. Use the Test Card to validate the assumptions strategically. 

5. The methodology that we’ve recommended in this article works not only for products. We’ve applied it at the end of 2019 for setting the strategic goals of the company and end up with robust results, engaged and aligned team.

You'll have a better idea of which features would lead to a successful product with hypothesis-driven development. Rather than vague assumptions, the consolidated data from users will provide a clear direction for your development team. 

As for the hypotheses that don't make the cut, improvise, re-test, and leverage for future upgrades.

Keep failing with product launches? I'll be happy to point you in the right direction. Drop me a message here.

Tell us about your idea. We will reach you out.

Mobile Menu

• Skip to primary navigation
• Skip to main content
• Skip to primary sidebar
• Skip to footer

HDD & More from Me

Reference for ‘Hypothesis-Driven Development (Book)

Table of Contents

About the Book

Chapter 1: you, and the business of ‘digital’, chapter 2: from idea to design, chapter 3: from design to code, chapter 4: from code to deploy, chapter 5: from release to experimentation, chapter 6: from inference to your next product priorities, chapter 7: the economics of code, chapter 8: application infrastructure, chapter 9: data, big data, data science & machine learning, chapter 10: security, chapter 11: growth hacking & revops, chapter 12: you, the hypothesis-driven developer.

Hello! This is a reference for the book ‘ Hypothesis-Driven Development: A Guide to Smarter Product Management “.

But waste is not inevitable anymore.

Hypothesis-Driven Development (HDD) is an emerging approach to digital product management for both business people and engineers. It emphasizes rigorous, continuous experimentation as a way to both minimize waste and focus teams’ creative capabilities in directions that drive growth and innovation.

The sections that follow detail the recommended practice for each chapter of the book . If you’re reading the book, I’ll just mention again that the idea is not to go through all off this practice each time you finish a chapter. Instead, what I’d recommend is finishing the book so you get the larger picture of how you might apply HDD to your work and then consider, week to week, month to month, where you might find the most relevant opportunities for practice, using this guide.

How do you prepare yourself for a successful career in tech?

After this chapter, you will be able to:

• Explain the key operating foundations of tech and how it might relate to a given career trajectory
• Analyze the economic significance and performance criteria of a technical team in terms of a product pipeline
• Apply a disciplined, focused, yet innovation-friendly view of strategy to a tech product or company

The following subsections describe the recommended practice from Chapter 1.

Describe a Business and its Product/Market fit with the Business Model Canvas

Tutorial on the Business Model Canvas Printable Business Model Canvas Digitally Editable Canvas Template on Google Docs

For more depth on the topic, check out this online course: Online Course: Facilitating with the Business Model Canvas (≈4 hours including quizzes and practice)

Charter and Focus a Team with OKRs

John Doerr’s website has a set of examples I’ve found quite sufficient for getting started with OKR’s: What Matters: OKR Examples .

If you want a lot more depth, he also has a book on the topic: Measure What Matters .

Practice Agile

Online Tutorial: Agile: Just the Basics

If you prefer something more business school, here is a tech note I wrote with a colleague: Agile Development

Finally, if you want something more directed and even broader because knowing about agile is one of your top priorities right now, here is my online course: Managing with Agile .

Strategy Meets Digital

For this, I highly recommend my colleague Mike Lenox’s book on digital transformation (coming soon) or his online course ‘ Digital Transformation ‘.

Teaching a Degree Program Course or Workshop with This Material

The links below reference full syllabi, including assignment templates, for a few HDD-related classes I teach: Digital Product Management Software Design Software Development Digital Capstone Hypothesis-Driven Development for Analysts [COMING SOON]

As the business lead on a tech team, what is your role and how do you do it well?

• Explain the concept of product/market fit in economic terms
• Analyze the work of a business or product lead in terms of a sequence of testable hypotheses that minimize waste and maximize wins
• Do the work to make sure your team is focused on a job/problem/need/desire that actually exists for a certain identifiable buyer or user.
• Avoid false positives and minimize waste by applying Lean Startup to testing whether your proposition is competitive with the prevailing alternatives
• Make a habit of designing for and testing usability early and often so you minimize friction for the user
• Identify the analytics necessary to make crisp decisions for an iterative, emergent design

The following subsections describe the recommended practice from Chapter 2.

Build Your Personal Innovation Portfolio

This tutorial is a good place to start: Creating Your Personal Innovation Portfolio . It offers example entries ( like this one ) and templates in Google Slides you can use to jump start the process.

The page offers examples and starter templates for project types like customer discovery, application design, application development, and data science.

Focus and Test a Right Problem (Persona & JTBD) Hypothesis with Subject Interviews

This template is a good place to start: HDD Template/Personas, JTBD, and Discoveries . A good batch of steps is to: a) draft personas & JTBD, b) draft an interview guide and conduct interviews, c) revise your draft personas and JTBD.

‘Day in the Life’ is another related tool, particularly useful for bringing your discovery work to life for colleagues who weren’t out there with you. This tutorial offers notes, examples, and a template: Day in the Life Tutorial .

Focus And Test A Demand Hypothesis With Lean Startup

This tutorial is an excellent place to start: Tutorial on Proposition Testing with Lean Startup . It links to a template, which is a section in the same HDD Template I mentioned above (in the section on Right Problem Hypothesis).

Focus and Test a Usability Hypothesis with User Stories

This tutorial is an excellent place to start: Tutorial on Anchoring a Usability Hypothesis with User Stories . Here again, the HDD Template above also offers a place to organize and integrate this material with your work on ‘right problem’ and demand/proposition testing.

Define and Apply Visual Consistency

This tutorial offers a step by step view on how to do this: Tutorial on Creating Visual Consistency with a Style Guide . It also links to a set of example guides (ex: COWAN+ ) whose format you can use as a starting point.

Learn More about the Practice of Design and Design Thinking

For more on the core practice of product design, you may be interested in Donald Norman’s book The Design of Everyday Things , which is heavily cited in my HDD book.

For more on the practice of design thinking, applying the lens of product design to a wider set of business problems, I highly recommend The Designing for Growth Field Book .

What does it take to go from design to code? Do you need to learn to code? If so, what does that mean? When and why does it matter?

• Unpack digital infrastructure in terms of the model-view-controller (MVC) framework
• Identify and practice the key steps to creating a View
• Analyze and decompose a user experience in terms of its Controllers or algorithms
• Structure real world entities in terms of an implementation-friendly Model

The following subsections describe the recommended practice from Chapter 3. This sub-sections lean pretty heavily on the MVC (model-view-controller) framework. For a review on that, check out this 5 minute video ( MVC Tutorial ).

Create a Style Guide to Delivery Consistent Visceral Reactions to Your Views

If you already did this in the recommended practice for Chapter 2, you’re probably in good shape. However, if you’re getting ready to go from design to code on something, this is a useful thing to do now- even if it’s just a text file with some notes on which colors and typefaces you’re going to use. The tutorial here will help you get started: Tutorial on Creating Visual Consistency with a Style Guide .

Create Views with HTML & CSS

The #1 most important thing for this practice and the practice in the sub-sections that follow has nothing to do with coding itself. The most important thing is to have an idea that you’ve thought through. This could be a whole application or even just a simple interaction you’ve noticed and would like to try improving. The reason this is so, so important is that without some kind of focal point for what you want to have happen, you’ll likely get lost in all the hooks and dials and technical minutiae of the various coding facilities. This is a bad thing. Even professional developers don’t go through and memorize everything a given programming language can do. They figure out what they want to have happen first, and then figure out how to make it happen with the coding facility in question.

Yes, there are some fundamentals for each programming language and there’s an experience curve across which going from design to code gets easier. The various options below (online courses, cases, and tutorials) all focus on introducing fundamentals and then quickly transitioning you to a specific problem/task for applied practice. You’ll need to use references (aka Google search) to figure out individual items, like ‘How do I make a rounded border with CSS?’. However, what you should not do is first try to memorize all the hundreds of things that HTML or CSS can do. All that said, here are a few ways to get started with creating Views.

If you like to dive in and just start fiddling, you may want to start with the various case challenges, which you can find here: Coding Cases (Design to View) . The first three cases (From Prototype to HTML, Making HTML Manageable, and Debugging HTML & CSS) will give you a solid footing in going from design to code in this area. From there, you can use the same development environment to start working on your own project.

If you’d like something more structured and stepwise, I’ve created a set of online courses around those same cases and some of the related reading. The first one, Coding for Designers, Managers, and Entrepreneurs I , will take you through the cases above.

Finally, if you’re interested in teaching a class or even just maybe a peer support group (kind of like a book club for product people- but for coding!), then you may want to check out the syllabus I use for the degree program course in this area that I teach: Software Development . The first five sessions deal with creating Views with HTML & CSS.

Creating View Interactions with Javascript

Somewhere between the View and the Controller (words are faulty instruments), there’s the ‘front controller’, basically logic that creates interactive Views- views that afford more dynamic responses to user input for example.

If you want to dive in and start fiddling, check out this case: Making Stuff Happen with Javascript , and then invest some time on refining your skills with analytical debugging with this case: Debugging Javascript .

If you want something more step by step, check out the second course in the series above ( Coding for Designers, Managers, and Entrepreneurs II ).

Finally, if you’re looking to teach or facilitate a class/study group, sessions 7 & 8 from the Software Development syllabus in the section above focus on this aspect of going from design to code.

Creating Controllers with Javascript

Here you’ll transition more squarely to going from design to code with Controllers or ‘algorithms’.

If you want to dive in and start fiddling, this case deals with process automation and data transformation (super fun): Automating Your Gruntwork with Javascript .

Course II in the online course series (see above) offers a more step by step approach and session 9 from the syllabus above is where you’d do this case.

Mapping Data to Models and Operationalizing Them

Finally, putting the ‘M’ in MVC, we have a case on going from design to code with a data model. Rather straight away diving into all the mess and complexity of databases, the material here instead uses a modern NoSQL approach by way of Google’s Firebase ‘backend as a service’.

If you want to dive in and start fiddling, the case is: Creating & Managing Users with Google Firebase .

Course III in the course series above covers this case and the related concepts, as does session 10 in the syllabus (see above).

How do you rapidly get new features in front of customers without stressing your team? What investments does this require and how do you evaluate them?

• Explain the major catalysts for change and success criteria for a modern, continuous product pipeline
• Unpack the steps from test to deploy for a given application
• Explain the role of version control in a product pipeline
• Analyze and evaluate investments in test capabilities, automated or otherwise
• Analyze and evaluate investments in deploy capabilities

The following subsections describe the recommended practice from Chapter 4.

Sketch Your Team’s Product Pipeline

The basic idea here is that you can’t improve a pipeline that you don’t (as a team) understand. The objective of this workshop/meeting is primarily just to sketch out an answer to ‘What is?’. Most likely, a set of ideas about where the roughest spots are and what changes you might want to test will arise from that naturally. Here are some notes on how you might do that with your team.

Who should be there? Ideally, everyone on your team and, if the team’s not dedicated, then also all colleagues you regularly work with across the pipeline, including, say- test, sysadmin/ops, and analytics. Is that absolutely necessary? No, not to get started. It’s better to start texturing out the pipeline with a smaller group than to delay. But ultimately you’ll be best off with a fuller group for identifying, prioritizing, and testing changes.

What’s the agenda? Have a whiteboard (physical or virtual) and bracket the pipeline as we’ve done here where it starts with ‘idea’ and finishes with ‘released software’. You may want to use one of the pipeline diagrams from the book/this site to introduce the general idea, but the terms your team or company uses may be different and that’s OK- the idea here is just to describe ‘What is?’.

From there, the idea is to fill in what happens between idea and released software. I highly recommend coming in with 1-2 specific examples, features that you’ve already released, and using those to anchor and the discussion and get all the way through the process. The current process may have forks, etc.- sometimes you do one thing, sometimes the other. I would get those both up there as messy as it may be- the idea here is to get to what is, and then, in small, success-based batches, test your way to what you all think should be.

Then what? Getting a bunch of ideas on the board about which parts of the pipeline you want to improve is usually not too hard. You can simply right them on the board, take notes, or, have individuals write ideas on post-it’s (physical or virtual) and post them on the wall (again, physical or virtual). Converging on a prioritized set of choices is usually a little harder. Practices like dot voting can make that a little easier and more collaborative. However, the big thing to focus on is that you’ve got to start somewhere, each of these ideas has a specific, observable (ideally measurable) outcome that you’re testing it against, and you’ll by doing just then in your next or future team retrospective meeting after you’ve had a chance to test these ideas for improvement.

Concept and Prioritize Testable Changes to Your Product Pipeline

(see notes on the subsection above)

Draft an Agile Team Charter

The fundamental idea here is to have an entry point that answer the questions “What is this team working on and why?” as well as “How is the team currently collaborating?”. Does that mean everything that answers these questions needs to reside in the team charter itself? No. For example, you may have seen the HDD Template in some of the sections above, and it has some overlap with the team charter template. That’s fine. The core thing with the charter is to have an entry point. So, for example, if your team is happily keeping notes on its personas, JTBD, customer interviews, or other evidence someplace else just link to that from your charter. If you all keep meeting notes from your agile retrospectives someplace else, same idea- just link to it from your charter.

How do you know if it’s working? I’d look at two things. First, how often do various members of the team visit it? If it’s a Google Doc you have ready access to this, for example. Second, how well does it help onboarding new employees or team members?

All that said, this tutorial is a good place to start: Tutorial on Agile Team Charters . If you just want to get started, here is the template (just make yourself a copy): Team Charter Template .

Explain the Fundamentals of DevOps and Continuous Delivery

If you’re all in on the DORA metrics and generally like learning by book, Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim is a great place to go next.

If you might prefer online courses, well, then I can’t help but recommend my own masterwork: Continuous Delivery & DevOps (Coursera) .

Learn How to Manage Work with Version Control

On this one in particular, I would not invest a lot of time unless either a) you’ll be interacting with version control at least once in awhile or b) you just want to try it out and you’re OK forgetting most of the particulars if you don’t use it for, say, six months.

If you’re ready to give in, Github themselves (the leading VCS) offer a pretty good written tutorial: Hello World- Github .

If you want something more step by step and explanatory, Codecademy offers a good (free) online tutorial with hands-on practice: Codecademy Micro Course on Github .

Write Some Test to See How it Works

The Coursera course I mentioned above (Continuous Delivery & DevOps) steps through specific test development across unit, integration, and system tests.

If you want something more immediately hands-on, Codecademy offers tutorials here as well (though this one is paid only)- for example, this one is on unit testing Javascript: Unit Testing with JS .

Finally, it’s worth noting here that across all the different programming languages and the different layers of the test pyramid, there is a lot of stuff out there. If you’re curious about how testing happens in your current work, I would find out what toolchain your team uses and find resources to match- there will be plenty, you just have to know what you’re looking for. If you want a more general introduction that’s more in-depth than what you saw in Chapter 4, I would consider the online course:  Continuous Delivery & DevOps (Coursera) .

Understand the Managerial Side of How a Team gets to Healthier Pipelines

Full disclosure: this is currently a placeholder for something I’m working with some collaborators. I think it’s going to be useful if a general management perspective is what you’re after, but it’s not quite ready. That said, I think the items above are an excellent way to start on this.

How do you make a habit out of experimentation? How do you make it useful enough for the team and easy enough to consistently do as you release?

• Explain the fundamentals of experimentation in digital
• Pair important questions with rigorous but practical experiment designs
• Make a habit of pairing focused, relevant analytical questions with your agile user stories for purposeful analytics

The following subsections describe the recommended practice from Chapter 5.

Practice and Apply Innovation Analytics to Your work

As I mentioned in the book, my #1 recommendation here is to go deep on using experiments in your current work- whatever analytics suite you have, etc., it doesn’t really matter that much. Relevance is the big thing. I’d start by figuring out what questions you want to answer, how you might answer them (tools aside), and then figuring it yourself or working with colleagues, go get yourself some answers!

That said, if you want more depth on general practice, I recommend my own super great online course: Agile Analytics . I’m a biased? Yes. But, I did build the course specifically around the various focal points you’ve seen in the book so, for my money, it’s a good place to start.

That said, there are a lot of really good items out there. Stephen Thomke’s book Experimentation Works is an excellent intro to how companies are changing the way they operate with experiments. Trustworthy Online Controlled Experiments by Kohavi, et all is a more clinical take and widely read.

Design & Conduct an Experiment

#1 tip for getting started with experiments? Find something that interest you and an idea to make it better. Then test it. This section of the HDD template: Experiment Design is a good place to start.

The only other thing worth mentioning is that I would make sure you have a point of view on what general type of hypothesis you want to test. If it’s a motivation/demand hypothesis, look for relevant MVP patterns and make sure you’re not confusing it with a hypothesis about usability. Likewise, if it’s a hypothesis about usability, make sure you’re testing the subject’s ability to achieve a goal assuming motivation. Finally, if it’s a simple test in the wild (released software) of user action vs. inaction, that’s also fine- just make sure that’s clear in your hypothesis formulation.

Understand Frequentist vs. Bayesian Statistical Reasoning

Beyond what we’ve covered in the chapter, Google has done some excellent work in support of their Optimize tool. For more depth, this post is a good place to start: General Methodology . For frequentist statistics, just about any leading stat’s primer (which are the only ones you’ll find) is fine. For more on Bayesian inference, this is good as a follow-on from the Google piece: Bayesian Inference .

How do you know if you got it right, or you need to rinse and repeat? How do you instrument observation and set thresholds to make the hard but necessary decisions about what to prioritize?

• Start from a set of objectives and key results to charter a teams ‘true north’ success criteria
• Work backward from tangible goals in your descriptive analytics
• Think of your experiment designs as part of a longer game of adaptively getting to product/market fit

The following subsections describe the recommended practice from Chapter 6.

Sketch a CX Map

Here again, the main thing is just to pick something familiar with (a specific value proposition relative to a specific job-to-be-done) and start sketching. This tutorial offers a quick reference: CX Mapping , and you can start sketch on this section ( CX Map ) of the HDD template.

Take Your Metrics and Make them Better

My general preference is to start with the CX Map, but what if you already have a measurement infrastructure you generally like? The CX Map certainly is not the only way to go. This subsection of that tutorial offers a more general point of view on how to assess and (where applicable) improve your metrics from an HDD perspective: Making Better Metrics .

What drives the cost of releasing a successful feature? And how does that relate to the technology choices a team makes when building an application?

• Explain the major functional and economic dimensions between programming languages, libraries, and frameworks
• Identify and analyze the functional and economic footprint of application frameworks and development
• Analyze the economics of monoliths vs. microservices relative to an application’s key economic drivers

The following subsections describe the recommended practice from Chapter 7.

Learn about Your Product’s Stack or Stacks

Find the right person and start with a simple question like “What’s our tech stack?” or “What’s our tech stack for [specific application]?” More questions may not be necessary, but you can always follow that up with “How is that stack for you? How did you decide on it?” Be sure to express plain curiosity and you’ll almost certainly learn all you want and more!

Where do applications run? Given the pain points of deploying applications and keeping them running, what is the marketplace for approaches to do this? When, why, and how are those economical?

• Explain the fundamentals of the stack where your application runs and its key outside interfaces
• Unpack the key cost drivers for operating your application and evaluate alternative solutions like data centers vs. cloud vs. platforms-as-a-service
• Explain the relationship between data exchanged through API’s or data-interchange formats vs. databases

The following subsections describe the recommended practice from Chapter 8.

Learn more about Technology, Economics, and Design Interact in Application Infrastructure

If you like to learn through case studies, as we do at UVA Darden, this is a case study on retrieving a large scale IT project: Saving Griffin . That page also has a teaching note for the case and a tech note on agile- the tech note is a subset of the material in this book (not surprisingly), but if you’re doing an introduction for others you might find it useful.

Get Hands-On Practice Coding with API’s

This case will give you hands-on experience interacting with the API for Google Firebase, a very substantial backend-as-a-service platform: Creating & Managing Users with Google Firebase . It requires some familiarity with Javascript- but you can get that over the course of a few cases which I mentioned up in the materials for Chapter 3.

Deploy a Data Science Model to the Web

While it requires some familiarity with Javascript (for the client application), Python, and building predictive models with Python, it is amazingly easy to set up your own functioning application with multiple services (microservices, if you will). This ‘cookbook’ is a working item in Google Docs and will help you get started: Notes on Deploying Your Model with Flask . Finally, it’s worth noting that Flask itself is quite popular and well liked and you’ll find lots of other great resources with a few Google searches.

What can you do with a lot of data? How do you decide when, where, and how to invest? How do you integrate such a capability with your general product program to maximize its relevance?

• Analyze how and where to store your data to maximize your economics
• Frame your customer discovery in terms of ground truth and link it to dependent variables for actionability
• Identify and manage data science charters across descriptive, diagnostic, predictive, and prescriptive analytics
• Understand the data science process from data collection, preparation, exploration, modeling, and communication
• Identify key data science skill sets, including methods and statistical literacy, mechanics & algorithms, data intuition, and communication

The following subsections describe the recommended practice from Chapter 9.

Charter a Big Data Program with the Data Innovation Canvas

Data may be the new oil in a certain sense, but not all data is valuable. The Data Innovation Canvas is a thorough but lightweight tool for thinking through which data might end up valuable for you, both in terms of data you have and in terms of data you can create. This template on Google Slides (which you can download for PowerPoint) is a good way to get started: Data Innovation Canvas .

Get Hands-On Practice with Data

If SQL literacy is a big thing for you on the job, I recommend this relatively short (and free) tutorial from Codecademy: Learn SQL . The main thing is not to spend too much time on details ahead of meaningful, active practice either on the job or on a specific side project with some relevance to you.

If you want a more expansive introduction to data science, I like Jose Portilla’s course on Udemy: Python for Data Science and Machine Learning Bootcamp . There are plenty of resources, though I would avoid those with the language R and find ones that use Python, which is more generally applicable and ascendant. Here too, I’d look to go through the course relatively quickly and not worry about everything with an eye to getting started with active practice on a specific project.

Where’s the insecurity in digital? It’s pretty much everywhere, but how do you identify, prioritize, and manage risk?

• Frame vulnerabilities across human and machine attack surfaces
• Understand several of the most prevalent attack vectors and publicly available data sources to identify them
• Identify how to link security with the rest of your product pipeline

Please Note: This chapter is terrifying and you should only read it if you want to know the truth.

The following subsections describe the recommended practice from Chapter 10.

Take Your Security Team Out for a Drink

Just do it. They need it. You’ll learn everything you wanted to know and more and you’ll never look at your computer the same way.

How has the move from physical products and print media to digital changed the way we test what will engage a customer?

• Charter the activities of a digital marketing/promotion/growth hacking team based on your understanding of product/market fit
• Understand the relationship between and strengths of organic and paid channels online
• Participate in the design, execution, and evaluation of growth experiments

The following subsections describe the recommended practice from Chapter 11.

Sketch a CX Through the Hooked Framework

In my experience, the best thing is just to put pen to paper for a product you know. And don’t assume that the Hook doesn’t apply to enterprise/B2B products. Here’s a tutorial with some examples: Hook Framework .

Frame Your Point of View on Growth using the Growth Hacking Canvas

A lot of marketing is done with the age-old justification that “we’ve always done it this way” with the supporting evidence that things aren’t that bad (yet). This isn’t necessarily the best way to reduce waste and drive new, organic growth, though. Here’s a tutorial that steps through the Canvas: Tutorial on Scaling P/M Fit with the Growth Hacking Canvas . Or, if you just want to get started, here’s a template on Google Slides: Template- Growth Hacking Canvas .

Think through and Describe Your Brand Personality with a Moodboard

This is a great way to both think through your brand personality and connect it with a relevant visual execution: Brand Lattice (Moodboard Tool) .

Facilitate Consistency with a Style Guide

OK, OK, this is my third mention, but if you don’t have one, this is definitely something I’d create (see above).

What are your next steps?

• Prioritize and charter your own professional development based on your particular focus and current activities
• Facilitate HDD-friendly team charters to help your team improve its practice of agile through a focus on testable outcomes
• Facilitate clear, testable points of view on business model design to help align the work of teams
• For larger companies, charter, focus, and steward and innovation program with testable charters and governance

The following subsections describe the recommended practice from Chapter 12.

Summary Reference for the Book

(this page is the summary reference)

Create a Personal Innovation Portfolio (Recap)

This is a great way to both focus and showcase your practice of HDD: see above on Innovation Portfolio .

Sketch a Business Model Design (Recap)

If you want to make the product or line of business you’re working on easier to align with, this is a great place to start. See above on the Business Model Canvas .

Draft an Agile Team Charter (Recap)

If you want to facilitate aligned, autonomous, hypothesis-driven executions within a team, this is a great place to start. See above on Agile Team Charters .

Charter a Company Innovation Strategy with the Corporate Innovation Canvas

If you’re at a larger firm with multiple lines of business and outside investments, the Corporate Innovation Canvas is a quick and easy way to think about how all that is cohering (or not!), where you’d like to focus, and how you’ll evaluate success: Tutorial on Innovation Strategy with the Corporate Innovation Canvas .

Copyright © 2022 Alex Cowan · All rights reserved.