‘Build, Measure, Learn’ doesn’t mean throwing things against the wall to see if they stick

I am always surprised when critics complain that the Lean Startup’s Build, Measure, Learn approach is nothing more than “throwing incomplete products out of the building to see if they work.”

Unfortunately the Build, Measure, Learn diagram is the cause of that confusion. At first glance it seems like a fire-ready-aim process.

It’s time to update Build, Measure, Learn to what we now know is the best way to build Lean startups.

Here’s how.

Build, Measure, Learn sounds pretty simple. Build a product, get it into the real world, measure customers’ reactions and behaviors, learn from this, and use what you’ve learned to build something better. Repeat, learning whether to iterate, pivot, or restart until you have something that customers love.

While it sounds simple, the Build, Measure, Learn approach to product development is a radical improvement over the traditional Waterfall model used throughout the 20th century to build and ship products. Back then, an entrepreneur used a serial product development process that proceeded step-by-step with little if any customer feedback. Founders assumed they understood customer problems/needs, wrote engineering requirements documents, designed the product, implemented/built the hardware/software, verified that it worked by testing it, and then introduced the product to customers in a formal coming out called first customer ship.

Waterfall Development was all about execution of the requirements document. While early versions of the product were shared with customers in alpha and beta testing, the goal of early customer access to the product was to uncover bugs not provide feedback on features or usability. Only after shipping and attempting to sell the product would a startup hear any substantive feedback from customers. And too often, after months or even years of development, entrepreneurs learned the hard way that customers were not buying their product because they did not need or want most of its features.

It often took companies three tries to get products right. Version 1 was built without customer feedback, and before version 1 was complete, work had already started on version 2 so it took till version 3 before the customer was really heard (e.g. Microsoft Windows 3.0).

Best practices in software development started to move to Agile Development in the early 2000s. This methodology improved on Waterfall by building software iteratively and involving the customer. But it lacked a framework for testing all commercialization hypotheses outside of the building. With Agile you could end up satisfying every feature a customer asked for and still go out of business.

Then came the Build, Measure, Learn focus of the Lean Startup.

The goal of Build, Measure, Learn is not to build a final product to ship or even to build a prototype of a product, but to maximize learning through incremental and iterative engineering. (Learning could be about product features, customer needs, the right pricing and distribution channel, etc.) The “build” step refers to building a minimal viable product (an MVP). It’s critical to understand that an MVP is not the product with fewer features. Rather it is the simplest thing that you can show to customers to get the most learning at that point in time. Early on in a startup, an MVP could simply be a PowerPoint slide, wireframe, clay model, sample data set, etc. Each time you build an MVP you also define what you are trying to test/measure. Later, as more is learned, the MVPs go from low-fidelity to higher fidelity, but the goal continues to be to maximize learning not to build a beta/fully featured prototype of the product.

A major improvement over Waterfall development, Build, Measure, Learn lets startups be fast, agile, and efficient.

The three-circle diagram of Build, Measure, Learn is a good approximation of the process. Unfortunately, using the word “build” first often confuses people. The diagram does seem to imply build stuff and throw it out of the building. A more detailed version of the Build, Measure, Learn diagram helps to clarify the meaning by adding three more elements: Ideas-Build-Code-Measure-Data-Learn.

The five-part version of the Build, Measure, Learn diagram helps us see that the real intent of building is to test ideas, not just build blindly without an objective. The circle labeled “code” could easily be labeled “build hardware” or “build artificial genome.” The circle labeled “data” indicates that after we measure our experiments we’ll use the data to further refine our learning. And the new learning will influence our next ideas. So we can see that the goal of Build, Measure, Learn isn’t just to build things, the goal is to build things to validate or invalidate the initial idea.

The focus on testing specific ideas counters the concern that Build, Measure, Learn is just throwing things against the wall to see if they work.

But it’s still not good enough. We can now do better.

Start with Hypotheses

What Build, Measure, Learn misses is that new ventures (both startups and new ideas in existing companies) don’t start with “ideas,” they start with hypotheses. It’s important to understand that the words “idea” and “hypothesis” mean two very different things. For most innovators the word “idea” conjures up an insight that immediately requires a plan to bring it to fruition. In contrast, a hypothesis means we have an educated guess that requires experimentation and data to validate or invalidate.

These hypotheses span the gamut from who’s the customer(s), to what’s the value proposition (product/service features), pricing, distribution channel, and demand creation (customer acquisition, activation, retention, etc.)

That the Lean Startup begins with acknowledging that your idea is simply a series of untested hypotheses is a big idea. It’s a really big idea because what you build needs to match the hypothesis you want to test.

The minimum viable product you’ll need to build to find the right customers is different from the minimum viable product you need for testing pricing, which is different from an MVP you would build to test specific product features. And all of these hypotheses (and minimal viable products) change over time as you learn more. So instead of Build, Measure, Learn, the diagram for building minimal viable products in a Lean Startup looks like Hypotheses – Experiments – Tests – Insights.

Generating Hypotheses

Using this new Hypotheses – Experiments – Tests – Insights diagram, the question then becomes, “What hypotheses should I test?” Luckily, Alexander Osterwalder’s Business Model Canvas presents a visual overview of the nine components of a business on one page. They are:

• value proposition, product/service the company offers (along with its benefits to customers)
• customer segments, such as users and payers or moms or teens
• distribution channels to reach customers and offer them the value proposition
• customer relationships to create demand
• revenue streams generated by the value proposition(s)
• activities necessary to implement the business model
• resources needed to make the activities possible
• partners and third parties needed to make the activities possible
• cost structure resulting from the business model

And it brings us to the definition of a startup: A startup is a temporary organization designed to search for a repeatable and scalable business model.

Testing Hypotheses

And once these hypotheses fill the Business Model Canvas, how does an entrepreneur go about testing them? If you’re a scientist, the answer is easy: You run experiments. The same is true in a Lean Startup. (The National Science Foundation described the Lean LaunchPad class as the scientific method for entrepreneurship.)

The Customer Development process is a simple methodology for taking new venture hypotheses and getting out of the building to test them. Customer Discovery captures the founders’ vision and turns it into a series of business model hypotheses. Then it develops a series of experiments to test customer reactions to those hypotheses and turn them into facts. The experiments can be a series of questions you ask customers, but most often, you pair those questions with a minimal viable product to help potential customers understand your solution.

So another big idea here is that startups are not building minimal viable products to build a prototype. They are building minimal viable products to learn the most they can.

Finally, the goal of designing these experiments and minimal viable products is not to get data. The data is not the endpoint. Anyone can collect data. Focus groups collect data. This is not a focus group. The goal is to get insight. The entire point of getting out of the building is to inform the founder’s vision. The insight may come from analyzing customer responses, but it also may come from ignoring the data or realizing that what you are describing is a new, disruptive market that doesn’t exist and that you need to change your experiments from measuring specifics to inventing the future.

Lessons Learned

1. Build, Measure, Learn is a great improvement over Waterfall product development and provided the framework to truly join the customer to agile development.
2. However, emphasizing “Build” or “Ideas” as the first step misses the key insight about a Lean Startup – you are starting with hypotheses to be tested and are searching for a repeatable and scalable business model.
3. Hypotheses, Experiments, Test, Insights better represents the Lean Startup process: Use the Business Model Canvas to frame hypotheses, Customer Development to get out of the building to test hypotheses, and Agile Engineering to build the product iteratively and incrementally.

Steve Blank is a retired serial entrepreneur-turned-educator who has changed how startups are built and how entrepreneurship is taught. He created the Customer Development methodology that launched the lean startup movement, and wrote about the process in his first book, The Four Steps to the Epiphany. His second book, The Startup Owner’s Manual, is a step-by-step guide to building a successful company. Blank teaches the Customer Development methodology in his Lean LaunchPad classes at Stanford University, U.C. Berkeley, Columbia University, UCSF, NYU, the National Science Foundation and the I-Corps @NIH. He writes regularly about entrepreneurship at www.steveblank.com.