Experiments to Achieve Product-Market Fit

Product-Market Fit

We explored the idea that not all business models are created equal, and that certain business models tend to lead to more valuable companies. As an entrepreneur, the onus is on you to construct each element of your startup business model, through a process of search and discovery. In parallel, you must evaluate those elements to ensure that you're building a sustainable, valuable company. To do so, each business model element must be aligned.

The search and discovery process brings us to our next theme, thinking about the startup as an experimentation machine, that is, an organization designed to run tightly defined experiments to rigorously test key hypotheses about your business model. We'll learn about the types of experiments that will allow you to be more effective in exploring and building an attractive business model.

To succeed in creating a business model that you will successfully scale, you need to achieve product-market fit or PMF. PMF is creating a compelling product that properly satisfies the target market, such that the market embraces the product. For example, Instagram has an obvious product-market fit. Over 2 billion users log on to the product on a monthly basis, and at least half a billion log on every day.

For a venture to be successful, your product needs to be what the market wants, differentiated from the competition, and contain enough attractive unit economics to be profitable on a sustained basis. As an entrepreneur, your goal is to test and validate all elements of your business model until you achieve product-market fit.

Your business model is a set of choices you make about the business. Your experiments test those choices. Once those choices prove out, you can transition from the jungle stage of your startup to the dirt road, unlock key fundraising milestones, and justify investing more capital to support scaling along a particular path.

Achieving product-market fit before scaling ensures your product meets customer needs and market demand, providing a solid foundation for sustainable and scalable growth. It's important to truly understand your customer and build a high-quality product before investing too much money growing the company. Being small, nimble, and continually learning until you feel confident you have achieved PMF.

We will focus on how to run experiments for three of the four business model elements: CVP, GTM, and PF (read here about all the elements).

Lean Startup Methodology

To learn how to achieve product-market fit, let's turn to American entrepreneur Eric Ries' pioneering work on the lean startup. The lean startup approach relies on a methodology for testing each element of a founder's vision.

As Eric puts it in his eponymous book, the key question is not, can this product be built, but rather, should this product be built? To answer this question, you can follow four steps:

1. Step one is ideation. You need to immerse yourself in a problem space and develop a vision for what the world might look like if that problem were solved. The ideation process is magical and creative. Ideas are often generated through brainstorming sessions and are informed by extensive customer research and discovery, a founder's past experiences, deep domain knowledge, or personal unmet need. Once you've come up with your innovative idea, it's time to test and validate it. We can use Design Thinking & Innovation in this step.
2. Step two then is hypothesis generation. In this step, you'll translate your vision into a series of hypotheses that cover each part of your business model - your CVP, GTM, and PF. Those hypotheses should be falsifiable. That is, you can test them and prove them to be true or false.
3. Step three is then to develop the actual tests. These tests are most efficiently run by developing a Minimum Viable Product, or MVP. The MVP is the smallest set of features or capabilities that take the least amount of development time to deliver the most learning as quickly as possible.
4. Finally, step four is test prioritization and sequencing. Based on your strategy,
   resources, time, and team, select which test to run on which elements of your business model and in a specific sequence. The choices behind your test sequencing are at their core a set of strategic decisions. There are an infinite number of tests that you can run, and today's software platforms allow for continuous experimentation. The travel website booking.com famously runs thousands of experiments each day using a sophisticated testing platform that allows them to determine which graphics, descriptions, and offerings are most compelling to prospective travelers. But unlike a multibillion-dollar public company, startups are constrained in terms of their resources and time. Thus, entrepreneurs must make hard choices about which tests are the most important, and if successful, could unlock the true power behind their business idea. For an entrepreneur, test prioritization is essential.

Once these four steps are completed - ideation, hypothesis generation, test development, and test prioritization and sequencing - you're ready to execute your tests and measure their results. The results of your tests will inform whether you:

1. Need to pivot and adjust your hypotheses before designing a new set of tests
2. Shut the business down because you are so far off the mark, or
3. Have achieved product-market fit and thus are ready to scale

Customer Value Proposition Experiments

Now that you have an ideal customer profile in mind, you need to look more closely at your CVP. It is critical to have a narrowly defined ideal customer profile (ICP) on which you can make deep customer discovery to best identify their pain points. This discovery will allow you to answer questions such as: What are the customer’s needs? What solutions do they currently use? How much better would a new product need to be in order to motivate them to switch? 

These customer “must haves” will serve as a map for product features. Using tools such as customer personas, journey maps, and MVPs can allow you to solicit feedback at every step of the process. 

How can you generate feedback that can be meaningfully interpreted? Run experiments that capture the essential elements of the customer experience as authentically as possible using the lean startup approach.

Profit Formula Implications

In the previous section, we learned about the CVP experiments to truly understand the needs of your ICP and build a product with the “must have” features that would meet those needs. Generally speaking, a founder’s next step would be to run experiments for GTM plan. This might involve analyzing the channels the venture will pursue to distribute its product; estimating the customer acquisition cost (CAC), which is the total cost of acquiring a customer; and selecting marketing tactics to target potential users. Put another way, building an amazing product that meets an important, “must have” requirement is not enough. You also must have a way to effectively and efficiently reach your target customers - effectively means you can reach and secure customers rapidly, while efficiently means you can also reach them profitably.

Once a founder has solidified their GTM plan, they would move on to their PF experiments. But, in practice, founders may not always follow this process exactly in this order. To achieve alignment across their business model elements, they instead might choose to test these elements in a different order. In fact, test selection and sequencing are important strategic decisions. Founders should always test the hypotheses that matter most.

Let's say that we feel confident about our CVP and GTM elements. What we need to test whether this value proposition and GTM plan would hold true with a larger market, and how much the business could expect to earn if we could continue to deliver value to our ICP.

Later in this article, we will practice with GTM experiments by analyzing the channel choices. For now, let’s learn about how CVP and PF can interact, and what you as a founder need to do to prove the venture is valuable both to the user with its offerings and to investors with its potential returns. 

Calculating Market Size

Ambitious startup founders have to determine whether the target market is a massive one. Only massive markets can support the invention of large, sustainable, profitable businesses, and only massive markets, will attract venture capital to fund your technology development and fuel rapid growth.

Before we dive deeper into how an entrepreneur can calculate their market size, let's consider an inherent tension between CVP and PF. On the one hand, you need to do things that don't scale to better understand the needs of your target customer. Yet, you also need to identify and target a massive market to demonstrate your product's potential profitability.

The way to think about addressing that tension is to view your initial market as a beachhead market. In World War II, allied forces sought to reconquer all of Europe in their battle with the Nazis. To achieve that ambitious goal, they landed on Normandy Beach in the famous D-Day invasion. Normandy Beach was their beachhead, the first place to establish a toehold. After which, a base can be built, and expansion can be achieved.

In the same way, entrepreneurs need to define their beachhead market, an initial product or initial geography, or vertical slice of the market, where they can focus their initial energy and scarce resources.

At the same time as you focus on your beachhead market, you have to demonstrate that you are targeting, and can succeed in a much larger market. If you can capture that larger market, will it be worth a lot of money?

Let's learn about the common terms entrepreneurs use to describe their market size. TAM, SAM, and SOM:

• TAM stands for Total Addressable Market, and represents the size of the market available to the company based on a simple equation, the total number of possible customers times the amount of money they will pay you each year. To determine how much a customer will pay over a set period of time, you can calculate the Average Revenue Per User, which is also referred to as ARPU or ACV. ARPU is the common label in a B2C company, while ACV - Average Contract Value is the common label in a B2B company. However, they can be used interchangeably. Both are calculated by dividing the total revenue of your business by the total number of customers, users, or contracts.
• SAM is the Serviceable Addressable Market, and represents a subset of the TAM. SAM forces you to think critically about what portion of the total number of target customers you could reasonably serve. Once you have the percentage of customers to target, you would multiply that number by the ACV, or ARPU, to get your SAM. For example, you can pick just a few top cities to serve and that would give you a percentage of TAM.
• Finally, SOM, is the Serviceable Obtainable Market. This term represents the percentage, or market share, of the SAM that the company might reasonably obtain when factoring in competition, pricing, and adoption curves. No company has a monopoly on the market. And so the SOM forces founders to be conservative in their estimates, and discount the market size even further. Determining SOM can be subjective because you have to estimate how much market share you may be able to get based on the number of competitors who are currently in or may enter the market. 

Unit Economics

Unit economics refers to the cumulative revenue and cost of an individual customer or unit of production. The underlying components of unit economics are more than just accounting calculations, but rather, important signals of product-market fit. Let's learn why.

A company that provides a service on an ongoing, recurring basis, whether it is a SaaS service like Cloud Storage from Amazon Web Services or a consumer subscription service like Netflix, can only be successful if customers stick with them for many, many months and years. The entire business model of a recurring business is to hold on to customers as long as possible, collecting that additional revenue every single month.

Early success driving signups for initial product is an important achievement, but you have to convince yourself (and investors) that customers would continue to use the product day in and day out, and that the willingness to pay would be high enough to justify the customer acquisition costs.

There are a few important concepts required to understand unit economics - your customer acquisition cost, your churn rate, and your customer lifetime value. Let's look at each in turn:

• Customer Acquisition Cost, or CAC, is the sum of all the sales and marketing expenses, both personnel and programs, required to acquire one customer. For example, according to Netflix's financial reports, the company spends roughly $100 in marketing to acquire one customer.
• The Churn rate is the percentage of customers that cancel or stop paying each month or year. If you start with 100 customers on day 0, and after one month, five of them cancel, then your churn rate is 5% per month. For Netflix, they lose roughly 3% of their customers each month. Churn rate is a measure of attrition or loss and is an important metric to track for any SaaS or subscription business. You can think of the churn rate as the user cancellation rate. Users churn when they are not compelled to continue to use and pay for your product. Sometimes churn can be expressed as a percentage of users and other times as a percentage of revenue. For example, if you have 100 users paying you $100,000 one month, and you lose 5 of them, but in aggregate, the other 95 increase their usage and pay you $120,000, then your user churn is 5%, but your revenue churn is 120%. In other words, your churn in this example is negative. Negative churn is when new revenue from existing customers is greater than the amount of revenue you lost from cancellations and downgrades. Understanding your churn rate is important because it allows you to test hypotheses about how long customers will remain as subscribers, eventually leading to an indication of the quality of both your CVP and PF. 
• Customer Lifetime Value, or LTV, is the total value of a customer over the lifetime that the customer remains engaged with the company's products or services. That figure is calculated by adding up the profit contribution of each customer during their lifetime with you as a customer. For example, the average Netflix subscriber pays $12 per month. Let's call that their ARPU - and stays on for approximately 30 months. Assuming 3% of customers churn per month, thus your entire customer base churns after 30 months. Then the total revenue is $360. If the company's gross profit margin is 50%, that implies a $180 lifetime value, or LTV.

These terms - TAM/SAM/SOM, ARPU, CAC, churn, LTV - are the building blocks of determining if you are achieving product-market fit and creating a valuable company. Remember this formula for a successful venture - an exceptional team, pursuing a massive market with an attractive business model.

A good rule of thumb to determine if you have product-market fit is if your churn rate is below 3% per month. That implies customers like your product well enough to stick around for three years. Typically, churn will be higher in the early days of a startup as the company centers in on the right customer segment and delivers a better and better product. The best companies get to the point where they can get their churn rates below 1% per month, or perhaps even achieve negative churn - that is, their set of customers as a cohort are expanding their use of the product or perhaps using multiple products as they get comfortable with the service.

You want to expand TAM and ARPU, and prevent customers from churning because they're using more of your services. With vertical SaaS, you want to have your customers using as many of the products as possible.

Cohort Analysis

Cohort analysis is a subset of behavioral analytics that takes a group of people with shared characteristics (a cohort) and examines their behaviors over a period of time. This method is particularly useful in understanding how specific groups of users or customers behave throughout their lifecycle with a product or service.

Key Concepts of Cohort Analysis

• Cohort: A cohort is a group of users who share a common characteristic or experience within a defined time period. For example, users who signed up for a service in January, or customers who made their first purchase in Q1.
• Time Periods: Cohort analysis tracks these groups over different time periods to observe how their behavior changes. This could be daily, weekly, monthly, or yearly, depending on the analysis requirements.
• Behavioral Metrics: Common metrics tracked in cohort analysis include retention rates, churn rates, engagement levels, and conversion rates. These metrics help in understanding user behavior patterns and identifying trends.

Types of Cohorts

• Acquisition Cohorts: These cohorts are based on the time of user acquisition. For instance, tracking users who signed up in a specific month and observing their behavior over the following months.
• Behavioral Cohorts: These cohorts are based on user actions or behaviors, such as users who performed a specific action (e.g., made a purchase, clicked on an ad) during a particular period.

Applications of Cohort Analysis

• User Retention: By analyzing cohorts, businesses can understand how long users stay engaged with their product and identify patterns in user retention. This helps in identifying successful engagement strategies and areas that need improvement.
• Customer Churn: Cohort analysis can help identify when users are most likely to churn (stop using the product or service). Understanding churn patterns enables businesses to implement targeted retention strategies.
• Marketing Effectiveness: Marketers can use cohort analysis to measure the effectiveness of campaigns and promotions. By comparing cohorts exposed to different marketing strategies, businesses can determine which approaches yield the best results.
• Product Development: roduct teams can use cohort analysis to understand how new features or changes affect user behavior. This helps in making data-driven decisions about product enhancements and updates.

Steps to Perform Cohort Analysis

• Define the Cohort: Determine the characteristic or event that defines the cohort (e.g., sign-up date, first purchase).
• Select the Time Period: Choose the time intervals for tracking the cohort’s behavior (e.g., weekly, monthly).
• Track Metrics: Identify the key metrics to track for the cohort (e.g., retention rate, conversion rate).
• Analyze Data: Use data visualization tools to plot the metrics over time and identify trends and patterns.
• Interpret Results: Draw insights from the data to understand user behavior and make informed business decisions.

Example of Cohort Analysis

Imagine an online subscription service wants to understand how well it retains users who signed up in January compared to those who signed up in February. By creating acquisition cohorts based on the month of sign-up and tracking their retention rates over six months, the service can identify any differences in user retention and investigate the causes, such as differences in marketing campaigns or onboarding processes.

Conclusion

Cohort analysis is a powerful tool for understanding user behavior over time. By breaking down data into specific groups and tracking their behavior, businesses can gain valuable insights into user retention, customer churn, marketing effectiveness, and product performance. This data-driven approach helps in making informed decisions to improve user engagement and business outcomes.

To assess whether the desired behavior would be exhibited by your customers, you have to apply an analytical technique known as cohort analysis. Cohort analysis allows you to diagnose changes in customer behavior over time, illuminating the impact of product changes and other experiments on your churn rate or ARPU. The most sophisticated startups measure cohorts by a range of variables, such as time period, customer segment, acquisition channel, or product type.

Assessing business model and PMF

Let's bring market sizing, churn, and cohort analysis together and analyze the quality of the business model using business model quality analysis and the LTV/CAC ratio to help determine if we have achieved PMF.

The LTV/CAC Ratio

The LTV to CAC ratio is a critical one for any business, as it compares how much money a customer is worth to the company over the total amount of time they engage with the company (LTV), as compared to how much it costs to acquire that customer in the first place (CAC).

A high ratio implies attractive unit economics because your essential profit formula is a success, while a low ratio implies you may need to adjust your business model, perhaps your value proposition, your go-to market methods, or your pricing, and run some more experiments to improve your profit formula.

To accurately calculate lifetime value, which, as a reminder, is the total profit contribution of a customer over the customer's lifetime, you need to make sure you don't forget a few important elements:

• The value of a customer must factor in your gross margin, which is the amount of money you have left after subtracting all direct costs of producing or purchasing the goods or services you sell. After all, not every dollar you take in revenue is pure profit. There are some costs that are required to deliver the service month in and month out.
• A realistic amount of time must be factored in, as you can't assume you can hold on to your customers forever. That's where your churn rate assumption comes in, and some reasonable time cap.
• If you really want to be precise, you should factor in the timing of the cash flows. If you make all your positive profits in year 10, it's not as attractive as if you received the cash flow upfront. That reflects the time value of money. Money today is worth more than money at some future date. For a big company, the cost of capital, that is, the interest payments required to borrow capital, can be quite low. But a startup's cost of capital is quite high. After all, it's not like they can borrow money from a bank at a super low-interest rate. Thus, the timing of cash flows matters. As always, sooner is better.

For now, let's do the math in a simplified fashion first, where we don't worry about the time value of money. In that case, LTV is the sum of the revenue times the gross margin times the number of periods in a customer lifetime.

Let's now interpret the LTV/CAC ratio, here's the rule of thumb:

• 0 to 1 is unattractive. Because you're paying more to acquire customers than they're worth. Thus, if you find yourself less than 1, you need to dramatically retool.
• 1 to 3 means new customers are slightly profitable on a marginal basis, but may not generate enough profits to cover your overhead. That said, there may be some adjustments you can make to improve your profit formula.
• Greater than 3 means, every dollar you invest in customer acquisition yields three or more dollars in profit. That's a good business. You should continue to invest heavily in customer acquisition, as it will be more and more profitable with time.

Assessing

Now, we will use everything we have learned to assess the quality of the business model and determine whether we have found PMF.

Recall the elements of a high-quality business model:

• Tight flywheel
• Strong network effects

Additionally, high-quality businesses will have:

• Recurring revenue with net negative churn: Users continue to pay to use your product over time, and these revenues are greater than the revenue lost from cancellations and downgrades. 
• High gross margins: The amount of money you can retain after paying for your overhead is high. 
• Metrics that improve with scale: The larger your venture gets, the better your LTV/CAC ratio, churn rates, and market size get.
• High switching costs: It would be expensive and difficult for users to switch to a different provider.
• Organic demand: There is a high demand for your product, so your marketing costs are low.
• Rich customers with high willingness to pay: You are going after a few rich customers or many smaller customers with a high willingness to pay.

Using these guidelines and the unit economics, you should be able to assess the quality of a business model.

We now have all the tools we need to determine product-market fit using a framework called HUNCH. Hunch because PMF is never a sure thing in light of the small amount of data you have to analyze in those early days. Thus, you should be humble in your assessment, recognizing that you can't be altogether sure, but at least you have a hunch if the following is true:

• Hair on fire value proposition: is the value proposition a must-have need for your target customer who is 10x better than the alternatives, and where many of your users would be very disappointed if you took it away?
• Usage high: your product usage and engagement are high and growing - something you can observe by carefully analyzing the behavior of certain customer cohorts over time.
• NPS, or net promoter score, exceeds 40: a net promoter score measures customer experience and satisfaction. It is calculated by surveying customers and asking how likely it is, on a scale of 0-10, they would recommend the company to a friend. You take the percentage of customers that rate 9-10 (promoters) minus the percentage of customers who rate 0-6 (detractors) to get the score.
• Churn low: ideally less than 3% per month, showing an enduring value proposition over time.
• High LTV/CAC ratio: ideally greater than three with conservative calculations, showing your unit economics are positive, and thus you can acquire customers and generate positive returns at scale.

Testing Go-to-Market Hypotheses

Testing go-to-market (GTM) hypotheses is a crucial step in the development and launch of a new product or service. It involves validating the assumptions you have about your target market, value proposition, sales strategy, and distribution channels. This process helps ensure that your product is well-received by the market and that your marketing and sales efforts are effective. Here are the key aspects of testing go-to-market hypotheses:

Key Components of Go-to-Market Hypotheses

• Target Market: Identifying and validating the specific audience segments you believe will be most interested in your product. This involves understanding their needs, preferences, and pain points.
• Value Proposition: Testing whether the benefits and unique selling points of your product resonate with your target market. This helps ensure that your messaging aligns with what your potential customers find valuable.
• Sales Strategy: Assessing the effectiveness of your sales tactics, including pricing models, sales channels, and sales processes. This involves experimenting with different approaches to see which yields the best results.
• Distribution Channels: Evaluating the channels through which your product will reach the market, such as online platforms, retail stores, or direct sales. This helps determine the most efficient and cost-effective way to distribute your product.

Steps to Test Go-to-Market Hypotheses

• Formulate Hypotheses: Clearly define your go-to-market hypotheses. For example, “Our target market is young professionals aged 25-35” or “Customers will prefer a subscription pricing model.”
• Design Experiments: Develop experiments to test these hypotheses. This could involve A/B testing different marketing messages, piloting different sales approaches, or launching in a limited market segment.
• Collect Data: Use qualitative and quantitative methods to gather data. This might include surveys, interviews, focus groups, and analytics from pilot programs or initial sales efforts.
• Analyze Results: Analyze the data to determine whether the results support or refute your hypotheses. Look for patterns and insights that can inform your go-to-market strategy.
• Iterate and Refine: Based on the analysis, refine your hypotheses and experiments. Continue testing and iterating until you have a validated go-to-market strategy.

Benefits of Testing Go-to-Market Hypotheses

• Risk Mitigation: Identifying and addressing potential issues early on reduces the risk of a failed product launch.
• Resource Efficiency: Focuses your efforts and resources on the strategies that have been proven to work, avoiding wasted time and money on ineffective tactics.
• Customer Insights: Provides valuable insights into customer behavior and preferences, helping to fine-tune your product and marketing strategies.
• Market Readiness: Ensures that your product is well-positioned in the market, increasing the likelihood of a successful launch and adoption.

Examples of Go-to-Market Hypothesis Testing

• Target Market Hypothesis: A company believes that their new app will be most popular among tech-savvy millennials. They run targeted ad campaigns on social media platforms frequented by this demographic and track engagement and conversion rates to validate this hypothesis.
• Value Proposition Hypothesis: A SaaS provider hypothesizes that their software’s time-saving features are the main selling point. They create different marketing materials highlighting various benefits and measure which message leads to higher conversion rates.
• Sales Strategy Hypothesis: A startup is unsure whether to offer a freemium model or a free trial. They run parallel campaigns offering both options to see which approach results in more paid subscriptions.

Conclusion

Testing go-to-market hypotheses is an essential part of ensuring a successful product launch. By systematically validating your assumptions about the market, value proposition, sales strategy, and distribution channels, you can develop a robust and effective go-to-market strategy. This process mitigates risks and provides valuable insights that help in refining and optimizing your approach for maximum impact.

Wrap Up

It's all about experimentation, testing each of the elements of your business model in a carefully constructed and rigorously executed sequence. During these early days, it's important to maximize learning rather than maximize metrics. Yes, there are important metrics to focus on to test product-market fit, but don't rush to scale it until you nail it.

• To determine CVP fit, you want to make sure that customers love your value proposition. Those target customers will be innovators. And the way you're going to reach them is by having you as a founder reach out to them directly, through all your networks and various referral channels. At this point in the journey, don't worry about pricing. Price your product low or even give it away for free, whatever it takes to drive initial adoption, and test your value proposition.
• To determine GTM fit, you want to run customer acquisition experiments across a range of channels so that you can develop a repeatable GTM playbook. At this stage, the founder may bring on board additional sales and marketing resources to help them scale customer acquisition, ideally in the form of a growth team or expeditionary sales leader, who is accustomed to uncertainty and a lack of structure. As you're running paid marketing experiments, you begin to focus a bit more on your unit economics and pricing, perhaps aiming for breakeven on a unit economics basis.
• After establishing your CVP fit and GTM fit, you can then turn your attention to the Profit Formula. Here, you focus more carefully on unit economics, willingness to pay, and a repeatable growth strategy. At this stage, everything is scaling. Your target customer scales from early adopters to the early majority. Your sales and marketing efforts scale from a nascent team to more of a systematic machine or process. And your demand generation work becomes multifaceted and multichannel.
• Now, at each of these three stages, there are risks to watch out for. At the CVP experimentation stage, you want to avoid false positives and distractions that come with successful financings, winning pitch competitions, or positive press. At the GTM experimentation stage, you want to avoid focusing on early metrics that are more vanity-oriented, for example, number of downloads versus PMF or usage-oriented, for example, number of log ins or active users. And at the PF experimentation stage, above all, you want to avoid premature scaling, burning a lot of money when you have high churn and an inefficient sales and marketing machine, that is, a low LTV to CAC ratio.