Line in the Sand Benchmarking
Know what good looks like before you measure anything
Line in the Sand Benchmarking is the practice of establishing specific, numerical targets for your key metrics before you start measuring, based on industry benchmarks, business model requirements, and what is known about typical startup performance. Without a line in the sand, you cannot interpret your results: most experiments end in the ambiguous middle where you cannot tell if you succeeded or failed.
The framework provides specific benchmarks across business models. For SaaS, monthly churn below 5 percent (ideally 2 percent) indicates a sticky product. For e-commerce, a 90-day repurchase rate determines whether you are in acquisition, hybrid, or loyalty mode. For growth, Y Combinator targets 5 to 7 percent weekly growth. For customer acquisition cost, spend no more than one-third of customer lifetime value. For engagement, Fred Wilson's 30/10/10 rule says 30 percent of registered users should visit monthly, 10 percent daily, and peak concurrent users should be 10 percent of daily users.
Critically, knowing what is normal for your industry prevents you from over-optimizing a metric that is already at its natural ceiling. WP Engine's founder discovered that 2 percent monthly churn is the best-case scenario for hosting companies. Without that benchmark, he would have wasted resources trying to push churn below a floor that the entire industry has established.
- Without a predefined target, most experiment results are ambiguous and unactionable
- Lines in the sand come from two sources: your business model requirements and industry benchmarks
- Average startup performance is nowhere near good enough; most startups fail because average is not sufficient
- Knowing the ceiling for a metric prevents you from over-optimizing something that is already as good as it gets
- It is acceptable to adjust your targets if you have genuine qualitative evidence for why the new target better reflects reality
- Research benchmarks for your business modelGather industry-specific data for the metrics you are tracking. Use resources like Startup Compass, industry reports, investor networks, and comparable companies. For each metric, find the average, the good, and the exceptional values.Pro tipAsk your investors and advisors for benchmark data. They often have portfolio-wide data that reveals industry norms not available publicly.
- Set targets from your business model mathWork backward from your financial model. If you need 10 percent of users to convert to paid in order to meet revenue targets, that is your line in the sand regardless of what the industry average is. Business model requirements take precedence over benchmarks.
- Establish the line before running the experimentAlways define what success looks like before you collect data. Know what you will do if you hit the target and what you will do if you miss it. This prevents post-hoc rationalization and ensures that data actually drives decisions.Pro tipAgree on the line in the sand with your co-founders and advisors before the experiment starts. Changing it afterward is cheating unless you have genuine new understanding.
- Compare your results to the line and act decisivelyWhen results come in, compare them to your predefined target. Dramatically above the line: double down. Dramatically below: pivot or iterate. In the ambiguous middle: this is where the discipline matters most. If you are close to the line, run a follow-up experiment. If you are clearly below, acknowledge it.WarningDo not lower the bar to avoid hard truths. Adjusting your definition of success without genuine qualitative evidence is just lying to yourself.
- Know when to stop optimizing and move onOnce you have hit or exceeded the benchmark for your current metric, stop over-optimizing and shift focus to the next critical metric. Diminishing returns on a well-optimized metric mean your energy is better spent elsewhere.Pro tipWP Engine's founder learned that 2 percent monthly churn was the industry best-case. Instead of trying to push it lower, he focused on other metrics where improvement was possible and impactful.
Jason Cohen of WP Engine was deeply concerned about his 2 percent monthly churn rate, which meant losing 24 percent of customers annually. He researched the hosting industry using investors and advisors, including Automattic, and discovered that 2 percent monthly churn is the best-case scenario for even the largest, best hosting companies.
Before shifting focus from stickiness to acquisition, Buffer founder Joel Gascoigne specifically checked churn rate against a target of below 5 percent. With churn hovering around 2 percent, he had the confidence to stop optimizing retention and shift resources to user acquisition.
The line in the sand concept was driven by the authors' frustration with a common startup pitfall: founders who track metrics religiously but have no idea whether their numbers are good or bad. This was validated repeatedly at Year One Labs, where startups would proudly present their numbers without any context for whether those numbers warranted celebration or alarm.
The benchmark data itself was gathered from over 100 interviews with founders, investors, and analysts, plus data from companies like Startup Compass, Price Intelligently, Chartbeat, and Totango. Jason Cohen of WP Engine provided one of the most compelling stories: he was deeply concerned about his 2 percent monthly churn until he discovered that this was actually best-in-class for the hosting industry.