Good Metric Framework
Five criteria that separate actionable metrics from vanity noise
The Good Metric Framework provides five essential criteria for evaluating whether a metric deserves your attention. A good metric must be comparative, understandable, a ratio or rate, and most importantly, it must change the way you behave. These four qualities separate actionable analytics from feel-good vanity numbers that only stroke your ego.
The framework also introduces five dimensions along which all metrics can be classified: qualitative versus quantitative, vanity versus actionable, exploratory versus reporting, leading versus lagging, and correlated versus causal. Understanding where a metric falls along each of these dimensions helps you choose the right measurement for your current situation and avoid common analytical traps.
Vanity metrics are the most dangerous pitfall because they provide false confidence. Total signups, page views, number of followers, and raw download counts all share a fatal flaw: they only go up over time, they tell you nothing about user behavior, and they do not change how you act. The antidote is always to ask: what will I do differently based on this information?
- A good metric is comparative: it can be measured against other time periods, groups, or competitors
- A good metric is understandable: if people cannot remember and discuss it, it will not change culture
- A good metric is a ratio or rate: ratios are easier to act on, inherently comparative, and good for showing tension between opposing factors
- A good metric changes the way you behave: if you cannot answer what you would do differently based on this data, stop tracking it
- Metrics often come in pairs that together reveal a fundamental truth: conversion rate and time-to-purchase together tell you about cash flow
- Audit your current metricsList the top three to five metrics you track daily. For each one, honestly evaluate whether it is comparative, understandable, a ratio, and behavior-changing. Cross off any that fail these tests.
- Apply the vanity testFor each remaining metric, ask: does this number only go up over time? Can it make me feel good without telling me anything useful? If so, replace it with an actionable alternative. Replace total signups with percent of active users. Replace total downloads with activation rate.Pro tipThe eight classic vanity metrics to avoid are: hits, page views, visits, unique visitors, followers and likes, time on site, emails collected, and raw downloads.
- Classify along five dimensionsFor each metric, determine if it is qualitative or quantitative, vanity or actionable, exploratory or reporting, leading or lagging, and correlated or causal. This classification reveals blind spots in your measurement strategy and helps you balance your analytics portfolio.Pro tipLeading metrics are always more valuable because you still have time to act on them. Customer complaints predict churn; by the time you measure churn itself, those customers are already gone.
- Ensure you have both qualitative and quantitative coverageQuantitative data tells you what is happening; qualitative data tells you why. Early-stage companies need more qualitative input through interviews and observation. Later-stage companies lean more quantitative. But you always need both to avoid flying blind.Pro tipWhen your quantitative metrics show something unexpected, pick up the phone and call customers. All the numbers in the world cannot explain why something is happening.
Circle of Friends had 10 million users but less than 20 percent engagement. Founder Mike Greenfield dug into the data and discovered that mothers had 50 percent longer messages, were 115 percent more likely to attach photos, and had friends who were 50 percent more likely to become engaged users. This was an unknown unknown that changed the company's entire direction.
HighScore House set a line in the sand of four uses per week to define an active family. When engagement fell short, founder Kyle Seaman called customers directly. He discovered that families using the product only once or twice per week were still getting significant value, with kids making beds consistently for the first time.
This framework draws from the collective wisdom of analytics practitioners including Avinash Kaushik (Google's Digital Marketing Evangelist), Seth Godin, and numerous startup founders who shared their experiences of being misled by the wrong numbers. Croll and Yoskovitz codified these principles after observing that the most common failure in startup analytics was not a lack of data, but a focus on the wrong data.
The car dealership anecdote that opens the discussion is telling: a salesperson spent more time asking for a good satisfaction rating than providing a great experience, perfectly illustrating how measuring the wrong thing distorts behavior rather than improving it.