The Rethinking Scorecard
Evaluate decision quality by the thoroughness of the process, not just the outcome
Grant argues that most organizations and individuals evaluate decisions backward, judging them solely by their results. A good outcome from a sloppy process gets celebrated, while a bad outcome from a rigorous process gets punished. This creates perverse incentives: people learn to optimize for luck rather than for thoroughness.
The Rethinking Scorecard flips this by tracking the quality of the decision-making process itself. Did you consider multiple options? Did you seek disconfirming evidence? Did you consult your challenge network? Did you define criteria for success and failure in advance? A bad process with a good outcome is luck. A good process with a bad outcome might be a smart experiment worth learning from.
This framework connects to Grant's broader argument about learning cultures versus performance cultures. Performance cultures reward results and inadvertently punish the rethinking that produces long-term improvement. Learning cultures reward process quality and create the conditions for sustainable excellence.
- A bad process with a good outcome is luck, not skill
- A good process with a bad outcome might be a smart experiment worth learning from
- Track how thoroughly different options are considered, not just whether the chosen option worked
- Process accountability creates sustainable improvement; outcome-only evaluation creates fragility
- Define success and failure criteria before making decisions to prevent post-hoc rationalization
- Define your process quality criteriaBefore making a decision, list the elements of a good process: Were multiple options generated? Was disconfirming evidence sought? Were diverse perspectives consulted? Were success and failure criteria defined in advance? Was there a pre-mortem?
- Score the process before knowing the outcomeRate each decision on the quality of the process used to make it. Do this before the outcome is known, or at least record your process assessment before revealing results. This prevents hindsight bias from contaminating your evaluation.
- Track process and outcome separately over timeMaintain a decision journal that records both the process score and the eventual outcome for each significant decision. Over dozens of decisions, patterns will emerge: good processes should correlate with better outcomes on average, even if individual results vary.
Before the Challenger disaster, engineers raised concerns about O-ring failures in cold weather. Because O-rings had not failed in previous launches despite the known risk, the concern was overridden. Before the Columbia disaster, engineers noticed foam striking the wing but normalized it because previous foam strikes had not caused problems. In both cases, good outcomes from past bad processes were treated as evidence that the process was fine.
Grant developed this concept from his work studying and advising change initiatives at NASA and the Gates Foundation. At NASA, he observed how a performance culture rewarded outcomes and inadvertently suppressed the questioning that could have prevented disasters. The Challenger and Columbia shuttle disasters both resulted from processes where dissenting evidence was dismissed because past outcomes had been fine. Had a rethinking scorecard been in place, evaluating whether all evidence was thoroughly examined regardless of the outcome, the warning signs might have been taken seriously.