The Intuition Validity Test
Trust intuition only in regular environments with feedback-rich practice
The Intuition Validity Test is Daniel Kahneman's framework for determining when intuition can be trusted and when it is an illusion of skill. The framework emerges from his work on System 1 (fast, automatic, intuitive) and System 2 (slow, deliberate, analytical) thinking. System 1 is always running, generating impressions, feelings, and inclinations. System 2 is lazy and tends to accept whatever System 1 suggests unless something triggers it to engage.
Kahneman identifies two specific conditions that must both be met for intuition to be reliable: First, the environment must be sufficiently regular and predictable -- there must be stable patterns to learn. Second, the person must have had enough practice with feedback to learn the regularities of that environment. Chess masters and experienced firefighters have reliable intuition because they operate in patterned environments with clear feedback. Stock pickers and political pundits do not because their domains are too random for intuition to be meaningful.
This framework matters enormously because most people default to trusting their intuition across all domains, when in reality intuition is only reliable in a narrow set of conditions.
- System 1 is always running and generates impressions that System 2 lazily accepts unless triggered to engage.
- Intuition is only reliable in regular environments where the person has had sufficient feedback-rich practice.
- In random environments like stock markets and politics, intuition is essentially an illusion of skill.
- Simple statistical models outperform human experts in virtually every studied domain because they eliminate noise.
- Assess Environmental RegularityBefore trusting your intuition on any decision, ask: Is this environment sufficiently regular and predictable? Does it have stable patterns that repeat? Chess is regular -- piece positions create knowable patterns. Fire behavior is regular -- flames follow physics. Stock markets are not regular enough -- too many random variables interact unpredictably. Political outcomes are not regular enough -- too many interacting factors defy pattern recognition. If the environment lacks sufficient regularity, your intuition is essentially random noise dressed up as expert judgment.Pro tipAsk yourself: Can an expert in this domain reliably predict outcomes better than a well-informed amateur? If the answer is no, the environment is too irregular for intuition to be valid.WarningOur subjective confidence in our intuition has almost no relationship to its actual accuracy. Feeling certain does not mean being right.
- Evaluate Your Feedback HistoryEven in regular environments, intuition requires extensive practice with clear feedback. A chess master has played thousands of games with immediate, unambiguous feedback (win or lose). An experienced firefighter has responded to thousands of fires with rapid feedback about what works. But a doctor who never learns the long-term outcomes of diagnoses, or a hiring manager who never follows up on rejected candidates, lacks the feedback necessary for intuition to calibrate. Without feedback, you are accumulating experience without accumulating learning.Pro tipMap your feedback loops: How quickly do you learn whether your decisions were right? How unambiguous is the feedback? Long delays or ambiguous feedback degrade intuition quality.
- Default to Algorithms Where PossibleIn domains where intuition fails the validity test, default to simple statistical models or structured decision processes. Kahneman reports that this is one of the most robust findings in psychology: in virtually every domain studied, simple algorithms outperform human experts. The reason is noise -- human judgment is incredibly variable. The same person looking at the same case on different days reaches different conclusions. Algorithms eliminate this inconsistency. The best approach is to use an algorithm as the primary tool and allow human judgment to adjust at the margins with clear rules about when and how much adjustment is permitted.Pro tipEven a simple checklist or scoring rubric performs better than unstructured expert judgment in most domains. You do not need sophisticated AI -- consistency beats brilliance.WarningExperts often resist algorithmic decision-making because it threatens their sense of expertise. Present it as noise reduction, not skill replacement.
Chess masters have reliable intuition because chess is a regular game with clear rules and immediate feedback. They have played thousands of games and can recognize board patterns instantly. Stock pickers, by contrast, operate in an environment too random for intuition to be meaningful. Despite years of experience, they cannot reliably outperform simple index funds because the patterns they think they see in stock markets are largely illusory.
In the 1950s, psychologist Paul Meehl demonstrated that simple statistical models outperform clinical experts in predicting outcomes across domains including parole decisions, academic performance, and medical diagnoses. This finding has been replicated hundreds of times in the decades since. Kahneman describes it as one of the most robust findings in psychology.
Daniel Kahneman developed this framework through his decades-long research collaboration with Amos Tversky, which won him the Nobel Prize in Economics in 2002. The specific conditions for valid intuition emerged from his engagement with naturalistic decision-making researcher Gary Klein, who studied expert intuition in firefighters, military commanders, and nurses. Kahneman and Klein discovered they could reconcile their seemingly opposite views -- Klein celebrating expert intuition, Kahneman questioning it -- by specifying the conditions under which each was right. Their 2009 paper 'Conditions for Intuitive Expertise: A Failure to Disagree' formalized the two-condition test. Kahneman's book 'Thinking, Fast and Slow' brought these ideas to a wide audience.