The Survivorship Bias Framework
Avoiding false positives
The Survivorship Bias Framework is a mental model used to identify and avoid false positives in data analysis. It recognizes that only survivors or successful cases are often reported, while failures or unsuccessful cases are ignored or hidden. This framework helps to adjust for this bias and provide a more accurate representation of reality.
- Only survivors or successful cases are often reported, while failures or unsuccessful cases are ignored or hidden.
- The absence of evidence is not evidence of absence.
- Adjusting for survivorship bias is crucial to avoid false positives and provide a more accurate representation of reality.
- Identify the potential for survivorship biasRecognize that the data may be incomplete or biased towards successful cases.Pro tipLook for inconsistencies in the data or missing information.WarningIgnoring survivorship bias can lead to false conclusions and poor decision-making.
- Adjust for survivorship biasUse statistical methods or data adjustments to account for the missing information.Pro tipUse techniques such as imputation or weighting to adjust for the bias.WarningFailing to adjust for survivorship bias can lead to inaccurate conclusions.
- Interpret the results with cautionConsider the limitations of the data and the potential for bias.Pro tipUse sensitivity analysis to test the robustness of the results.WarningOver-interpreting the results can lead to false conclusions and poor decision-making.
A mutual fund company reports an average return of 10% per year, but fails to disclose that 20% of their funds have gone bankrupt.
A diet company reports a 90% success rate, but fails to disclose that 50% of participants dropped out of the study.
The concept of survivorship bias was first identified in the field of finance, where it was observed that mutual fund returns were often misleading due to the exclusion of failed funds. Since then, it has been applied to various fields, including healthcare, business, and social sciences.