Evidence Hierarchy Filter
A structured filter for evaluating scientific claims based on evidence quality.
Dr. Norton outlines a systematic approach to evaluating health and nutrition information by prioritizing different types of evidence based on their quality and reliability. The framework emphasizes that not all evidence is created equal and provides a hierarchy to assess claims, moving from the strongest (meta-analyses) to the weakest (anecdotes or mechanistic pathways). The core insight is to distinguish between the existence of a biochemical mechanism and a real-world outcome, warning against the common mistake of assuming a pathway guarantees a result. This filter is designed to help individuals navigate conflicting information and make actionable decisions grounded in the highest available evidence.
- Not all evidence is created equal; quality varies dramatically.
- An outcome (e.g., fat loss) is the sum of countless pathways; a single pathway does not guarantee the outcome.
- Human randomized controlled trials (RCTs) are the gold standard for establishing causality.
- Mechanistic evidence (biochemical pathways) is hypothesis-generating, not conclusive.
- Always check if study design and methods actually test the claimed conclusion.
- Seek Meta-Analyses FirstLook for meta-analyses or systematic reviews on the topic. These compile multiple studies to identify overall consensus and effect size. Check the inclusion criteria to ensure they align with the question you're asking (e.g., were calories and protein equated in diet studies?).Pro tipA well-done meta-analysis with strict, relevant inclusion criteria (like controlled feeding trials) is the strongest starting point.WarningBad meta-analyses exist; be wary of those with biased inclusion criteria designed to support a narrative.
- Evaluate Randomized Controlled Trials (RCTs)If no clear meta-analysis consensus exists, examine the most tightly controlled human RCTs. Prioritize studies that measure the actual outcomes you care about (e.g., fat mass change, not just fat oxidation) and have robust methodology.Pro tipLook for RCTs that control for key confounding variables like calorie intake and protein intake when comparing diets.WarningBe cautious of conclusions that don't logically follow from the study's own data and design.
- Contextualize Cohort and Observational DataUnderstand that cohort or epidemiological studies (observing groups over time) show correlation, not causation. They are useful for generating hypotheses but are prone to healthy user bias and confounding factors (e.g., vegans might also exercise more).Pro tipUse this data to ask questions, not to draw definitive causal conclusions.WarningIt is easy to cherry-pick observational studies to support any narrative; always look for the overall trend.
- Scrutinize Mechanistic/Biochemical ClaimsWhen someone cites an isolated biochemical pathway (e.g., 'Compound X inhibits enzyme Y, therefore it causes Z'), ask: 'What is the overall human outcome evidence?' A pathway's existence does not mean it will produce a meaningful real-world result at typical exposures.Pro tipRemember the caffeine/glycogen phosphorylase example: a single inhibitory pathway is overwhelmed by the compound's net systemic effect.WarningThis is where most nutrition misinformation thrives—extrapolating from petri dishes or rodents to human health advice.
- Identify Narrative Cherry-PickingBe highly skeptical of sources that build a case using a handful of low-quality or outlier studies while ignoring the broader consensus. This is a red flag for agenda-driven information.Pro tipLook for force plots in meta-analyses; a true consensus will have most studies clustered on one side of the line.WarningAnyone can 'thread the needle of science' by selectively citing papers; your job is to see the whole forest.
- Separate Personal Anecdote from General AdviceRecognize that personal experimentation and anecdotal experience are a form of low-quality evidence. It's fine to try things based on this, but don't present them as scientifically proven general principles, especially if they contradict higher-level evidence.Pro tipDr. Norton's stance: 'It's okay to do things that don't have RCT support... but what I wouldn't do is come out and say what I do is the best thing ever... especially if there was human RCT data to the counter.'WarningConfusing what works for one person (or a few) with what works for most people is a common trap.
A common claim is that cruciferous vegetables (like broccoli) contain compounds (isothiocyanates) that bind iodine and impair thyroid function, leading to weight gain. This is based on an isolated biochemical pathway.
To answer the question 'Do low-carb and low-fat diets produce different fat loss results?', Dr. Norton points to a specific meta-analysis by Kevin Hall. The researchers used strict inclusion criteria: only controlled feeding trials that equated calories and protein, lasting at least 4 weeks, measuring actual fat mass change.
Developed through Dr. Norton's dual training in biochemistry and nutritional sciences, combined with his PhD research under Dr. Don Layman. His experience on bodybuilding forums in the early 2000s exposed him to common errors where isolated biochemical pathways were mistaken for definitive outcomes. This framework crystallized from his need to reconcile mechanistic understanding with practical, outcome-focused evidence in human nutrition and fitness.