How we evaluate research

In the past, articles on this site attempted to exhaustively review all the research on a subject. But the volume of observational nutrition research has grown far beyond what any small team can review — new cohort analyses, case-control studies, and meta-analyses are published at a rapidly increasing rate, across a huge and ever-expanding range of nutrient-disease pairings — and attempting to track and weigh all of it isn’t realistic. More importantly, though, most individual observational findings simply don’t deserve much weight in the first place. A statistically significant association is not the same as a causal one, and modest effect sizes — the kind that make up the large majority of published nutrition findings — are especially prone to confounding, reverse causation, and other biases that no amount of statistical adjustment fully resolves.

There’s also a benefit to you, the reader, that’s easy to overlook: when we don’t apply strong criteria up front, the alternative isn’t a cleaner article — it’s a longer one, where every marginal, often contradictory finding gets its own paragraph of caveats so you can weigh it yourself. Filtering first means we can tell you what the evidence shows without walking you through a dozen small, conflicting studies that wouldn’t have changed the answer either way.

This design requirement for observational research applies to studies testing an association between diet and a health outcome. Purely descriptive studies of nutrient or biomarker status in a population are evaluated separately.

Randomized controlled trials and other clinical trials don’t have the volume problem, and they carry more evidentiary weight to begin with—randomization handles confounding that observational data can’t. But they can still mislead in narrower ways: a trial can report a “significant” result on an outcome that wasn’t its pre-specified primary one, test enough outcomes that one comes up positive by chance, or find an effect too small to matter in practice even if it’s statistically real. So we apply a different set of criteria to trial evidence — centered on registered primary outcomes, correction for multiple comparisons, and a meaningful effect size — rather than the volume and weight concerns that apply to observational research.

As of September 2026, the criteria below are our attempt to draw a defensible line for each type of evidence: research that clears the relevant bar is treated as worth including; research that doesn’t is generally left out, regardless of how interesting or alarming it might sound in isolation. As this is a new experiment, there may be gray areas we haven’t yet anticipated.