Why so many nutrition studies are "fundamentally and fatally flawed"

What did you eat for lunch yesterday? Did you sprinkle nuts or dressing on your salad? Did you snack afterward? Exactly how many potato chips did you eat?

Chances are you probably don't remember. And yet, a lot of nutrition research today relies on just that kind of information: people's self-reporting from memory. Researchers ask people to recall the foods they consumed the previous day, keep a food diary, or check off from a list the foods they ate in the last day, week, month, or year.

It shouldn't come as a surprise, then, that researchers argue today in the journal Mayo Clinic Proceedings that this type of data is "fundamentally and fatally flawed" and "essentially meaningless."

They explain that "memory-based dietary assessment methods" — usually interviews, questionnaires, and surveys — are the dominant data collection method in many national nutrition surveillance and government-funded studies on food and obesity in the US. These studies inform things such as national food guidelines and food policy directions.

And yet, they argue, this data doesn't actually give an accurate picture of what people ate. For example, this table from the paper shows us that over the 39-year history of the National Health and Nutrition Examination Survey — which is based on self-reported food intake — the alleged number of calories consumed by 67.3 percent of women was not "physiologically plausible" given their body mass index (a measure of obesity).

Mayo Clinic Proceedings

This may be because people lie about what they eat, offering up answers that are more socially acceptable. Or it may be a failure of memory. The authors draw on social science and neuroscience studies to explain why that's highly likely:

"The use of [memory-based surveys] requires faith in the belief that human perception, memory, and recall are accurate and reliable instruments for the generation of scientific data. Nevertheless, more than 80 years of research demonstrates that this belief is patently false."

While the researchers argue that future scientific funding should not be invested in memory-based studies, they offer little by way of an alternative.

This may be because the problem of getting an accurate read on what people ate is challenging the entire field, food policy researcher Marion Nestle explained in this Atlantic piece:

Will you join us?

Our biggest supporters are our readers — and we’re so grateful to everyone who has made a contribution during our September campaign. We’re less than 1,000 contributions away from reaching our goal for the month, which in turn will allow us to say yes more often when our incredible journalists come to us with questions they want to answer and projects they want to pursue. Will you make a contribution before the month ends and support our policy coverage through 2024 and beyond?

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via