Conflict of interest in nutrition research is nothing new. Health food is a billion-dollar business, and the more virtuous the food, the better it sells.
From walnut growers to grape juice producers, there are hundreds of examples of companies and industry groups funding so-called independent studies that corroborate marketing claims about their product’s ability to improve health and fight disease.
Trouble is, consumers are almost always snookered by this shady science.
The newest example is a study funded by Ocean Spray, the world's leading producer of cranberry juices. It concludes that "cranberries can be a nutritional approach to reducing symptomatic [urinary tract infections]."
Cranberries contain compounds that are thought to prevent bacteria from sticking to the cells that line the bladder. So cranberry juice makers have long hyped the claim that their products prevent UTIs, those pesky infections that lead to pain and discomfort with peeing after harmful bacteria get into the urinary system.
The study had a fantastic veneer of legitimacy: The randomized, double-blind, placebo-controlled trial, published in the American Journal of Clinical Nutrition, involved nearly 400 women at 18 clinical sites throughout the US, as well as a Boston University scientist.
Naturally, Ocean Spray trumpeted the research as "landmark," suggesting it conclusively settled the debate about whether the red stuff can truly prevent infections. It even stated that cranberry juice is key to the fight against antibiotic resistance, since the drugs are often prescribed for UTIs and could be replaced with cranberry juice instead.
Cranberry juice doesn’t prevent UTIs — and this study didn’t prove it does, either
:format(webp):no_upscale()/cdn.vox-cdn.com/uploads/chorus_asset/file/6667603/shutterstock_262367660.0.jpg)
Darryl Brooks/shutterstock
There were some things the press release left out, however.
The study wasn’t just funded by Ocean Spray; it was also co-authored by Ocean Spray staff scientists. Not only was the food company involved in nearly every step of the process but its scientists even helped write the manuscript.
At first blush, it was hard to see how the company’s involvement might have biased the research.
But there are a few details that raised red flags. According to the study, a glass of cranberry juice a day reduced UTIs by nearly 40 percent in women. Most drugs don’t have such large effect sizes — and analyses of the best available research on cranberries and UTIs, like this Cochrane systematic review, have found that cranberry juice doesn’t reduce the occurrence of UTIs (probably because the juices — or even cranberry supplements — don’t contain enough of the compounds that fight bacteria).
As the Cochrane review notes, "To maintain levels of cranberry PAC [the bacteria-fighting compound in cranberries] that are necessary to prevent [UTIs], people would have to continuously drink the juice twice a day in serving of 150 mL for an indefinite period of time."
So how did the Ocean Spray group manage to come to such striking conclusions?
It turns out this had much to do with how the researchers chose to measure and analyze their data.
First, they focused on a much broader — and more favorable — definition of what a UTI is.
The gold standard measurement is a urine test that finds higher than normal levels of bacteria. In the Ocean Spray study, the cranberry juice group and the placebo group had about the same number of infections by this measure.
But instead of focusing on that, the researchers emphasized their findings on "symptomatic UTIs" — meaning women who simply complained of UTI symptoms but didn’t actually have a positive urine culture. By this much looser measure, the two groups indeed differed, and in a way that favored drinking cranberry juice: There were 39 episodes of symptomatic UTIs in the cranberry group compared with 67 episodes in the placebo group.
I asked independent researchers what they thought of this approach — and they called it "smoke and mirrors."
"They made [cranberry juice] appear much more effective by using a clinical definition — symptoms — which is rubbery at best," said Jonathan Craig, one of the authors of the Cochrane review and a clinical epidemiology professor at the University of Sydney. "By definition, a UTI means you have an infection in the urinary tract. How can you have a UTI without the 'I'?"
The other way the results were interpreted to make them more favorable to Ocean Spray was by counting the number of UTI episodes among the women, instead of noting how many women saw a benefit in each group.
The technical term for this is clustering, Craig explained. And the problem is that it clumps together the women who might have one or no UTIs and the women who have many.
Imagine you give cranberry juice to 10 women, and another 10 women act as the control group and get sugar water. Let’s say everyone in the sugar water group gets a UTI and no one in the cranberry group does. That would mean you prevent 10 women getting 10 events — and that the cranberry juice is a very effective treatment.
But this study actually looked more like a second scenario: "Imagine all the women in both groups get a UTI, but one woman in the control group gets 11 events, so that we have 10 UTIs in the intervention group and 21 in the control group," Craig said. "The number of events prevented is still 10, but it's clearly not the same thing as in the first scenario."
Even if you ignore these two problems with the research and accept that cranberry juice indeed reduced the risk of UTIs, the result the researchers found was actually pretty dismal in absolute terms: Drinking cranberry juice every day for 3.2 years averted one symptomatic UTI (and remember, that means not necessarily one that's confirmed through a lab test).
"That's a lot of sugar to take for over three years to prevent one episode — which probably isn’t a UTI anyway," Craig said.
The lead author of the study was Kevin Maki, who's now president and chief science officer of MB Clinical Research and Consulting, a firm that runs nutrition studies. (Maki says Ocean Spray contracted the clinical research firm BioFortis, where he was formerly an employee, to plan and execute the study published in AJCN.)
I asked him about how he reconciled his findings with the rest of the evidence. He noted that some 30 percent of women have symptoms but never test positive in a UTI test. And according to his study, those women would be helped by cranberry juice even if they didn't treat positive for an actual infection.
"There are several possible explanations for this finding. One is that cranberry is preventing infection," Maki said, "and another is that cranberry compounds are having anti-inflammatory effects that prevent symptoms when a mild infection occurs."
In Craig’s view, the juice may indeed help with symptoms, but he noted that not all symptoms are due to actual urinary infections — and the similar infection levels in the two groups would suggest cranberries made no difference on that front.
Why it’s so hard to trust industry-funded studies
This "landmark study" may not have finally proved that the red stuff is a UTI preventer. But it did remind us of why we need to be skeptical of industry-funded nutrition research.
"Industry-funded science tends to come out with results favorable to the sponsor’s interests, as this example shows," said Marion Nestle, an author and food policy and nutrition professor at New York University who studies conflict of interest in science. "Consciously or unconsciously, industry-funded scientists end up tending to put a positive spin on studies that otherwise might be negative or equivocal. It’s best to be even more skeptical than usual when looking at the results of industry-funded studies claiming miraculous health benefits."
Just consider this review of studies on sugary drinks. Independently funded studies tend to find a correlation between soda consumption and poor health outcomes. Studies funded by soda makers, by contrast, are less likely to find such correlations.
Or take this investigation of 206 publications on the health effects of milk, soft drinks, and fruit juices. Studies that were funded by beverage companies were four to eight times more likely to come to favorable conclusions about the health effects of those beverages.
To guard against this potential bias, Nestle added, at the very least the sponsor of a study should pass along the money and "have nothing whatsoever — no phone conversations, no emails, no suggestions, no reviews, nothing at all — to do with it or the investigators until the paper is published. Paying for the study already exerts influence. Anything beyond that is exerting undue influence."
The new cranberry research is a case study for what can happen when food companies exert that undue influence, and how much their science can confuse health matters.
