To quote recent headlines from Yahoo and Fox News, respectively, “Calling All Peanut Butter Lovers! The Sticky Snack May Boost Your Breast Health,” and “Girls who eat peanut butter may have improved breast health later in life.” Unfortunately, this study is merely the latest in a long string of scientific studies that are the subject of celebratory headlines but that turn out to be a mirage. There is no reason to think that teenage girls should now start eating more peanut butter.
The study, published in the journal Breast Cancer Research and Treatment, arose from a long-term project that has tracked over 9,000 females from 1996 (when they were all nine to 15 years old) through today. They received multiple surveys about their lives, diet, and health. After reviewing the responses, researchers concluded that women who ate more vegetable fat and protein (specifically including peanut butter) in 1996 – 98 seemed to have fewer non-cancerous lumps in more recent surveys.
There are at least two glaring problems with this study.
First, as we have all heard, correlation is not causation. The mere fact that some women who ate more peanut butter in 1996 – 98 had fewer benign lumps in 2010 in no way means that eating peanut butter protected them from benign lumps. Unlike a randomized trial, there could be any number of other unknown factors that are different about such women, and this study has no way of identifying what those factors might be.
At best, a study like this can only show correlation. But this did not stop the researchers from heavily insinuating that they had found causation. Indeed, they went even further: even though they had only studied benign cysts, they actually told news media in an official press release that “peanut butter could help reduce the risk of breast cancer.”
Second, the researchers have no business claiming to have found a valid correlation in the first place. In a large enough dataset with enough outcomes and variables, seeming correlations can often be due to randomness and measurement error. Think of the possibilities here: the survey given to girls in 1996 gave them five to seven choices to rank how often they consumed 141 types of food and drink, including everything from chocolate milk to alcohol to salami to celery to Twinkies. Then the survey given to women in 2010 asked them about 18 specific health conditions, plus an “other” box at the end. (Oddly enough, the women were specifically asked about cancer, so if there had been any correlation between peanut butter and actual cancer, we can be sure the researchers would have reported on that rather than on benign lumps.)
Taking all of those possibilities into account, the researchers had well over 10,000 possible correlations to choose from. But that’s not all: the surveys were given multiple times to the same girls. Thus, the researchers actually could have had their pick of 30,000+ correlations.
Unsurprisingly, some of the same scholars have already mined through this dataset to publish on other supposed associations between adolescent diet and adult benign breast disease. (Previous studies looked at benign breast disease versus dairy consumed at any of five different ages, family history along with alcohol consumption, and alcohol consumption by itself.)
But when there are this many possible correlations, some are going to seem significant just by chance. Chance is not correlation and definitely is not causation. Yet, even without a proven correlation, the researchers issued a press release suggesting that “peanut butter could help reduce the risk of breast cancer.” If they had made this overstatement in a press release about an FDA-regulated drug, they would have found themselves facing federal prosecution.
Sadly, this study is far from unique. Every day, we are bombarded with headlines telling us what to eat or not eat, but very few of these headlines are based on rigorous randomized trials. Indeed, Stanford researcher John Ioannidis famously found that the vast majority of food ingredients have been the subject of low-quality “studies” claiming that they either cause cancer or prevent it.
It is a shame that these sorts of studies are constantly foisted on us. As a society, we need better science and better science reporting.
