David Baxter PhD
Late Founder
Surviving Bad Nutrition Headlines in the Age of BS
SciBabe Blog
January 16, 2018
It seems increasingly harder to nail down a fact lately. If you have a study that shows evidence of something, does that make it a fact? What if there are three websites that you trust that have a compelling infographic on it loaded with statistics, does that make it factual? Do headlines about studies always represent unbiased, irrefutable facts? And how do can you tell if something isn’t factual if it’s not your field of expertise? Nutrition headlines disagree with each other often. If you’re not an expert, how do you start checking for facts in nutrition through headlines?
There is a never-ending wellspring of bullshit nutrition headlines. With so much confusion about what’s right to eat, who the experts are, and how to maintain your health via diet, almost anything goes in the internet landscape of misinformation. I don’t blame anyone for having a lot of questions. I’m ready to pack it in and have a radish with a side of gin. Balance or something.
I’m all out of radishes, so let’s have a talk on how to preen the facts from the headlines.
What Are Some Signs of Bullshit Nutrition Headlines?
If you’re just trying to figure out what’s healthy and what’s not, which list do you believe? They’re not telling you any clear cut terrible advice, but can you be sure if it’s factual advice either? They say things like “studies say,” but they’re not really linking to studies. And how can all of these things be killing you on one list and perfectly fine on another? It’s breakfast, I’ve got to cover up my under-eye circles and get out the door, not figure out if blueberries are secret assassins.
There’s no clear delineation of “science and pseudoscience” because, in many cases, pseudoscientific thought relies on science to deploy faulty reasoning. From the nuances of declaring a diet healthy or unhealthy to explaining why a diet makes you lose weight, treat a disease, or sometimes just as importantly not treat a disease, there are many different flavors of misinformation about what your diet can do to you. There are a few basic questions to ask when you first see a headline to start thinking about if it may or may not be accurate:
Does the claim make sense given what we know about the world?
Is the claim overblown given the provided evidence?
Has the person making the claim tried to prove or disprove it?
How Do Bad Nutrition Headlines Make Their Way Across the Internet?
Let’s have a look at a few ways these have landed on your newsfeed and then in my inbox. There are a few ways that just plain inaccurate information can seep into our collective subconscious. Let’s explore a few of them!
P-Hacking: Pick Enough Cherries, and Chocolate is a Weight Loss Supplement
What happens when someone puts purposefully misleading information out on to the internet? Enter Bohannon and the chocolate weight loss study.
John Bohannon, a scientist with a bit of a sense of humor (I can’t say I’m not a fan), went on a small scale dietary study with 18 participants. All were put on a low carb, controlled calorie diet, but half did this and included a small piece of chocolate daily within their caloric allowance. The researchers tracked a number of variables, and then they P-hacked the hell out of it. P-hacking is a term that means “they ****ed with the data until it was useful.”
Then, they put the data in a pay-for-play journal.
The plan worked a little too well and the story went viral. Unfortunately, they had to explain to disappointed chocolate lovers that no, at this point there is not evidence to suggest chocolate that chocolate can cause weight loss. You’re free to attempt to replicate the experiment just because… chocolate.
"We Might Have Had a Miscommunication"
Sometimes, ensuring that science is communicated accurately from the scientist to the university’s press office and then out to the public seems to operate like a really bad game of telephone. Inevitably, someone has to explain that no, they did not say to put the log in the vegetable quiche (I said put the dog on the extendable leash, I swear). Or equally ludicrous, that a glass of red wine is not the same as an hour of exercise.
Jason Dyck, a researcher at the university of Alberta, looked into the effects of resveratrol, a component of things like grapes (and red wine) for people to maximize exercise benefits for people with limited mobility. It was found to have benefits.
In mice.
In a supplement.
In a quantity of resveratrol that you’d generally find in 100-1000 bottles of wine.
“A lot of times when you work away in your lab you hope that people read your studies and that it makes an impact. When you see all your hard work distilled down to one phrase that isn’t correct it’s a little disheartening,” Dyck said in an interview. This all should make you question, how often are nutritional studies taken out of context, and how do you safeguard yourself from having bad science filtered down to you?
And more importantly, if you see someone binging on chocolate and wine, should we just cut them some slack? Maybe they’re trying to drop some weight and fit in a quick workout.
The "Linked to" Problem: Correlation is Not Causation
We’re going to have to talk about Nicholas Cage. Because he’s been up to some ****.
In the first chart, we have Nicolas Cage films correlated with deaths by drowning in a swimming pool.
In the second, we have Nicholas Cage films correlated with men slipping or tripping to their deaths.
Oh, those deaths could have correlated with anything else, but no, according to tylervigen.com and his spurious correlations, they correlated with the amount of work that Nicholas Cage was putting out. And I don’t know what it was man. I don’t know if it’s the amount of Nicholas CAAAAAAAAGE in the atmosphere, I don’t know if it’s that he’s beaming psychic beams of Nicholas Cage right into your mind making you trip and fall to your death, I don’t know if it’s that when he’s working, people aren’t paying attention to his more nefarious deeds…
But I know he’s a wizard. I’ve done my research.
(Ahem.)
Correlations aren’t a bad place to start in science. You see two things happening at the same time, it makes sense to investigate if there’s a causative link between variables. That’s how research starts. But bad reporting muddies the waters on these frequently. Especially with…
Don’t Touch My Goddamn Diet Coke
You’ve heard the news. Low calorie sweeteners are kicking puppies, disappointing their parents, and they ghosted you on Tinder last week. Jerks.
The good news is that before a consumer ever hears about a new low calorie sweetener, there’s a huge amount of study that goes into ensuring that they’re safe. Over 100 safety studies are performed on any low calorie sweetener before they’re allowed to hit market. If you like being scared of your food, you’ll need a new hobby; these are exonerated with sweet sweet science, but sometimes communicating that has gotten in the way.
Remember, all the methods that we use to fact check bad science in other types of science apply to these too, and you can safeguard yourself from it with a little practice.
Take the data from the Framingham Heart Study, published in the journals Alzheimer’s & Dementia and Stroke, and reported in EurekAlert! Initially it was reported that, even though consumption of regular soda showed a slight increase in the incident of Alzheimer’s, that diet soda consumption seemed to show a higher increase in the incident of dementia and stroke. But Business Insider noted several problems with this, chief among them that this was an observational study without controls. When a researcher studies a group over a long period of time they have so many more dietary and lifestyle factors than just sweetened or diet soda to control for. To exclude the possibility of all other correlations and present their findings as though this correlation was much firmly causal seems premature, at best. It puts any plausible association to diet soda in doubtful, especially given that they’re sweetened with mixes of low calorie sweeteners that are chemically dissimilar from one diet soda to the next. Furthermore, many of the people surveyed had or developed diabetes over the course of the study and some issues with neurodegenerative diseases in diabetics are believed to be due to the abnormal processing of insulin, not their dietary intake. The factors here are much more complicated than just the presence of a small amount of one substance.
Observational studies are good for some things in epidemiology, but not necessarily for directly assigning the causal relationship of a disease to one substance.
What else can these magically powerful substances do? They’ll make you glucose intolerant by changing your gut bacteria, a study published in Nature says. This is allegedly accomplished by altering the gut microbiota, but there are some glaring errors with both this study and the conclusion. Somehow aspartame, sucralose, and saccharin, which you would never group together chemically for any reason other than “you put this on your tongue,” were handed the blame for changes to gut microbes. But research into the microbiome is in its infancy. We still haven’t figured out exactly what a perfect gut microbiome looks like, and all dietary changes alter it. Could low calorie sweeteners alter it? Possibly, but so does everything else you eat. Given how small of a fraction of a percent of your diet low calorie sweeteners are (one hopes), at this point the evidence doesn’t support that they’re the cause of microbiome issues.
Does this all mean that you should gorge yourself on diet soda, endless packets of low calorie sweetener, and ignore all new research the microbiome? Obviously. But for me, I’m going to keep in mind that studies like these are in the minority and the balance of well designed studies show that low calorie sweeteners, from stevia to sucralose, are safe. I can eat a balanced diet and occasionally swap out sugar for a low calorie sweetener in my iced coffee (as always, I take it with one raw sugar and three Splenda), and do so knowing it just makes my mornings a little sweeter.
Scientific Breakthroughs via Dr. Oz and Dr. Mommy Blogger?
You find yourself reading a blogger who claims that they alone have found the secret breakthrough to a hard to treat health issue. Diabetes. Cancer. Obesity.
That is freaking rad, right? It’s the real ****. It’s the untainted science. There aren’t any dangerous chemicals, no corporate control, a mom made this and they have… proof or something? Well seeing can be believing, right?
Well, let’s at least look at some of the pictures and how they can do them. These were from my friend Jennifer. She took these pictures the same day to demonstrate that before/after pictures were bs. In one picture, she stuck out her stomach and put lighting at its least flattering. Then she changed the lighting and the pose. Bam! Fat melted.
And then she found the pictures being used for a diet pill ad.
Trust, but verify. Even from your BFF’s blog, and especially from Dr. Oz.
What Exactly Did That Study Say?
One phrase I see frequently is "studies say". And then upon examination of the article, I either can’t find the study, or it doesn’t really reflect the study at all. So I went hunting for the phrase "studies say" on the google machine, and found a few prime examples. "The season you were born in dictates your life". Really? Well, there was a study, and it can… affect your dopamine and serotonin levels.
So this article made the leap that since your dopamine and serotonin levels regulate your emotions, that makes your personality, so the season you’re born in is responsible for everything. Just ignore genetics, trauma, mental illness, experience, interests, any of that. Season. We’re living by headlines and whatever we find on twitter for science now!
Well ****. Didn’t we just read a headline saying that red wine was the equivalent of an hour of exercise? Science just can’t make up its mind. I’m going to go back to divination with tea leaves now.
This article was based on a real study – which I had to dig into another article to find. Out of 210,252 participants over the course of 18 years, 1,374 cases of melanoma occurred. That’s barely 1% of the population. And there was a small increase in the risk in melanoma associated with white wine drinkers.
Epidemiological studies present a challenge to parse through. You’re looking for a needle in a mountain of rhino turd. The takeaway shouldn’t be to dump out your white wine or anything like this from one such headline, but to look at any pre-existing health factors and behaviors and consider your individual risk.
Fact check. And find the source study.
Lastly, I will be fighting rumors about apple cider vinegar until the apple cider vinegar supplies on the planet run out. It’s true that apple cider vinegar can slow the adsorption of blood sugar slightly, but within two hours, no matter what, you will get the full hit of blood sugar, regardless of how much vinegar, ****, or whatever you’ve pumped yourself full of.
So Head Out Onto the Internet a Bit Better Armed to Find Facts on Nutrition, and Remember…
Be open to new ideas, but make sure that they’re actually new and not something we already have stacks of evidence telling you to save your time and money. New ideas shouldn’t cause alarm, they should cause investigation. No matter how pretty the meme is, thirty years of studies will always outweigh a week old viral meme that just hit your Facebook feed. If you want something to be true, question it harder. Most hypothesis will be wrong, act accordingly.
Now go fact check the internet. It’s so easy a fifth grader could do it.
SciBabe Blog
January 16, 2018
It seems increasingly harder to nail down a fact lately. If you have a study that shows evidence of something, does that make it a fact? What if there are three websites that you trust that have a compelling infographic on it loaded with statistics, does that make it factual? Do headlines about studies always represent unbiased, irrefutable facts? And how do can you tell if something isn’t factual if it’s not your field of expertise? Nutrition headlines disagree with each other often. If you’re not an expert, how do you start checking for facts in nutrition through headlines?
There is a never-ending wellspring of bullshit nutrition headlines. With so much confusion about what’s right to eat, who the experts are, and how to maintain your health via diet, almost anything goes in the internet landscape of misinformation. I don’t blame anyone for having a lot of questions. I’m ready to pack it in and have a radish with a side of gin. Balance or something.
I’m all out of radishes, so let’s have a talk on how to preen the facts from the headlines.
What Are Some Signs of Bullshit Nutrition Headlines?
If you’re just trying to figure out what’s healthy and what’s not, which list do you believe? They’re not telling you any clear cut terrible advice, but can you be sure if it’s factual advice either? They say things like “studies say,” but they’re not really linking to studies. And how can all of these things be killing you on one list and perfectly fine on another? It’s breakfast, I’ve got to cover up my under-eye circles and get out the door, not figure out if blueberries are secret assassins.
There’s no clear delineation of “science and pseudoscience” because, in many cases, pseudoscientific thought relies on science to deploy faulty reasoning. From the nuances of declaring a diet healthy or unhealthy to explaining why a diet makes you lose weight, treat a disease, or sometimes just as importantly not treat a disease, there are many different flavors of misinformation about what your diet can do to you. There are a few basic questions to ask when you first see a headline to start thinking about if it may or may not be accurate:
Does the claim make sense given what we know about the world?
- No one food really cures, prevents, treats, or manages all diseases for the entire population. Before you say “but my diet is nutritionally dense/loaded with superfoods/makes you a level 9000 vegan/only has foods that a caveman ate,” I guarantee you, it not only doesn’t cure everything, there is some medical condition out there for which your normally really healthy diet could be lethal. Be wary of claims from small websites that claim they have discovered a medical breakthrough that solves all disease, because nothing causes or solves all disease. Furthermore, a headline gives itself away if it claims a food -that you generally think of as safe- should “never” or “always” be eaten by healthy people because it’s so blatantly out of touch with what you know about the world. It might be a huge breakthrough, but it should at least make you think twice about the claim.
Is the claim overblown given the provided evidence?
- Is a claim about a food’s safety based on a parent’s observation of what happened when they removed or added something from their child’s diet once? Well then that’s compelling evidence that the parent should talk to their doctor or perhaps a more mathematically inclined friend about statistically significant sampling, and stop dispensing medical advice.
Has the person making the claim tried to prove or disprove it?
- Odds are if someone is giving advice based on anecdotes from their own health, they haven’t run trials on the whole population. In that vein, a component of diet being bad for someone with an allergy or an intolerance does not make it bad for everyone. You have a lactose intolerance, you avoid lactose. You don’t declare war on brie.
How Do Bad Nutrition Headlines Make Their Way Across the Internet?
Let’s have a look at a few ways these have landed on your newsfeed and then in my inbox. There are a few ways that just plain inaccurate information can seep into our collective subconscious. Let’s explore a few of them!
P-Hacking: Pick Enough Cherries, and Chocolate is a Weight Loss Supplement
What happens when someone puts purposefully misleading information out on to the internet? Enter Bohannon and the chocolate weight loss study.
John Bohannon, a scientist with a bit of a sense of humor (I can’t say I’m not a fan), went on a small scale dietary study with 18 participants. All were put on a low carb, controlled calorie diet, but half did this and included a small piece of chocolate daily within their caloric allowance. The researchers tracked a number of variables, and then they P-hacked the hell out of it. P-hacking is a term that means “they ****ed with the data until it was useful.”
Then, they put the data in a pay-for-play journal.
The plan worked a little too well and the story went viral. Unfortunately, they had to explain to disappointed chocolate lovers that no, at this point there is not evidence to suggest chocolate that chocolate can cause weight loss. You’re free to attempt to replicate the experiment just because… chocolate.
"We Might Have Had a Miscommunication"
Sometimes, ensuring that science is communicated accurately from the scientist to the university’s press office and then out to the public seems to operate like a really bad game of telephone. Inevitably, someone has to explain that no, they did not say to put the log in the vegetable quiche (I said put the dog on the extendable leash, I swear). Or equally ludicrous, that a glass of red wine is not the same as an hour of exercise.
Jason Dyck, a researcher at the university of Alberta, looked into the effects of resveratrol, a component of things like grapes (and red wine) for people to maximize exercise benefits for people with limited mobility. It was found to have benefits.
In mice.
In a supplement.
In a quantity of resveratrol that you’d generally find in 100-1000 bottles of wine.
“A lot of times when you work away in your lab you hope that people read your studies and that it makes an impact. When you see all your hard work distilled down to one phrase that isn’t correct it’s a little disheartening,” Dyck said in an interview. This all should make you question, how often are nutritional studies taken out of context, and how do you safeguard yourself from having bad science filtered down to you?
And more importantly, if you see someone binging on chocolate and wine, should we just cut them some slack? Maybe they’re trying to drop some weight and fit in a quick workout.
The "Linked to" Problem: Correlation is Not Causation
We’re going to have to talk about Nicholas Cage. Because he’s been up to some ****.
In the first chart, we have Nicolas Cage films correlated with deaths by drowning in a swimming pool.
In the second, we have Nicholas Cage films correlated with men slipping or tripping to their deaths.
Oh, those deaths could have correlated with anything else, but no, according to tylervigen.com and his spurious correlations, they correlated with the amount of work that Nicholas Cage was putting out. And I don’t know what it was man. I don’t know if it’s the amount of Nicholas CAAAAAAAAGE in the atmosphere, I don’t know if it’s that he’s beaming psychic beams of Nicholas Cage right into your mind making you trip and fall to your death, I don’t know if it’s that when he’s working, people aren’t paying attention to his more nefarious deeds…
But I know he’s a wizard. I’ve done my research.
(Ahem.)
Correlations aren’t a bad place to start in science. You see two things happening at the same time, it makes sense to investigate if there’s a causative link between variables. That’s how research starts. But bad reporting muddies the waters on these frequently. Especially with…
Don’t Touch My Goddamn Diet Coke
You’ve heard the news. Low calorie sweeteners are kicking puppies, disappointing their parents, and they ghosted you on Tinder last week. Jerks.
The good news is that before a consumer ever hears about a new low calorie sweetener, there’s a huge amount of study that goes into ensuring that they’re safe. Over 100 safety studies are performed on any low calorie sweetener before they’re allowed to hit market. If you like being scared of your food, you’ll need a new hobby; these are exonerated with sweet sweet science, but sometimes communicating that has gotten in the way.
Remember, all the methods that we use to fact check bad science in other types of science apply to these too, and you can safeguard yourself from it with a little practice.
Take the data from the Framingham Heart Study, published in the journals Alzheimer’s & Dementia and Stroke, and reported in EurekAlert! Initially it was reported that, even though consumption of regular soda showed a slight increase in the incident of Alzheimer’s, that diet soda consumption seemed to show a higher increase in the incident of dementia and stroke. But Business Insider noted several problems with this, chief among them that this was an observational study without controls. When a researcher studies a group over a long period of time they have so many more dietary and lifestyle factors than just sweetened or diet soda to control for. To exclude the possibility of all other correlations and present their findings as though this correlation was much firmly causal seems premature, at best. It puts any plausible association to diet soda in doubtful, especially given that they’re sweetened with mixes of low calorie sweeteners that are chemically dissimilar from one diet soda to the next. Furthermore, many of the people surveyed had or developed diabetes over the course of the study and some issues with neurodegenerative diseases in diabetics are believed to be due to the abnormal processing of insulin, not their dietary intake. The factors here are much more complicated than just the presence of a small amount of one substance.
Observational studies are good for some things in epidemiology, but not necessarily for directly assigning the causal relationship of a disease to one substance.
What else can these magically powerful substances do? They’ll make you glucose intolerant by changing your gut bacteria, a study published in Nature says. This is allegedly accomplished by altering the gut microbiota, but there are some glaring errors with both this study and the conclusion. Somehow aspartame, sucralose, and saccharin, which you would never group together chemically for any reason other than “you put this on your tongue,” were handed the blame for changes to gut microbes. But research into the microbiome is in its infancy. We still haven’t figured out exactly what a perfect gut microbiome looks like, and all dietary changes alter it. Could low calorie sweeteners alter it? Possibly, but so does everything else you eat. Given how small of a fraction of a percent of your diet low calorie sweeteners are (one hopes), at this point the evidence doesn’t support that they’re the cause of microbiome issues.
Does this all mean that you should gorge yourself on diet soda, endless packets of low calorie sweetener, and ignore all new research the microbiome? Obviously. But for me, I’m going to keep in mind that studies like these are in the minority and the balance of well designed studies show that low calorie sweeteners, from stevia to sucralose, are safe. I can eat a balanced diet and occasionally swap out sugar for a low calorie sweetener in my iced coffee (as always, I take it with one raw sugar and three Splenda), and do so knowing it just makes my mornings a little sweeter.
Scientific Breakthroughs via Dr. Oz and Dr. Mommy Blogger?
You find yourself reading a blogger who claims that they alone have found the secret breakthrough to a hard to treat health issue. Diabetes. Cancer. Obesity.
That is freaking rad, right? It’s the real ****. It’s the untainted science. There aren’t any dangerous chemicals, no corporate control, a mom made this and they have… proof or something? Well seeing can be believing, right?
Well, let’s at least look at some of the pictures and how they can do them. These were from my friend Jennifer. She took these pictures the same day to demonstrate that before/after pictures were bs. In one picture, she stuck out her stomach and put lighting at its least flattering. Then she changed the lighting and the pose. Bam! Fat melted.
And then she found the pictures being used for a diet pill ad.
Trust, but verify. Even from your BFF’s blog, and especially from Dr. Oz.
What Exactly Did That Study Say?
One phrase I see frequently is "studies say". And then upon examination of the article, I either can’t find the study, or it doesn’t really reflect the study at all. So I went hunting for the phrase "studies say" on the google machine, and found a few prime examples. "The season you were born in dictates your life". Really? Well, there was a study, and it can… affect your dopamine and serotonin levels.
So this article made the leap that since your dopamine and serotonin levels regulate your emotions, that makes your personality, so the season you’re born in is responsible for everything. Just ignore genetics, trauma, mental illness, experience, interests, any of that. Season. We’re living by headlines and whatever we find on twitter for science now!
Well ****. Didn’t we just read a headline saying that red wine was the equivalent of an hour of exercise? Science just can’t make up its mind. I’m going to go back to divination with tea leaves now.
This article was based on a real study – which I had to dig into another article to find. Out of 210,252 participants over the course of 18 years, 1,374 cases of melanoma occurred. That’s barely 1% of the population. And there was a small increase in the risk in melanoma associated with white wine drinkers.
Epidemiological studies present a challenge to parse through. You’re looking for a needle in a mountain of rhino turd. The takeaway shouldn’t be to dump out your white wine or anything like this from one such headline, but to look at any pre-existing health factors and behaviors and consider your individual risk.
Fact check. And find the source study.
Lastly, I will be fighting rumors about apple cider vinegar until the apple cider vinegar supplies on the planet run out. It’s true that apple cider vinegar can slow the adsorption of blood sugar slightly, but within two hours, no matter what, you will get the full hit of blood sugar, regardless of how much vinegar, ****, or whatever you’ve pumped yourself full of.
So Head Out Onto the Internet a Bit Better Armed to Find Facts on Nutrition, and Remember…
Be open to new ideas, but make sure that they’re actually new and not something we already have stacks of evidence telling you to save your time and money. New ideas shouldn’t cause alarm, they should cause investigation. No matter how pretty the meme is, thirty years of studies will always outweigh a week old viral meme that just hit your Facebook feed. If you want something to be true, question it harder. Most hypothesis will be wrong, act accordingly.
Now go fact check the internet. It’s so easy a fifth grader could do it.
