Saturday, February 4, 2017

Taking Apart Bad Diet Studies: Sugar-Free Edition

A few weeks ago, I wrote this take down of some recent commentary on the dangers of low-calorie sweeteners. That post is a good place to start on this subject because it covers the most common objections to consuming sweeteners like Splenda, and it's full of references to the peer-reviewed literature.

But this is a hot topic in nutrition these days, so I want to return to it now by picking apart one study in particular, which suggests that low-calorie sweeteners don't aid weight loss. The paper was published in PLoS One in November and its authors argue that

Low-calorie sweetener use is independently associated with heavier relative weight, a larger waist, and a higher prevalence and incidence of abdominal obesity suggesting that low-calorie sweetener use may not be an effective means of weight control.
Even if everything in the above sentence is correct, the authors have done nothing more than find an association (and a weak one at that, as we'll see) between low-calorie sweeteners and obesity. But there's much in the paper's conclusion that is problematic, so let's dive into the details and briefly explain why.


To perform this study, the researchers utilized body composition and diet data from the Baltimore Longitudinal Study of Aging, which you can read more about here. The present study includes 1,454 men and women from the BLSA who are at least 20 years old. Their height, weight and sweetener use were assessed at several points between 1984 and 2012, and the study participants were followed for an average of 10 years.

There's nothing especially challenging about measuring a person's weight or height, but assessing how much food they eat is really hard to do. So how did the BLSA acquire the numbers on sweetener consumption?

Dietary intake was assessed using a 7-day dietary record
Oh, boy. Food surveys. The details on the collection methods are here. But in essence, the BLSA researchers ask people to track what they eat over 7-day periods, selecting from 41 different food groups, including low-calorie sweeteners.

The authors of the PLoS paper followed 1,455 participants from this study. They compared those who reported consuming sweeteners to those who didn't to see which group fared better. Simple, right? Nope.

Food Frequency Surveys Are Garbage

We've known for years now that surveys of what people eat are useless. They consistently under report consumption of certain food groups and over report consumption of others, for example. We know this because studies have been done in which people are fed carefully measured diets then allowed to report back what they ate on food frequency surveys.

In this study from 2000, people wildly misreported what they were consuming. They did so poorly that the authors concluded that "Our data indicate that the food-frequency questionnaire may be unreliable and inadequate for assessing ... macronutrient intakes." Mind you, this study involved only 19 subjects and lasted just six weeks. The BLSA data, so far, cover more than two decades and include more than a thousand individuals! If 19 people can't accurately report what they ate over a month and a half, there's no way 1,500 people can do the same over 20 plus years.

These questionnaires are so awful, in fact, that this 2005 article in Cancer Epidemiology says they've created a crisis in nutrition research, primarily because human memory is faulty after only a few days. People can remember what they ate yesterday, maybe, but by the end of the week, they begin substituting what they actually ate with "general knowledge about foods, most probably based on beliefs (even hopes) about one's usual or characteristic diet."

These general limitations on food frequency questionnaires are enough to sink any diet study that relies on them. But let's look at a specific issue with this study of low-calorie sweetener consumption.

Sweetener Use Varied Over Time

Clinical trials can be a powerful method for discovering scientific facts, because you can control different variables to see how they affect the outcome of the trial. You could, for instance, put people on identical diets but feed only one group artificial sweeteners and see what happens. The PLoS researchers tried to mimic this method, but they couldn't because "[l]ow-calorie sweetener use by participants fluctuated over time..."

The authors describe the statistical methods they employed to account for this variable, in which they try to compare how much sweetener people say they are consuming to their weight and waist circumference throughout the study. But if people are changing their consumption patterns throughout the study, there's no meaningful way to track how sweeteners are affecting their health.

As Gary Taubes has pointed out, there is often a time delay between a dietary intervention itself and the effect that the intervention exerts on your health. Maybe sweeteners do stunt weight loss. Or maybe they encourage weight loss, if you replace three regular sodas with three diet sodas everyday for three months. The point is, you have to accurately track how much of a sweetener people consume over a given period of time to test either hypothesis in a meaningful way.

What did they find?

So with all these caveats about statistics out of the way, what did this study find? If you look at table 1 in the paper, you can see that the group who consumed sweeteners had a BMI of 26.4 at baseline. This number increased to 26.8 by the end of the study, a whopping .4 increase over the course of a decade! Depending on your height and lean muscle mass, this increase in BMI represents probably less than 1/2 pound of body fat for most people. Even if every single data point in this study is accurate, cutting artificial sweeteners out of your diet will contribute a minuscule amount of weight loss to your overall goal to get slim and sexy.

Parenthetically, both groups in the study consumed more than 230 grams of carbohydrate on a daily basis! And approximately half of the people in both groups had poor glucose tolerance, which almost certainly has something to do with their ability to lose weight. Studying the obesity-promoting effects of low-calorie sweeteners in this context doesn't make much sense, because everyone is eating a fattening diet. Whatever influence sweeteners have is probably dwarfed by the overall effects of the diet.


In summary then, there's way too little reliable data available and far too much modeling going on in this study, especially when there is better research available. The worst part is that you can get the result you're looking for even if your model is bad! I suspect that that's what happened in this case, and that's why I remain unconvinced that low-calorie sweeteners have anything to do with weight gain.

Wednesday, February 1, 2017

Lousy Study Links E-cigarette Use To Heart Disease

E-cigarettes have been widely used for going on 15 years now. In that time, multiple studies have demonstrated that the smoking alternatives are far safer than cigarettes, and much more effective for smoking cessation than nicotine gum or patches.

So with a relatively impressive body of research available to justify "vaping" - the cool kid term for smoking an e-cig - any study that contradicts this ever-growing consensus should ping your poser radar.

Why? Because the new study is probably garbage, as is this new paper published in the Journal of the American Medical Association. The study found that e-cigarette users, "vapers," experienced a temporary shift in heart rate variability also seen in regular cigarette smokers.

Naturally, the conclusion drawn from these results is that e-cigs may cause the same kinds of damage to the heart that tobacco cigarettes do. That's nonsense for a number of reasons, the most important of which is explained by Steve Milloy of

Transient physiological effects do not translate into little-understood heart disease which is a natural long-term degenerative process.
Trying to draw conclusions about the long-term effects of vaping based on the results of such a limited study is useless. It's like arguing that a single cheeseburger will make you fat because it temporarily spikes your insulin immediately after you eat it, which coincidentally was also argued this week.

Anyway, heart disease takes decades to develop. Analyzing heart rate variability and oxidative stress one time in 40 people, only 23 of whom were e-cigarette users, is beyond useless. This is why governments spend millions of dollars financing decades-long studies of people with heart disease or who are at risk of developing the condition. It really takes that long to understand what actually causes chronic disease.

Let's briefly put this measly, poorly-designed study in the context of our broader understanding of e-cigarette science. Public Health England said just last fall that e-cigs are 95% less harmful than traditional cigarettes. That review covered dozens of studies published mostly in the last decade, and nearly every one of them comes down on the side of e-cigarettes being a safe alternative to tobacco.

These studies cover everything from lung cancer risk to smoking cessation and secondhand risk to bystanders. Yet we're all supposed to gasp in horror because of elevated adrenaline levels in 23 people, one time.


Monday, January 30, 2017

Will Trump Be Bad for Science? His Cabinet Appointments Suggest Not

As President Trump settles into office and begins filling cabinet positions, we're starting to get a sense of how the new administration will influence science policy in America.

American Council on Science and Health president Hank Campbell has some interesting thoughts on Trump's cabinet picks as they relate to science:

When it comes to science and health, someone has clearly thought about these nominees, and they are also in line with the stated goals that got the President elected. These selections are certainly not anti-science, unless you are framing science through political or social goals.
A few quick thoughts:

Campbell rightly criticizes the EPA for the propagandizing they've engaged in over the last eight years, but acknowledges that they've also done some good research, especially on fracking in my opinion. Since Trump has been vocal about the importance of increased energy production and ordered the EPA to tone down their environmentalist preaching on twitter, I anticipate the agency will improve under the new president.

The Department of Energy will no longer preach the blessings of alternative energy, nor will they force all of us to pay for it, under the direction of former Texas governor Rick Perry. Greenpeace is breaking out in hives. Snort.

Francis Collins is staying on as director of the National Institutes of Health. He's a great science communicator and a great researcher besides. He's also been a voice of reason in the religion vs. science debate, mainly because he's convincingly argued that the two aren't mutually exclusive.

It's still early. But so far, so good. Trump could have done a lot worse with these cabinet picks.