Saturday, February 4, 2017

Taking Apart Bad Diet Studies: Sugar-Free Edition

A few weeks ago, I wrote this take down of some recent commentary on the dangers of low-calorie sweeteners. That post is a good place to start on this subject because it covers the most common objections to consuming sweeteners like Splenda, and it's full of references to the peer-reviewed literature.

But this is a hot topic in nutrition these days, so I want to return to it now by picking apart one study in particular, which suggests that low-calorie sweeteners don't aid weight loss. The paper was published in PLoS One in November and its authors argue that

Low-calorie sweetener use is independently associated with heavier relative weight, a larger waist, and a higher prevalence and incidence of abdominal obesity suggesting that low-calorie sweetener use may not be an effective means of weight control.
Even if everything in the above sentence is correct, the authors have done nothing more than find an association (and a weak one at that, as we'll see) between low-calorie sweeteners and obesity. But there's much in the paper's conclusion that is problematic, so let's dive into the details and briefly explain why.


To perform this study, the researchers utilized body composition and diet data from the Baltimore Longitudinal Study of Aging, which you can read more about here. The present study includes 1,454 men and women from the BLSA who are at least 20 years old. Their height, weight and sweetener use were assessed at several points between 1984 and 2012, and the study participants were followed for an average of 10 years.

There's nothing especially challenging about measuring a person's weight or height, but assessing how much food they eat is really hard to do. So how did the BLSA acquire the numbers on sweetener consumption?

Dietary intake was assessed using a 7-day dietary record
Oh, boy. Food surveys. The details on the collection methods are here. But in essence, the BLSA researchers ask people to track what they eat over 7-day periods, selecting from 41 different food groups, including low-calorie sweeteners.

The authors of the PLoS paper followed 1,455 participants from this study. They compared those who reported consuming sweeteners to those who didn't to see which group fared better. Simple, right? Nope.

Food Frequency Surveys Are Garbage

We've known for years now that surveys of what people eat are useless. They consistently under report consumption of certain food groups and over report consumption of others, for example. We know this because studies have been done in which people are fed carefully measured diets then allowed to report back what they ate on food frequency surveys.

In this study from 2000, people wildly misreported what they were consuming. They did so poorly that the authors concluded that "Our data indicate that the food-frequency questionnaire may be unreliable and inadequate for assessing ... macronutrient intakes." Mind you, this study involved only 19 subjects and lasted just six weeks. The BLSA data, so far, cover more than two decades and include more than a thousand individuals! If 19 people can't accurately report what they ate over a month and a half, there's no way 1,500 people can do the same over 20 plus years.

These questionnaires are so awful, in fact, that this 2005 article in Cancer Epidemiology says they've created a crisis in nutrition research, primarily because human memory is faulty after only a few days. People can remember what they ate yesterday, maybe, but by the end of the week, they begin substituting what they actually ate with "general knowledge about foods, most probably based on beliefs (even hopes) about one's usual or characteristic diet."

These general limitations on food frequency questionnaires are enough to sink any diet study that relies on them. But let's look at a specific issue with this study of low-calorie sweetener consumption.

Sweetener Use Varied Over Time

Clinical trials can be a powerful method for discovering scientific facts, because you can control different variables to see how they affect the outcome of the trial. You could, for instance, put people on identical diets but feed only one group artificial sweeteners and see what happens. The PLoS researchers tried to mimic this method, but they couldn't because "[l]ow-calorie sweetener use by participants fluctuated over time..."

The authors describe the statistical methods they employed to account for this variable, in which they try to compare how much sweetener people say they are consuming to their weight and waist circumference throughout the study. But if people are changing their consumption patterns throughout the study, there's no meaningful way to track how sweeteners are affecting their health.

As Gary Taubes has pointed out, there is often a time delay between a dietary intervention itself and the effect that the intervention exerts on your health. Maybe sweeteners do stunt weight loss. Or maybe they encourage weight loss, if you replace three regular sodas with three diet sodas everyday for three months. The point is, you have to accurately track how much of a sweetener people consume over a given period of time to test either hypothesis in a meaningful way.

What did they find?

So with all these caveats about statistics out of the way, what did this study find? If you look at table 1 in the paper, you can see that the group who consumed sweeteners had a BMI of 26.4 at baseline. This number increased to 26.8 by the end of the study, a whopping .4 increase over the course of a decade! Depending on your height and lean muscle mass, this increase in BMI represents probably less than 1/2 pound of body fat for most people. Even if every single data point in this study is accurate, cutting artificial sweeteners out of your diet will contribute a minuscule amount of weight loss to your overall goal to get slim and sexy.

Parenthetically, both groups in the study consumed more than 230 grams of carbohydrate on a daily basis! And approximately half of the people in both groups had poor glucose tolerance, which almost certainly has something to do with their ability to lose weight. Studying the obesity-promoting effects of low-calorie sweeteners in this context doesn't make much sense, because everyone is eating a fattening diet. Whatever influence sweeteners have is probably dwarfed by the overall effects of the diet.


In summary then, there's way too little reliable data available and far too much modeling going on in this study, especially when there is better research available. The worst part is that you can get the result you're looking for even if your model is bad! I suspect that that's what happened in this case, and that's why I remain unconvinced that low-calorie sweeteners have anything to do with weight gain.

No comments:

Post a Comment