An exercise in vitriol rather than sound journalism

Gary Taubes from the March 2003 issue

(Page 3 of 6)

Fumento then goes on to accuse me of shoving aside "decades of published, controlled randomized clinical trials comparing nutrient intake and weight loss." Regrettably, Fumento seems unclear on the concept of a "controlled randomized clinical trial," an unfortunate failing for a wannabe medical journalist. The relevant clue and the salient point is that there were no such trials. There are decades of trials looking at the effects of diet on weight loss, but they are for the most part, neither randomized nor controlled, and there are decades of observational studies trying to compare what people say they eat to how much they weigh. The latter are meaningless in this context and would require more time than I prefer to spend to explain why.

What Fumento does is turn for support to an April 2002 article in the Journal of the American Dietetic Association <(JADA) that was, he says, "`a review of all studies identified' that looked at diet nutrient composition and weight loss. It found over 200." It is this article he cites as one leg of a three-legged "crushing" mass of evidence rejecting the efficacy of Atkins-like low carbohydrate diets. On his own Web site, Fumento refers to this article repeatedly as evidence that he has done his library research, as though he himself personally read all 200 articles.

If Fumento even read this particular review article, however, he does a poor job of demonstrating that fact. For starters, it was not published in April 2002, but in April 2001. Secondly, while the authors of the study refer to the "more than 200 individual studies" that they allegedly included, they reference only 58, thus begging the question of what happened to the other 142+. Thirdly, the study was written by 4 employees and former acting undersecretary of the USDA, an organization that has been, bar none, the foremost advocate of high-carb, low-fat diets in America for 25 years.

This raises the issue of whether the authors might be tempted to bias their review to support the USDA's long-standing public position on the health-benefits of fat-reduced diets. It doesn't suggest they did, but it does beg the question of why Fumento readily intimates that a researcher who received funding for a diet trial from Atkins's Foundation would assuredly slant his results to please Atkins, while Fumento treats these government employees as somehow inherently beyond suspicion of the same kind of bias. Who knows. Maybe they might be motivated to please the bureaucrats who pay their salaries? Maybe their goal, consciously or subconsciously, was to get an article into print that appeared to support the agency's ubiquitous low-fat diet advice? Either way, it seems only fair that Fumento hold government employees up to the same level of malicious insinuation to which he holds private citizens. Why he doesn't escapes me.

Finally, biased or not, the USDA authors do happen to come to conclusions, which Fumento promptly quotes, that are not supported by the data from the studies they reference. In some points they simply err on the side of low-fat diets and against the low-carb diets. For instance, the authors state that of the 22 low carbohydrate diet trials they included, "there is a pattern of weight loss ranging from -2.8 to -12.0 kg." This happens to be incorrect. The range reported in the trials referenced runs from -2.8 to -16.8 kilograms. Compare this, in any case, to the range of weight loss in the low-fat studies referenced: +0.4 kg to -11.8 kilograms.

More to the point, it's possible to compare the efficacy of low fat and low carb diets from the data the USDA authors provide. Although the USDA authors chose not to engage in this exercise, and although the two groups are not strictly comparable, it's still relatively easy with a simple calculator to come up with an average rate of weight loss for the two different types of diet. Even Fumento could have done it. The average weight loss for the 28 low fat trials listed--at an average fat intake 25 percent and energy intake of 1665 calories per day--is a little over four kilograms in nearly 23 weeks, or less than 1/2 pound per week. The 22 low carbohydrate diets referenced led to an average weight loss of 7.4 kg over 48 days, or more than two pounds a week at an average intake of 1300 calories each day. This is over four times the rate of weight loss from the low-fat diets. The number speak for themselves, despite the author's conclusions, repeated faithfully by Fumento, that low carbohydrate diets seemed to offer no advantages over low fat diets.

What's more, the USDA authors say that "the results of several of the [low-carb] studies actually refute the contention that low-carbohydrate diets, in the absence of energy restriction, provide a metabolic advantage for weight loss." They cite four studies supporting this proposition. I happen to have two of the four in my files, and neither support their point.

One was an uncontrolled Atkins's diet trial, published in 1980 by Larosa et al., that reported an average weight loss from the Atkins diet of .9 kg (2 lbs) per week. The researchers report they would have predicted only half that from the apparent reduction in calories. They then say they can account for another quarter of a kilogram by taking into account water weight, but still fall nearly .2 kg short per week. This sounds trivial, but if sustainable, it would amount to some 10 kg or 20+ pounds of weight lost in a year beyond that explicable by the reduction in calories. There may be a simple answer to this discrepancy, but Larosa et al. don't offer any. To say this study refutes the contention that low carbohydrate diets provide a metabolic advantage is patently untrue.

The other study is even more interesting, if for no other reason than the fact that one of the co-authors is the recently infuriated Gerald Reaven. His collaborator and first author was Alain Golay, a Swiss researcher. The study was published in 1996. In this study, Golay and company randomly assigned obese individuals to receive one of two 1000-calorie diets. One was a low carb diet--32 percent protein, 15 percent carbohydrates and 53 percent fat--and the other was a high carb diet--29 percent protein, 45 percent carbohydrate and 26 percent fat. After six weeks, the low carb dieters lost an average of 8.9 kg plus or minus .6 kg, while the high carb dieters lost 7.5 kg plus or minus .5 kg. While the authors concluded that there was "no significant difference in the amount of weight loss in response to the diets," the difference between these two diets is 1.4 kg or 3 pounds in six weeks, and the error bars--the plus or minus--do not overlap, which is an elementary measure of statistical significance. If this 3 pounds in six weeks difference were to be real and sustainable, which Golay's study cannot establish, it would translate to an extra 25 pounds of weight loss over a year, without eating a single calorie less. It may or may not be real, but it is certainly evidence that this study does not refute the aforementioned contention. (It's worth noting that Golay did a second study, without Reaven, that was the identical story. When I asked Golay why he didn't follow-up on this suggestion that low-carb diets do offer a metabolic advantage, he said longer term studies were prohibitively expensive.)

Fumento then moves on to tout the merits of a type of review called a meta-analysis and cites a December 2000 study in the International Journal of Obesity and Related Metabolic Disorders co-authored by one of Fumento's favorite sources, James Hill of Colorado. This is the second leg of Fumento's three-legged "crushing" mass of evidence. It is a quasi-systematic review of the literature that concludes that low-fat diets lead to reduction in calories and weight loss.

Fumento then gets something right. He says the only meta-analysis (of which I am aware) on this issue that was properly conducted was done by the Cochrane Collaboration. Fumento doesn't bother to describe this organization, which is what made this statement relevant, so I will. The Cochrane Collaboration is a world-wide network of researchers dedicated to providing unbiased reviews of the scientific literature. The collaboration was created specifically because the 77 scientists from 11 countries who founded it a decade ago believed that meta-analyses could be so easily biased by researchers' prejudices that their disciplines needed a standardized methodology to minimize the influence of such prejudice and a venue that would allow for the publication of unbiased reviews.

The Cochrane methodology makes it effectively impossible for researchers to pick and choose which studies they would like to include in their analyses on the basis of which studies are likely to give them the result they want, a common practice in this field. Cochrane Collaboration reviews must include all studies that fit a pre-specified set of criteria, and they must exclude all that don't. Indeed, the Cochrane review to which Fumento refers started out with 3,000 citations of potential fat-restricted dietary trials and ended up with only a dozen trials that were performed to the Collaboration's standards for what constitutes good science.

Fumento's statement that the review found "no advantage to low-carbohydrate diets" is true but slightly farcical and more than slightly dishonest. The review was only a review of "fat-restricted diets" and so the authors made zero attempt to review the efficacy of carbohydrate-restricted diets. The Cochrane Collaboration review can be found on-line at http://www.update-software.com/abstracts/ab003640.htm. Its conclusion is worth noting:

The review suggests that fat-restricted diets are no better than calorie restricted diets in achieving long term weight loss in overweight or obese people. Overall, participants lost slightly more weight on the control diets but this was not significantly different from the weight loss achieved through dietary fat restriction and was so small as to be clinically insignificant." [My emphasis.]

Fumento then goes on to communicate in his idiosyncratic style my explanation of the why these dietary trials fail to live up to reasonable criteria of good science. Regrettably, Fumento doesn't seem to understand the critical point, and perhaps even the less critical points, and so makes us both look like idiots.

Comments

|10.22.09 @ 3:56PM|#

Having read all three pieces in this debate, Fumento's critique of Taubes is so wholly inadequate I feel embarassed for him, and disappointed that Reason didn't vet Fumento seriously before publishing his original critique of Taubes. Taubes' position is increasingly permeating the research community and will likely be vindicated, leaving Fumento's piece as an embarassing example of Reason failing its mandate to provide critical analysis of contemporary debates. Reason's standards should be higher than merely being a forum for dissent -- you should have some standards for the dissent, and recognize dogmatic clinging to establishment thinking when you see it. The sooner the Lipid Hypothesis dies and refined carbs are recognized for their negative health effects the better the health of the world. We owe Taubes a serious debt of thanks for his integtrity and iconoclastic pursuit of reason. He shouldn't have had to offer this defense, likely only read by a fraction of those who read the original critique.