On Monday, I wrote about a recent study, published in the New England Journal of Medicine1 , which is intended to refute last year’s CDC study (published in JAMA2 ) showing that “overweight” people live longer than “normal” weight people.
According to media reports, this new study proved that even a slight weight gain can be deadly. The AP wrote “Being a little overweight can kill you, according to new research that leaves little room for denial that a few extra pounds is harmful.” NPR wrote “being even a bit overweight can potentially kill you.” The Star-Ledger ominously asked “Those few extra pounds won’t kill you … or will they?”
The NEJM study, by Adams et al., is — frankly — a badly conducted study. It uses a non-representative sample (members of The American Association of Retired People, or AARP) with an incredibly low response rate of 18%. Even worse, height and weight were self-reported.
But for the study’s authors, who were determined to come up with a study proving that fat equals death, the data presented a much more serious problem: the people in the “overweight” category were longer-lived than the people in the “normal weight” category. How to spin data showing that overweight people live longer, into their desired message that fat is always deadly?
In Monday’s post, I quoted Linda Bacon extensively. Today I want to expand on one of her points:
…Let’s take a look at the data itself. The authors worked hard for their conclusion. They examined records from over a half million AARP members that had been surveyed over a ten year period. What they found was entirely consistent with the earlier JAMA report: “overweight” people had the lowest mortality risk. But that wouldn’t serve their purposes. NEJM’s press release wouldn’t look nearly as attractive with that headline.
So they subjected their data to numerous manipulations before finally arriving at a suitable conclusion. First they threw out data on people who were smokers or former smokers. Nope, still shows overweight as benign. They hid this with a sleazy method: using only the top (BMIs of 23 to 24.9) of the “normal weight” group compared to the whole of the “overweight” group.
Here’s a table of some of the relative risks reported in the Adams et al. study (if you have trouble reading it, click on the image for a larger version):
The yellow column indicates the relative risk of death for “normal weight” people (the heaviest set of “normal” weight people are used as the baseline; all other risk ratios on this table are in comparison to those folks). The red outlines indicate the areas where the relative risk of death is as low or lower for “overweight” people as it is for “normal weight” people.
- Note that for “all men” and “all women,” the risk of death is less for people who are slightly “overweight” than it is for people in the “normal weight” categories.
- Looking at where the red outlines are located, can anyone seriously argue that the take-away finding of this data is that being a little overweight is generally a killer?
- In nearly every category, being in the “normal weight” BMI ranges of 18.5-20.9 or 21-23.4 carries a higher risk of premature death than being in the “overweight” BMI ranges.
- In most categories, being slightly “obese” (BMI 30-35) is slightly less risky than being in the middle of the “normal weight” category (BMI 21-23.5). In many categories, being solidly “obese” (BMI 35-40) is slightly less risky than being at the thin end of the “normal weight” category (BMI 18.5-21).
- In order to produce the finding that “overweight” is less healthy than “normal weight,” Dr. Adams did a very dishonest statistical manipulation – he compared just one “normal” BMI range, representing the heaviest people in the “normal” range, to the entire “overweight” range. This is because the majority of people in the “normal weight” categories had a greater risk of death than the majority of people in the “overweight” category.
- Race and sex have a strong impact on these findings. The “fat equals death” findings that were reported in the media shouldn’t be taken seriously by anyone, but black women especially would be well advised to ignore the reporting. This study actually found that black women, more than anyone else, have an elevated risk of death if they’re in the “normal weight” category, and live longer if they’re in the “overweight” category.
- The strongest association between “overweight” and increased mortality were for Latina women, and for the combined category of Asians, Pacific Islanders and American Indians (both male and female). However, the the sample size for these groups were significantly lower than the sample sizes for the other groups. For example, the finding that white men in the BMI range of 35-39.9 have a relative risk of 1.32 was based on the deaths of 1,862 white men in that BMI range; in contrast, the finding that Hispanic women in the BMI range of 35-39.9 was based on the deaths of only 10 Hispanic women in that BMI range. This makes these findings much less reliable.
- None of these risk ratios should be taken seriously. Why? Because they’re all very low3 — almost none of the risk ratios rise above two (for smokers the risk ratio of death by lung cancer, compared to non-smokers, is 23.3 for men and 12.7 for women4). Many of the risk ratios are so small (e.g., 1.07 – remember, 1.00 is a null result) that it’s impossible to imagine that they could have any realistic real-world significance to someone evaluating her own health.
With very small risk ratios – especially in a study with poor data-gathering and many possible factors not controlled for, such as this study – determining causation isn’t really science; it’s guesswork. Maybe these differences are being caused by fat (even though at many BMI ratios fat people live longer); but the cause may also be one of the many factors this study did not account for, such as yo-yo dieting, use of weight loss drugs, body shape, discrimination, and socioeconomic class, to name a few.
So that’s one of the ways Adams and his co-writers twisted the data to produce their desired result. But they weren’t done yet, and so neither am I: watch for a future post on how they used retrospective data (i.e., asking people “do you remember what you weighed when you were 50?”) to twist their data further, and for what they actually found out (but didn’t report): Losing weight is deadlier than gaining weight.
- Adams, K., et al., Overweight, Obesity, and Mortality in a Large Prospective Cohort of Persons 50 to 71 Years Old. New England Journal of Medicine, 2006. 355(8): p. 763-8. [↩]
- Flegal, K.M., et al., Excess deaths associated with underweight, overweight, and obesity. Journal of the American Medical Association, 2005. 293(15): p. 1861-7. There are “Alas” posts discussing this study here and here. [↩]
- The sole exception is the 4.12 risk ratio for Hispanic women with BMIs of 40+. This finding — based on the deaths of just 16 women — is such an extreme outlier that it’s almost certainly statistical noise. [↩]
- The Surgeon General’s “Health Consequences of Smoking 2004,” chapter 7, page 881 [↩]