Via Rachel and Meowser, I learned of a study, “Eating Behavior and Obesity at Chinese Buffets,” published by the academic journal Obesity. The nice folks at Cornell University’s Food and Brand Lab emailed me a copy of the study.
In a nutshell, they sent “trained observers” to various Chinese buffet restaurants. 213 patrons were spied upon for behaviors such as sitting in a booth or at a table, using chopsticks or forks, placement of napkin, how much was left on the plate, how many times each bite was chewed, etc.. The study authors theorized that these behaviors caused higher weights, but admitted that they couldn’t establish the direction of causality.
Personally, I don’t mind creepy spy studies like this one; a restaurant is a public place, and I like my science creepy. (If the researchers could indulge me with an occasional “bah hah hah!” laugh and gigantic goggles, so much the better.) But the methodology and thinking behind this particular study are jaw-droppingly awful.
The post is going to be very long, so I’ll first list some highlights:
1) There’s no reason to think visual assessments of weight by secret observers are accurate — and the citations the study used to support this methodology are, when you look them up, inapplicable.
2) Nothing in the study protects against the observations being tainted by bias and stereotypes about fat people.
3) Their sample of “normal” weight people included underweight, and even severely underweight, people.
4) A theory they describe on page one — that fat people go to the buffet more often — is not only not tested for by the study, the study is designed to exclude buffet trips from the study’s results.
5) Some causal connections speculated about in this study incorporate ridiculous anti-fat stereotypes. For instance, the study assumes that chopstick users “probably always eat with smaller utensils” when at home, and that people who don’t put their napkins on their laps lack “table manners” and therefore lack “careful consumption monitoring.”
Details after the fold.
1. “That guy across the room looks 214 pounds to me.”
Their measurement of weight and height consisted of “trained observers” making their best guess. To justify this, the authors write that “training has been shown to improve these estimates with the worst estimates still resulting in very high correlations.” But the two studies they cite to support this claim — here and here — show that even trained medical professionals, who have years of experience asking hundreds of people their weights, are “significantly inaccurate for individual observers.” Furthermore, doctors and nurses in an E.R. closely examine their patients — unlike observers seated “in unobtrustive locations in the restaurants.”
The authors don’t establish a credible basis for assuming their observers estimate height and weight accurately. They should first have conducted a study to find out if non-medical observers can be trained to accurately access weight and height of strangers in street clothes in secret from across a room.1
In my experience, perceived weight has a lot to do with class; someone in cheap, ill-fitting clothes may seem fatter to observers than someone of the same weight but wearing better fabrics and cuts. This is important, since many of the behaviors this study focused on may be related to class background (use of chopsticks and placement of napkins, for example).
2. Observer Bias
It’s well-known that observer bias can effect both how subjects are classified, and how their behavior is measured. (For example, social scientists have shown that interviewers for the highly-respected National Longitudinal Survey of Youth are less likely to classify the same subject as racially “white” if he’s in jail, and more likely to classify him as “white” if he’s not in jail.)
According to this study’s results, “low BMI” subjects chewed their food an average of 15 times per bite, versus 14 times per bite for overweight subjects and 12 times per bite for obese subjects.2 A small difference like that could easily be accounted for by observer bias, if observers believe that fat people chew their food less. Similarly, how reliable is the observation that low bmi eaters leave 11% of their food on their plates, compared to 7% or 6% for heavier eaters? Will a thin person who walks five or six steps before putting food on her plate be coded as “browsing the buffet” before serving, while a fat person with identical behavior is coded as diving right in?
Observer bias could also operate in the other direction. An observer, observing a subject chewing only 10 times or leaving only 3% leftover or taking food without “browsing,” could be influenced to decide that the subject’s weight is “actually” 180, rather than 150.
There are established methods of reducing uncertainty — such as having multiple observers independently observe and report on the same subjects and actions. That wasn’t done in this study.
3. Underweight and severely underweight patrons included in “normal” weight group.
Quoting from the study:
As Table 1 indicates, the 213 patrons were grouped into categories that represented the bottom third (n = 71), middle third (n = 70), and top third (n = 72) of their estimated BMI. These generally corresponded to the common descriptors given to normal weight, overweight, and obese individuals.
Yet according to Table 1, their classification of “normal” weight included patrons with BMIs as low as 13.5, a BMI that is medically classified as “severely underweight.”3 “Underweight” people in general have a greater risk of premature death than “overweight” (pdf link). If you take the weight/health link seriously (and if you don’t, what on earth is the point of this study?),4 then it’s absurd and irresponsible to suggest that more people should take on the eating habits of a group that includes underweight people.
When I asked study co-author Collin Payne about the inclusion of severely underweight people in his sample, he argued that there were very few underweight people included. But if there were too few underweight subjects to matter, why not exclude them altogether? Or report results for underweight people separately, as most BMI studies do? Even a couple of outliers can easily skew the statistics of a larger group.
As a reader, I shouldn’t have to be told by the study’s co-author, over the phone, not to worry about the inclusion of underweight people in the “normal weight” sample, because in his opinion (without having the numbers in front of him) it didn’t matter. That’s not the right way to do a study.
4. The study’s stated theory was never tested for in the study.
This is the study’s first sustained discussion of a theoretical connection between BMI and behavior, and was printed on page one:
Observers coded whether patrons sat at a booth vs. a table and whether patrons faced the buffet while eating vs. their side or back. Increased effort has been shown to decrease consumption (9). Because booth seating (vs. table seating) may make it more difficult to return to the buffet (i.e., eating companions may need to move, heavier patrons may not comfortably fit) it would be expected that this type of habitual seating would be related to lower BMIs. Increased salience of food has also shown to increase consumption (10). Patrons facing the buffet are expected to have higher BMIs than those whose side or back are toward the buffet. The continual salience of food that one is facing might keep it too temptingly top-of-mind.
So they found that heavier patrons were more likely to sit at tables, facing the buffet. They speculated that this behavior encourages heavier patrons to “increase consumption” by “return[ing] to the buffet.”
But here’s the problem: The study notes that their analysis “controlled for the random effect of observer bias and the fixed effects of sex, perceived age, and number of return trips to the buffet.”5 This means that the theory written about on page one of the study — that fatter patrons return to the buffet more often because of their seating behavior — is never tested for by this study.
To test their theory, they would have had to run statistical tests to see if seating patterns are actually associated with number of trips to the buffet or not. (Or, for that matter, to see if number of trips to the buffet is actually associated with BMI.) They could have run these tests. But they didn’t. Collin Payne told me that there wasn’t room — but how can there be room to explain a theory at length, but no room for even a single sentence regarding if their results support their theory or not?
In fact, by showing that higher BMI patrons are more likely to sit at tables facing the buffet even after controlling for return trips to the buffet, they’ve proven a relationship between weight and seating that can’t be explained by return trips to the buffet. That’s what “controlled for…. return trips to the buffet” means.6
So — aside from the study’s implication that fat people are ravenous pigs — why is it that fat people are more likely to sit at tables, and facing the buffet? Because booths are often too small for fat people’s comfort. (This explains why obese people avoided booths but merely “overweight” people didn’t, according to Table 2.) And, in restaurants where tables are adjacent to the buffet, fat diners might sit opposite the buffet to avoid blocking foot traffic.
5) Ridiculous variables which incorporate anti-fat stereotypes: chopsticks and napkins.
From the study:
Those using these smaller utensils would be expected to have lower BMIs than those who use larger utensils. This is because it is assumed that patrons’ utensil use is not unique to the single eating occasion noticed in this study—patrons who eat with smaller utensils probably always eat with smaller utensils. The placement of a napkin on a patron’s lap may also be related to lower BMIs. This may be because napkin wearing is reflective of table manners, which has been associated with more careful consumption monitoring.
In other words, fat people are slobs with no manners.
There’s an obvious reason fat people are less likely to put the napkin on their lap — it doesn’t do any good there. As “Buttercup” writes in comments at The F-Word, “I can put a napkin on my lap six ways to tuesday and it won’t do a damn bit of good. my boobs or belly catch any wayward food long before the napkin would.”
The study’s lead author, Brian Wansink, knows that fat people may not put napkins in our laps because they’re useless there — he’s quoted saying as much in a LiveScience article charmingly entitled “the science of pigging out” (no bigotry there!).7 Yet he doesn’t mention this possibility in the study, instead focusing on insulting stereotypes about fat people’s bad “table manners.”
To compound the silliness of measuring chopstick usage, the study didn’t control for race or class. But Chinese-Americans have lower average BMIs, and — at least in the restaurants I go to — are much more likely to use chopsticks than forks, and are a significant portion of chopstick users.
Nor did the study control for arthritis or class — factors probably related to both body shape (especially perceived body shape, since people with less money may wear lower-quality clothing) and chopstick usage.
Conclusion: Rachael at “The F Word” says that this lead author has published credible work in the past, and I don’t doubt her. But this study is nonsense. It is poorly conceived and badly executed, and if it didn’t align neatly with anti-fat stereotypes, it probably wouldn’t have gotten through peer review.
Please, scientists, please: If you can’t study fat people intelligently and without bias, just leave us alone.
- I admit, this might be hard to accomplish. But “it would be hard to do it right!” is no excuse for doing it wrong. [↩]
- Yes, there are people who sit in restaurants counting how many times you chew per bite. How creepy will that thought be the next time you eat out? [↩]
- To picture a 13.5 BMI, imagine someone five and a half feet tall who weighs 84 pounds. [↩]
- To be clear, I think the health/weight link is seriously overstated. But I doubt the authors of this study would agree with me. [↩]
- I also confirmed this on the phone with Collin Payne. [↩]
- Full disclosure: When I suggested this criticism to study co-author Collin Payne, he argued that I’ve misunderstood the study. Could be. [↩]
- Obviously, I don’t blame the study authors for the LiveScience headline. [↩]
Great analysis. Thanks again for the copy of the study.
Pingback: Meowser’s Chinese Buffet Experience « fat fu
Fantastic analysis, Amp. But your link with my name on it doesn’t lead to my site.
It’s truly remarkable to me that the ‘authors’ (I won’t called them ‘researchers’) of this study somehow failed to take these factors into account when it should be obvious to anyone that race, particularly, makes a *HUGE* difference re: body weight/size in this case.
i’m glad this was posted. there is a lot of bias towards fat people. just the other day i was having a conversation with someone, who happens to be a med student, and he was telling me a story about a nurse he works with, and ended it with “i feel really bad for her.” not knowing what he was referring to (since it was a happy story about how she shares chocolates with her students) i asked him what he meant, to which he replied “well, she’s really fat.” i sort of just sat there with my mouth open before finally blurting, “so what?” i just didn’t understand that kind of leap; i mean, i didn’t know what he “felt bad” about–was it that he thinks she shouldn’t have chocolate because of her weight, or did he feel bad accepting it?? i finally understood he simply took the medical standpoint on it, saying he felt bad because fat people have health problems, so i finally told him that fat people don’t need to be pitied, and it’s none of his business anyway. I don’t understand why he felt bad anyway, as if assuming that since she’s fat she must feel bad about herself, and therefore must need the pity of her students and coworkers…he wouldn’t have felt bad if the nurse giving him chocolate was “normal” weight…
anyway, this was my rant. sorry, but the article sort of reminded me of it and i thought i’d share.
Thanks for the comments, everyone. And Meowser, I’ve fixed the link.
Thanks!
I love this quote. How many people who are not actively dieting for one reason or another possess “careful consumption monitoring.” How many thin people do? Do you carefully monitor your consumption each time you eat? I don’t. I don’t know about the rest of you, but when I eat at a buffet, I eat until I’m done without considering how much I have consumed. Unless they’re talking about tuberculosis as “consumption.” In which case I don’t carefully monitor for that while eating, either.
I use chopsticks when I go to a Chinese restaurant. It has been noted by my dining companions (both here and by the locals in Japan) that I eat such food as quickly as someone else does with a fork and knife. And it has absolutely no bearing on the size of utensils I use at home or when eating Western food.
One factor that the researcher didn’t take into account was whether the buffet seats its patrons. Some require the host or hostess to seat customers, and unless the person requests a table because of their size (I’m a pretty large gal and I actually prefer booths because most of the time, the padding supports my back), you sit where you are directed. Or, if the buffet allows you to seat yourself and the only tables left are the ones closest to the food, you have to sit there. It’s not always a case of the fatties needing to sit nearest the food—sometimes it’s your only choice.
Great post, Ampersand.
The chopsticks assumption is especially egregious. I’ll use them at a Chinese or Japanese restaurant where they’re the default utensil, but at home, I pretty much never do.
And while I may eat slowly with chopsticks — being a white girl who learned how to use them relatively late in life — anyone who grew up using them isn’t going to be slowed down (or may be slowed down by a fork).
(If the researchers could indulge me with an occasional “bah hah hah!” laugh and gigantic goggles, so much the better.)
It’s not my study (or it would have been better done) and gigantic goggles are out in favor of less romantic but more protective face shields, but if it’ll help any…BAH HAH HAH!
Good analysis. You ought to be doing peer review.
This study is hilarious. It reads like a parody of itself.
“That guy across the room looks 214 pounds to me.”
As teenagers, my brother and I ate the same food. I had a BMI of 23 and he had a BMI of 14. Guess which one of us our mother (a pediatric nurse) thought needed a special diet? Observers are flawed and biased.
I don’t get it… are they saying chopsticks speed up or slow down eating? And that fat people are more or less likely to use them.
As an Asian who grew up using chopsticks, the “obvious” answer, if there is one here, eludes me. I could go back and parse what was said, but that would require re-reading everything around it as well, so… no.
That “study” is as scientific as a webpoll.
Patrons facing the buffet are expected to have higher BMIs than those whose side or back are toward the buffet. The continual salience of food that one is facing might keep it too temptingly top-of-mind.
HA HA HA HA!!!!
So what do they suppose happens when a whole pride of fatties shows up at a Chinese buffet – are we all jockeying for the seats that face the steam tables??
And to further Bree’s point about whether a host/hostess guides you to your seats or not – I’ve often wound up taking my seat on a side of the table I might not have preferred at a restaurant, just because I couldn’t work my way around the host/hostess in order to reach it.
Gawd, the whole thing is just ridiculous and teeming with fat stereotypes. I mean, they assume that sitting at a table is more conducive to more trips to the buffet? Um, I don’t know about the rest of y’all, but as a fatty? I try to stick to booths or tables along the perimeter of a dining space, rather than a table smack in the middle, if I’m given an option. Despite my efforts at cultivating a more brazen attitude, I’m still leery of effectively putting myself on display as The Fat Woman Eating In Public.
Seriously, is it just me, or does this study just seem mostly like an excuse to use grant money for poor grad students to eat at the Chinese buffet? I expect their next study will involve the relationship between blood-alcohol levels (as determined by ‘trained observers’ from across a room) and what pizza toppings are ordered after the bars close.
From the minute I heard that they were guessing height and weight I knew that they study could not be accurate. Two people can look at the same person and make completely separate conclusions about that persons body. Every time we make an assessment about the body of another we bring to bare our own personal biases based in our frame of reference. This is junk science at best looking for causality to fat shame people.
I’m just happy that as a fat chopstick-user who prefers to sit in booths, chews his food thoroughly, and always puts his napkin in his lap (because I wasn’t raised by wolves), I blow these people’s minds.
You have used my photograph here WITHOUT MY PERMISSION. Please remove it immediately from your post. Thank you for your prompt attention to this and I trust there will be no further copyright infringement of my artwork.
Lenore Locken
[Photo removed, and I apologize. –Amp]
“3) Their sample of “normal” weight people included underweight, and even severely underweight, people.”
I find this very telling. Underweight, even severely underweight, people are still “normal” but over-weight and obese people are not.
Thank you so much for being prompt, but also for being gracious. I do appreciate it.
Lenore Locken
You know how, when you’re Fat in Public, you think everyone is looking at you and judging you about whether or not you’re eating or waddling or breathing differently or wearing sweatpants, and then you say to yourself, “geez, stop being paranoid and self-involved!”
Yeah.
I think I’ll stay inside and play video games some more.
I find this very telling. Underweight, even severely underweight, people are still “normal” but over-weight and obese people are not.
This is why, when I got a “results of your visit” handout at my doctor’s office that told me “those with a BMI over [some number] should begin weight-loss efforts immediately!”, I was not surprised that those with a BMI under 17.5 were not told they should begin weight-gain immediately, even though their death rate is quite significantly higher than that for even obese people. I wasn’t surprised, but I did write my doctor a letter about it.
Notice how there’s “obese” and “underweight,” and then there’s “morbidly obese” but there’s no “morbidly underweight” category?
Yep. But I’m sure it’s all objective and science-y, and doctors have really really good reasons for that!
Hi Barry,
Thanks for sending me a note of your post. You did a very thoughtful and thorough analysis of the article.
It would have been useful to have had you as a reviewer when it was submitted. Your insights would have led us to put more detail into the methodology and to more clearly write both the justification for the study and the interpretation of the results.
As someone who has suffered experiences related with obesity, it hurts me to know that these findings have unintendedly hurt others. My hope, as with the establishment of my Cornell Food and Brand Lab and my writing of Mindless Eating, was to help people change behavior in easy ways that can help them eat a little less rather than a little more.
Again, thanks for taking the time to note the findings in the article. It seems like they’ve given something for people to think about. If that leads them to think twice at the buffet, I will be pleased.
All my best,
Brian Wansink
http://www.MindlessEating.org
Brian – while I as a reader appreciate that you stopped by here, as both a fat person and someone interested in research I’m incredibly disappointed. All of the points that have been made here by Amp and others about methodology gaps in your research that weaken the conclusions you drew from it are fairly obvious critiques that should have been considered by you and your research team. It’s still much easier to find badly-designed research which is misinterpreted to support a pre-judged conclusion that “OMG FAT PEOPLE ARE FAT BECAUSE THEY OVEREAT” than it is to find the critiques or rebuttals to such research, but they exist, and even a cursory literature review should reveal them.
It is also not that hard to ask the question “are there any assumptions we’re making in our research design that might get in the way of producing valid results?” and in fact that question should be a major part of every research process.
Although your comment here is polite, your concluding remarks:
suggest that you are still wedded to the assumptions that fat people are fat because they overeat, that thin people do not overeat, and that if only fat people would change how much they eat, they would lose weight, which would be a good thing.
I could send you to Junkfood Science, or to Shapely Prose and the rest of the fatosphere, or to “Rethinking Thin” and “The Obesity Myth,” but for now I’ll just send you to the perfectly fine internal link here.
And for the love of god, until you are ready to critically engage with these harmful and all-too-common “conventional wisdom” beliefs about weight, please stop doing research in this area.
I used chopsticks, but not a napkin, when I was anorexic. I used chopsticks, but not a napkin, when I got my weight up to a healthy level. I used chopsticks, but not a napkin, when I was so depressed that I was lucky if I ate much at all. I used chopsticks, but not a napkin, when I was bicycling all over Dallas and ate 9,000 calories a day.
I’m thinking the study was stupid. And some people know how to keep food off their lap and so leave the napkin on the table.
(Which reminds me of an old joke — A Sailor and a Marine are in the head doing their business. After they leave, the Marine says to the Sailor “They taught us to wash our hands after going to the head.” The Sailor says “They taught us not to pee on ourselves.”)
Wow, it’s always kind of amazing when the subject of a critique shows up, but this:
“It seems like they’ve given something for people to think about. If that leads them to think twice at the buffet, I will be pleased.”
makes me think that Mr. Wansink missed the point entirely.
If this and other critiques of his research lead Mr. Wansink to think twice when it comes to research design, I will be pleased.
I know he’s coming from the “we’ve got to get people to eat less, right now! For their health!” camp, but he is happier to have people eat less in public due to stigmitization, and a creepy feeling that people may be watching and judging them. These “unintended consequences” are maybe not so unintended.
If the ultimate goal is improved health, and not just fewer fat people in public, perhaps a more complete view of health that includes the mental, social and spiritual health of individuals would make a difference.
WRT2 —
I doubt he’ll learn anything.
There is a certain truth to “If you eat less, you’ll gain less”, but it’s a lot like “If you hammer fewer nails, you’ll hit your thumb less often.” It could well be that hitting MORE nails achieves the same result because of the extra practice, and it could be that hitting fewer nails achieves the wrong result for the opposite reason. Without knowing the cause behind someone’s weight — over, under, or “just right” by whatever standards one chooses — conclusions are pointless. I leave my napkin on the table, not because I’m a slob, but because it’s more convenient there so I can wipe my mouth or chin — something I need to do more than catch stray food before it hits my lap.
— Julie.
Pingback: Some Facts/What This Means for You
Pingback: Five Links That Are Actually Important, « Our Descent Into Madness
Pingback: Alas, a blog » Blog Archive » Today Is International No Diet Day. Also, buy Kate and Marianne’s book.