Why I Am Not Losing Sleep About NHANES’ Flawed DataMonday, October 21, 2013
Regular readers may be well aware of the US-National Health and Nutrition Examination Survey (NHANES), the billion dollar juggernaut of a research enterprise conducted by the the US-CDC and the Department of Agriculture since the early 70s.
It turns out that rather simple calculations, published in PLoS One by Edward Archer, from the Arnold School of Public Health at the University of South Carolina in Columbia, and colleagues, show that the dietary data collected in this survey (using a 24-hour recall), widely used by researchers and policy makers as the primary source of data on the impact of nutrition and diet on health, is rather “implausible”.
As the calculations in the paper show, the caloric estimates that people are reportedly eating are simply not physiologically possible given their reported age, size, weight and levels of physical activity.
In short, given the overall “iffyness” of the dietary information in NHANES, it is rather impossible to accurately determine the relationship between changes in caloric intake (and I would assume the same goes for nutrient composition) and the obesity epidemic.
So, why do I not really care?
For one, even if these data were fully accurate, knowing that Americans are consuming more calories today than they did before does not really tell me much – irrespective of whether or not these numbers have gone up or down, the fact is that somewhere along the line, the net amount of calories Americans consume is larger than the net amount of calories Americans need – probably by several hundred calories a day – end of story!
Secondly, as in all observational studies, what you get are correlations, which may or (more likely) may not give you useful insights into causality. Thus, as larger bodies need more calories to sustain (and larger people consume significantly more calories by just moving themselves around), I would expect average caloric consumption to have increased. Thus, even if we assumed for a minute that the obesity epidemic was caused by a viral infection that makes people fat, those larger people would now be eating more calories than before to sustain their larger bodies. Thus, showing that obese people eat more calories (which incidentally is not always the case), does not really provide much insight into what is causing the problem.
Thirdly, we are dealing with population averages on an issue where we know there exists a rather wide range of inter-individual variation. Think of your average nerdy hyperactive teenager, who can consume thousands of extra calories a day and still have his ribs sticking out (disclaimer: I was one of those!). Then think of your formerly obese dieters, who have yo-yo dieted themselves into a corner where it appears that they can happily gain weight on little more than water and fresh air. We have ample scientific proof of this variability in terms of both susceptibility to weight gain as well as ability to defend body weight (remember, we are dealing with physiology not physics!). It’s not about what the average person eats – it is about what you eat (or not).
So, not only should the physiological implausibility of the NHANES nutrition data not come as a surprise to anyone (ask me what I ate and drank in the last 24 hours and the best you’ll get is a hopelessly inaccurate guesstimate – and I’m supposedly an “expert”) but their relevancy for understanding the cause of the obesity problem or helping find viable solutions is no less doubtful in my mind than it was before.
Yes, there is certainly some scientific merit to the exercise – it is after all the best we have in terms of national representative health information (readers may recall that we used NHANES data linked to death registers to validate the Edmonton Obesity Staging System). As for finding solutions to obesity? I certainly do not (and never have) expected any useful insights from NHANES that would help me deal with obesity at the individual level. As for public health recommendations? Yes, we may all benefit from eating less fat, sugar and salt (as well as washing our hands, getting enough sleep and looking right and left before we cross the street) – but then again, we could probably have guessed that.
Archer E, Hand GA, & Blair SN (2013). Validity of U.S. Nutritional Surveillance: National Health and Nutrition Examination Survey Caloric Energy Intake Data, 1971-2010. PloS one, 8 (10) PMID: 24130784