Why I Am Not Losing Sleep About NHANES’ Flawed Data
Monday, October 21, 2013
Regular readers may be well aware of the US-National Health and Nutrition Examination Survey (NHANES), the billion dollar juggernaut of a research enterprise conducted by the the US-CDC and the Department of Agriculture since the early 70s.
It turns out that rather simple calculations, published in PLoS One by Edward Archer, from the Arnold School of Public Health at the University of South Carolina in Columbia, and colleagues, show that the dietary data collected in this survey (using a 24-hour recall), widely used by researchers and policy makers as the primary source of data on the impact of nutrition and diet on health, is rather “implausible”.
As the calculations in the paper show, the caloric estimates that people are reportedly eating are simply not physiologically possible given their reported age, size, weight and levels of physical activity.
In short, given the overall “iffyness” of the dietary information in NHANES, it is rather impossible to accurately determine the relationship between changes in caloric intake (and I would assume the same goes for nutrient composition) and the obesity epidemic.
So, why do I not really care?
For one, even if these data were fully accurate, knowing that Americans are consuming more calories today than they did before does not really tell me much – irrespective of whether or not these numbers have gone up or down, the fact is that somewhere along the line, the net amount of calories Americans consume is larger than the net amount of calories Americans need – probably by several hundred calories a day – end of story!
Secondly, as in all observational studies, what you get are correlations, which may or (more likely) may not give you useful insights into causality. Thus, as larger bodies need more calories to sustain (and larger people consume significantly more calories by just moving themselves around), I would expect average caloric consumption to have increased. Thus, even if we assumed for a minute that the obesity epidemic was caused by a viral infection that makes people fat, those larger people would now be eating more calories than before to sustain their larger bodies. Thus, showing that obese people eat more calories (which incidentally is not always the case), does not really provide much insight into what is causing the problem.
Thirdly, we are dealing with population averages on an issue where we know there exists a rather wide range of inter-individual variation. Think of your average nerdy hyperactive teenager, who can consume thousands of extra calories a day and still have his ribs sticking out (disclaimer: I was one of those!). Then think of your formerly obese dieters, who have yo-yo dieted themselves into a corner where it appears that they can happily gain weight on little more than water and fresh air. We have ample scientific proof of this variability in terms of both susceptibility to weight gain as well as ability to defend body weight (remember, we are dealing with physiology not physics!). It’s not about what the average person eats – it is about what you eat (or not).
So, not only should the physiological implausibility of the NHANES nutrition data not come as a surprise to anyone (ask me what I ate and drank in the last 24 hours and the best you’ll get is a hopelessly inaccurate guesstimate – and I’m supposedly an “expert”) but their relevancy for understanding the cause of the obesity problem or helping find viable solutions is no less doubtful in my mind than it was before.
Yes, there is certainly some scientific merit to the exercise – it is after all the best we have in terms of national representative health information (readers may recall that we used NHANES data linked to death registers to validate the Edmonton Obesity Staging System). As for finding solutions to obesity? I certainly do not (and never have) expected any useful insights from NHANES that would help me deal with obesity at the individual level. As for public health recommendations? Yes, we may all benefit from eating less fat, sugar and salt (as well as washing our hands, getting enough sleep and looking right and left before we cross the street) – but then again, we could probably have guessed that.
@DrSharma
Edmonton, AB
Archer E, Hand GA, & Blair SN (2013). Validity of U.S. Nutritional Surveillance: National Health and Nutrition Examination Survey Caloric Energy Intake Data, 1971-2010. PloS one, 8 (10) PMID: 24130784
.
Monday, October 21, 2013
In medicine as in fields of social study there is often a lack of something that exists in the “hard” sciences. In the “hard” sciences it is common to publish studies that do NOT pan out because those, too, usually provide useful data. Birdwhistle was said to have told students that “Absence of evidence is not evidence of absence.” but so to should it be realized that documentation of negative results can show the way to positive results.
Also, it is common to include in “hard” science research papers what the shortfalls are (for instance, if there are aspects which need further study, if there are things which just can not be defined as clearly as the researcher would want due to current state of technology or other cause, how many trials failed compared to succeeding, etc. Each of these is important. Yet in medical studies those points are too often not included.
Another thing that is often pointed out is how a study shows something that was unexpected by the researchers — which you are showing can be important. I do not know if the study to which you refer points out what you make obvious: that is provides a great example of people’s inability to track their food consumption if notes are not kept and judge their portions if weights or volumes are not taken, so it sounds like the study does make a contribution on that score — whether the authors noted it or intended it or not.
Monday, October 21, 2013
The very design of the study was its downfall. It was never a question as to whether or not the conclusions were valid. Any high school psychology student could have told you why this design was hopelessly flowed. It is of course never a good idea to rely on individuals to recount their eating or exercising behaviors. That is not just bad science, it is hopelessly naive. As far as the implications for individual weight loss from such a compilation of individual data points; you are right, no help can be found for individual weight loss because of all the points that you so succinctly already made. However, I do believe that there is good reason to hope for better studies that influence public policy. The way we allocate our shrinking resources is often based on studies just as flowed as this one and we deserve better. I could easily suggest a better design using modern technology and I am sure you could too.
Tuesday, October 22, 2013
…Thirdly, we are dealing with population averages on an issue where we know there exists a rather wide range of inter-individual variation….
…and this is the KEY question, which hasn’t yet been broadly addressed in the DAILY life!!
In our nutritional practice in Basel Switzerland (as illustrated in details in the book: “Eating healthy and dying obese…elucidation of an apparent paradox”), we base our counseling on the inter-individual variations, by measuring the INDIVIDUAL Resting Metabolic Rate… see our data presented at the last ECO2013 (European Congress on Obesity, n Liverpool).
http://www.vitasanas.ch/wp/wp-content/uploads/2013/06/poster-only-eco-liverpool-ok.pdf
Tuesday, October 22, 2013
In order to fully appreciate the wide range of inter-individual variations, even at REST, see please this graph illustrating the discrepancies between MEASURED and CALCULATEd (WHO-formula) Resting Metabolic Rate (= Resting Energy Expenditure) http://www.vitasanas.ch/wp/wp-content/uploads/2013/02/who-ree-mesured-rm_mueller.jpg
Tuesday, October 22, 2013
…and here how we illustrate the inter-individual variations in our book… http://www.vitasanas.ch/fileadmin/user_upload/pdf/BOOK4_The_personal_caloric_requirement__2_.pdf
Monday, February 22, 2016
NHANES is responsible for the current upsurge in low carb high fat dieting. The reported increase in % dietary carbs from 1965 to 2000 has been used to justify the argument that increased carbohydrate consumption is the cause of the current obesity crisis.