While we often criticize those working for industry for lack of independence and “biased” interpretation of their research findings, we tend to ignore the fact that those of us in “public-funded” research have as large (if not greater) self-interest in presenting our findings in the best possible light – even if that may occasionally involve an overly enthusiastic interpretation of our own findings or “massaging” these till they appear to fit our own favourite hypotheses.
A new analysis of this issue by Menachemi and colleagues from the University of Birmingham, Alabama is now published in the American Journal of Preventive Medicine.
The researchers examined almost 1,000 papers on nutrition or obesity published in 2001 and 2011 in leading specialty, medical, and public health journals to estimate the extent to which authors overstate the results of their study in the published abstract.
They were particularly in overreaching statements that included (1) reporting an associative relationship as causal; (2) making policy recommendations based on observational data that show associations only (e.g., not cause and effect); and (3) generalizing to a population not represented by their sample.
Not surprisingly, they found that almost 10% of studies (8.9% to be exact) had overreaching statements with this being more common in papers published in 2011 than back in 2001.
Interestingly enough, unfunded studies were two and a half times more likely to have an overstatement of results, whereas studies with a greater number of coauthors tended to have fewer such statements.
While these findings are disconcerting, I do neither believe that they are in any way specific to studies in obesity and nutrition – my guess is that a similar number of overly enthusiastic interpretation of studies will be found in most scientific fields.
I would also not just put the blame on authors, who, in a highly competitive environment (and passionately held beliefs) may be tempted to “oversell” their findings.
Rather, I would put the blame squarely on the peer review system and journal editors, whose job it is to identify and insist that such statements be toned down to match the actual findings.
Unfortunately, as most journals are for-profit, there is also considerable pressure on journals to try and oversell the findings published in their pages in the hope that these will be picked up by the media and thus ultimately lead to more visibility, citations and impact factors of their journals.
Unfortunately, all of this can negatively effect the “public trust” when it comes to scientific publishing – something that should concern all of us – especially those dependent on public funding for their work.
Menachemi N, Tajeu G, Sen B, Ferdinand AO, Singleton C, Utley J, Affuso O, & Allison DB (2013). Overstatement of results in the nutrition and obesity peer-reviewed literature. American journal of preventive medicine, 45 (5), 615-21 PMID: 24139775