Truth TOPS Bias at TOS MeetingTuesday, October 27, 2009
Yesterday, my friend and colleague David Allison, a professor of biostatistics at the University of Alabama, was given the 2009 TOPS Research Achievement Award from the Obesity Society (TOS).
In his award lecture “Experiments with Truth”, Allison discussed the issue of truth and bias in scientific reporting. Using examples from obesity publications, he illustrated how sometimes research findings can be reported or emphasized in ways that may not fully and accurately reflect the actual data. The reasons for this are manifold and raise the important issue of bias and hidden agendas in science.
Interestingly, in a letter just published by Allison in the journal SCIENCE, he discusses the value of the now common practice of disclosing financial interests in scientific presentations and publications. (Readers will recall seeing my own disclosure statements at the end of blog postings on products of companies from which I have received honoraria or research support).
This emphasis on disclosing “financial” interests, as pointed out by Allison, may well be less valuable or effective in countering potential bias than most people may think.
As Allison writes in his letter, “dissenters” to these often stringent disclosure policies often present the following objections:
1) Such policies may actually increase biased behavior among some persons.
2) Judging scientists’ credibility by their associations is tantamount to McCarthyism.
3) Financial interests are neither the sole nor necessarily the most compelling motives for conflicts of interests.
4) Judging credibility of scientific conclusions based on characteristics of the scientist offering them is antithetical to the essence of science, which should rely on data and deductive reasoning alone.
As Allison goes on explain, financial disclosures do nothing to address other important sources of bias in scientific reporting such as personal predilection, philosophical leanings or personal aggrandizement.
Based on my own experience, I can assure readers that far less in scientific reporting is bias free than most of us may generally assume. Not only does personal bias affect what scientists chose to study and what not to study, they also have their “pet” hypotheses and ideologies around which they conduct their research and interpret their results. Not surprisingly, these biases find themselves reflected in presentations and publications.
Although pharmaceutical and food companies may have their own commercial interests in supporting research and of course tend to “spin” science to sell their products, bias is by no means limited to industry or to organisations that stand to make financial gains.
Government agencies have political agendas as do research foundations, which will fund certain projects and not others. I have personally sat on enough non-commercial review and advisory panels over the years to be well aware that funding decisions are not always made on scientific merit alone.
Importantly, the very people who talk about and raise the issue of bias are never without bias themselves, clearly promoting ideologies and philosophies that can highlight some sources of bias, while ignoring others (including the “positive” impact that raising the issue of bias may well have on their own careers).
Thus, while declaring financial interests may be important, declaring other sources of bias may be as important if not more so.
In the end, as Allison points out, no amount of “disclosures” can guarantee unbiased science reporting. The only way to truly guarantee objectivity is to return to the very elements of scientific discovery and that is the element of reproducibility – anyone repeating the same experiment or analysing the same data set should be able to come to exactly the same results and conclusions.
The only way to fully eliminate “fiddling” and “misrepresentation” of science is to publicly share data. When data are public, no one need take analyses on faith.
“Anyone with the skills can conduct their own analyses, draw their own conclusions, and share those conclusions with others. This is more constructive than simply casting doubt on the analyses’ integrity because of the analyst’s affiliations.”
For example, although I may well have my own biases in deciding which articles I chose to blog about and how I present the results, anyone is free to look up the original publications and read them to draw their own conclusions.
I congratulate Allison on receiving this prestigious award and on using this opportunity to raise and discuss this important issue.
Tuesday, October 27, 2009
Great article! This same question has been on my mind a lot as of late. The conclusion is one that I can fully agree with. Share the data so that others can come to their own conclusions. This is especially useful in seeing how the original questions or hypothesis is formulated. They say that a well formed question gives it’s own answer. So it would stand to reason that you simply ask the question that will give the answer you are looking for.
Wednesday, October 28, 2009
The emotional bias that I see everywhere against people who are overweight is an issue that should also be addressed. The supposed “financial costs” of obesity are awfully hard to swallow ! The clear confusion of association and causation is appalling. The concept that someone with asthma might be heavy because undertreated or undiagnosed asthma prevents them from exercising, rather than they have asthma because they are heavy, is totally missing in action from what I see in what the lay publications print from the research.
This bias is present in the research community as well, I’m sure.
I would like to see someone take an obese population, rigorously diagnose the “subclinical” conditions such as depression, arthritis, low thyroid (which may still have levels in the “normal” range but nevertheless be low for that individual), asthma, and so on, treat them effectively and completely, and then see what happens with weight. Then and only then will we really be able to see the difference between cause and association, and how to effectively treat the conditions underlying the symptom of excess weight.
Friday, October 30, 2009
I think slccom has an excellent point.
If someone looses weight because of a certain cause, for example depression, people are concerned and sympathetic and offer help.
If someone gains weight because of the same cause, for example, overeats instead of undereats in response to depression, the person is viewed with contempt instead of sympathy, and medical problems are ignored or seen as a result of self-indulgence.