How Much Of The Confusion In Childhood Obesity Studies Is Simply Regression To The Mean?

When it comes to childhood obesity interventions, there is much confusion as to what works and what doesn’t.

Thus, for every study showing that a given “intervention” (e.g. school intervention programs, exercise programs, removing vending machines, etc. ) changes weight measures, there is at least another study showing that it doesn’t.

Although this problem is by no means specific to research in childhood obesity, for reasons stated below, research in this area appear to be particularly prone to this problem.

Now, a paper by Cockrell Skinner Asheley and colleagues, published in Childhood Obesity, suggests that much of this confusion may simply be due to the statistical phenomenon of “regression to the mean” (RTM).

As readers may be well aware, regression to the mean refers to the well-described phenomenon that “outliers” (up or down) tend to “regress” towards the mean on repeated measures.

Or as the authors explain,

“Today, RTM is often conceptualized primarily in the context of measurement error or repeated measures. Blood pressure provides a reasonable example. If one measure of blood pressure is obtained and is either much higher or lower than the mean, a second measure will likely be closer to the mean. If conceptualized as measurement error, then an average of multiple measures is often used to reduce measurement error, thereby also reducing regression to the mean.”

Repeated measures however do not solve the problem when the measured values actually do change over time (as in a child’s body weight). As the authors note,

“However, this does not address changes in the true value of the variable over time, which are not due to measurement error. Whenever two variables are not perfectly correlated (such as blood pressure at two time points), there will always be RTM when measured in terms of standardized variables. This occurs regardless of measurement error, the order of measurement, and whether the two variables are repeated measures of the same construct. Additionally, as noted by Barnette et al., regression to the mean can occur in nonnormal distributions and those that are not continuous. For example, RTM can occur in binary data and cause subjects to change categories without a change in their actual status.”

While this issue tends to affect all types of research, which is why every experiment would ideally have rigorous controls and the most robust research methods generally use some form of randomisation, this is particularly difficult in studies in childhood obesity.

“Many intervention efforts, including policy changes and community-based interventions, do not easily lend themselves to the gold standard randomized, controlled trial (RCT) designs. Quasi-experimental designs provide stronger evidence than do uncontrolled interventions in which investigators simply look at change from baseline in a single group of treated cases. These designs, which lack the element of randomization, are common in pediatric obesity research, and include cohort studies, regression discontinuity, and panel analysis.”

“One of the most common errors associated with RTM, particularly in the obesity literature, is concluding an intervention is effective when the study design does not permit such a conclusion. Reports of school-based interventions commonly ignore this effect of RTM, reporting reductions in BMI z-score and prevalence of obesity, with no comparator other than baseline. Community-based interventions also claim success in reducing weight and blood pressure even when lacking a control group, as do many clinic treatment studies.

The researchers give a number of examples from the childhood obesity literature, where “findings” can easily be explained by RTM and highlight some of the erroneous conclusions that can be made when studies lack control groups or no consideration is given to RTM in power calculations or data analyses.

Furthermore,

“RTM can also be mistaken for evidence of differential effects of treatment as a function of baseline values on the outcome variable. Differential RTM indicates that RTM will be greater among groups defined as being further from the mean than other groups. One example involves weight gain among patients taking antipsychotics. Some studies noted that patients with higher baseline BMI gained less weight when taking an antipsychotic drug than did those with lower baseline BMI. Although this was initially interpreted to mean that the drugs caused less weight gain among persons who were more obese at baseline and thereby mitigated concerns about drug-induced weight gain, subsequent analyses showed that there was no evidence of such differential effects of the drugs as a function of baseline BMI, but rather just different expected weight changes as a function of baseline BMI, as is expected solely from RTM. Erroneous comparison of nonequivalent groups can also be seen when investigators report greater declines in BMI among study participants with higher baseline BMI compared to those with lower baseline BMI, and label it as evidence for differential treatment efficacy by baseline BMI.”

The authors then go on to suggest several ways in which to correct analyses for such effects or to better design studies and statistical analyses to avoid erroneous interpretation of findings (both positive and negative). In all of this, the importance of proper controls is paramount.

This issue is far from trivial  as many costly but ineffective policy or treatment interventions may be implemented based on “promising” findings that are simply attibutable RTM.

On the other hand, interventions that are in fact effective may fail to be implemented or be discarded because RTM masks their actual benefits.

Not least, failing to consider RTM in the design, implementation and analyses of research (particularly the type of research that by its nature lacks proper controls) can be a huge waste of valuable research funding and resources.

@DrSharma
Edmonton, AB