Not Everything that Works Works

Yesterday, my friend and colleague Yoni Freedhoff posted a thoughtful comment on the launch of Gary Taubes’ new non-profit organization NuSI, whose stated mission is to, “improve the quality of science in nutrition and obesity research“, and whose implied mission (according to Yoni), “is to prove Gary Taubes’ carbohydrate hypothesis of obesity is as correct as he clearly believes it to be“.

In this post, Yoni makes the important distinction between what works in a laboratory or the closed confines of a controlled clinical trial and what ‘really’ works in the ‘reality’ of clinical practice.

This issue is commonly referred to in the medical literature as the difference between ‘efficacy’ and ‘effectiveness’ – the former refers to the proof that a treatment actually works when used as determined by the researchers – the latter refers to whether or not that same treatment works when widely used in an actual clinical setting or in the general population.

As an example, there is no doubt in anyone’s mind that the DASH diet (Dietary Approach to Stop Hypertension) is amazingly effective in lowering blood pressure – to an extent comparable to, or even exceeding that of, taking a blood pressure pill. Clearly this diet is ‘efficacious’ for lowering blood pressure.

However, were I to take 100 people with high blood pressure off the street and educate them all on the benefits of following the DASH diet, I’d be surprised if 6 months later even 5% of individuals would still be following and achieving blood pressure targets on this diet – what works fine in a group of highly motivated, self-selected volunteers with all of the attention and dedication of a research team behind them, does not easily translate to routine clinical practice or a population-wide intervention.

Imagine in contrast, if the same 100 patients were told to simply take a blood pressure pill every day. Chances are that perhaps as many as 50% of individuals will still be taking their pills and controlling their blood pressure at 6 months.

Thus, even if the ‘efficacy’ of the DASH diet for lowering blood pressure was greater than that of the blood pressure pill, the pill would in fact be far more ‘effective’ as a treatment for blood pressure in real life than any diet could ever be.

The same, no doubt, could be said for anything that Taubes (or any one else) comes up with in terms of “the” solution to obesity. Irrespective of the fact, that (as pointed out by Yoni), obesity is an incredibly heterogeneous disorder with multiple causes and complex psychophysiology (which is why no one, who knows anything about obesity, expects there to ever be ‘the’ solution), the notion that whatever is found to work in trials will also necessarily prove ‘effective’ in actual practice (read: real life!) will always remain to be proven..

This is hard for non-researchers to understand and, sadly, even all too many researchers believe that once they prove ‘efficacy’ their work is done.

Diets don’t work in real life simply because most people (exceptions simply prove the point) cannot maintain them forever.

This is not a failing of the people – in the end, it is a failing of the diets.

Patients never fail treatments – it is always treatments that fail patients.

Toronto, Ontario

photo credit: osiatynska via photo pin cc