On Registered Clinical Trials

A new PLoS ONE study looks at the effect of mandatory pre-registration of medical study methods and outcome measures, starting in 2000. Major findings:

• Studies finding positive effects fell from 57% prior to the registry to just 8% afterward.
• "...focused on human randomized controlled trials that were funded by the US National Heart, Lung, and Blood Institute (NHLBI) [and so required advanced registration by a 1997 U.S. law]. The authors conclude that registration of trials seemed to be the dominant driver of the drastic change in study results."
• "Steven Novella of Yale University in New Haven, Connecticut, called the study 'encouraging' but also 'a bit frightening' because it casts doubt on previous positive results...”
• "Many online observers applauded the evident power of registration and transparency, including Novella, who wrote on his blog that all research involving humans should be registered before any data are collected. However, he says, this means that at least half of older, published clinical trials could be false positives. 'Loose scientific methods are leading to a massive false positive bias in the literature,' he writes."

Reported in Nature here.

Is Cohabitation Good for You?

Last week, Ars Technica (and I'm sure other news sites) posted an article on a large-scale survey of health outcomes in Britain, under the headline, "Good news for unmarried couples — cohabitation is good for you" (subtitle: "Married partners tend to be healthy, but living with someone works just as well"). Link.


I'm actually hyper-critical about people who sling around the phrase "correlation does not imply causation" too much in improper cases, but here's a golden example where it does apply; the headline "cohabitation is good for you", is totally unwarranted. Now, the findings do say that married & cohabiting people are healthier than people who live alone. But this could be either X causes Y, or Y causes X, or other more complicated interactions. One hypothesis is that "cohabitation is good for you [by improving health]"; another hypothesis is that "being healthy is good for your prospects of getting a partner", i.e., healthy people make for more attractive marriage/cohabitation partners. If you think about it, I'd say that the latter is actually the more common-sense direction of the causation here.

How could the direction of this effect be formally disentangled? Well, you could be on the lookout for a "natural experiment" where someone who did manage to get married/cohabited breaks up or gets divorced, and see if their health degrades during the later period in which they lack a partner. Of course, the researchers here were smart enough to do exactly that, and an entire paragraph of the Ars Technica article is in fact devoted to these findings:

"The study found that changes in status had no obvious impact—the transitions from/to marriage and nonmarital cohabitation did not have a detrimental effect on health. There wasn’t an obvious difference in these biomarkers when participants divorced and then remarried or cohabitated; they looked the same as participants who remained married. For men who divorced in their late 30s and didn’t remarry, the risk of metabolic syndromes in midlife was reduced."

In other words, for anyone in the category of at least being healthy and attractive enough to get married/cohabited once, being married or cohabited made no difference to their health. Which to my eye is overwhelming evidence that the causation is in the other direction, i.e., these headlines of "cohabitation is good for you" are flat-out wrong.

Might be a good example to include in my fall statistics course.