When pharmaceutical companies wish to market a drug, there is a legislative requirement that they first demonstrate efficacy and safety.  The pharmaceutical companies do this by contracting out clinical trials.  The history of these trials is one of refinement over time with the most recent trials meeting rigorous requirements for blinding (i.e. neither the participants nor the experimenters know who’s getting what kind of treatment) and randomisation (i.e. random allocation of participants to the different treatments).

I was listening to an old Skeptics Guide to the Universe (SGU) podcast from October 13th 2010 which featured an interview with Ben Goldacre, a psychiatrist, medical writer and author of the excellent blog/column/book Bad Science.  He made an argument that resonated so strongly that I have to reproduce it in its entirety:

…as far as I am personally concerned, if you selectively choose not to publish the trials that you do that don’t have a positive result, to me that is basically the same as research fraud.  I’ll explain why: if I did one trial on 1,000 patients and 300 of those patients did really well and 700 of those patients did really badly and I just deleted those patients who didn’t do very well and only reported on the 300 who did really well, then obviously I would be laughed out of town and people would say that’s really obviously research fraud because you are selectively deleting the data points in this one single study that you don’t like.  But what’s bizarre is, if you do that by wiping a whole trial from history then for some reason that’s not regarded as research fraud, for some reason that’s regarded as some kind of marginally questionable behaviour but to me it’s unambiguous: you know, you are poisoning the well and the really sad thing about that is there’s lots of really good quality evidence that I have written about in the column a lot, showing that the claims made in adverts – direct-to-doctor adverts – in medical academic journals, that the claims made in those adverts aren’t supported by proper evidence.  And there’s lots of evidence as well that the kind of stuff you get from drug reps or from paid key opinion leaders who are working on behalf of the pharmaceutical industry, that that evidence can be dodgy too.  But the thing is I feel like I can kind of ignore that stuff, but when you poison the well, when you distort the clinical evidence available out there that doctors use to make decisions about what’s the best treatment for the person in front of them, there’s nothing I can do about it.  Just today in fact, there was an absolutely outrageous example published in the British Medical Journal, which is free to access if anyone wants to read it at http://www.bmj.com, a drug called roboxitine, which is a fairly unusual anti-depressant, it’s a noradrenergic drug, and this appeared to be a reasonably effective, and reasonably side-effect-free anti-depressant drug and it’s been on the market for many years in the UK.  It just turned out today a bunch of people from Germany went out and collected all of the data that they could opportunistically from every source they could possibly find and badgered the company and badgered various different places to get unpublished data.  And they found that 74% of the clinical trial data on this drug had been left unpublished.  74% of the clinical trial data on this drug was unpublished!  And when they factored this in, the picture on roboxitine completely changed.  It actually turned out that it’s no good, it’s no better than placebo, it’s basically worse than other anti-depressants and god knows anti-depressants are basically pretty rubbish drugs for treating mild and moderate depression, anyway.  It basically is either useless or worse than useless.  I kind of feel like, as a doctor, I think I’ve prescribed roboxitine myself to people in the past, you kind of think “how am I supposed to know whether something works or not if not by reading clinical trials, and it seems obscene to me that there’s no law saying that you have to publish everything that you produce on a drug.”

The paper Goldacre cites (Eyding et al., 2010) can be downloaded from here, although there are a number of publications (including related comments at the end of that article) which have commented on the issue.  Another notable case of this is in a meta-analysis conducted on 12 anti-depressants which demonstrated that 31% of studies were unpublished (Turner et al., 2008).  Furthermore, the unpublished studies were almost all negative while the published studies were almost universally positive.  The bias in publication strongly affected the apparent efficacy of the drugs.

As Goldacre points out, it is crazy that there is no law to mandate the registration of clinical trials.  Elsewhere in the interview he points out that, even if those trials weren’t published, at least we would know that they existed!  However, even a legislated mandate to register trials (which has now been introduced in the USA, see http://clinicaltrials.gov/ with over 110,000 trials) does not resolve the issue.  There are pre-existing recommendations based on bodies of evidence that are demonstrably incomplete.  Current recommendations for drugs would not be affected by these new legislative tools because the trials were conducted before the introduction of the legislation.  This will be the subject of a British Medical Journal special issue, to be published at Christmas.

I was interested in what the current state of trial registration was in Canada and so I consulted Health Canada’s website on the issue.  There were two consultations on the issue in 2005 and 2006.  Without going into the reports, the websites suggest a lot of “issue identification” and “exploration of options” without any actual progress being made.  In fact, there has been no update on progress since a notice in Nov 2007 which stated that:

…Health Canada encourages sponsors to register their clinical trials of therapeutic products within 21 days of the trial’s onset… on one of the following publicly accessible registries of the World Health Organization’s Register Network:

• ClinicalTrials.gov (www.clinicaltrials.gov)
• Current Controlled Trials International Standard Randomised Controlled Trials Number Register (www.controlled-trials.com/isrctn)

Clearly there is almost no incentive for the sponsors of clinical trials to register those trials.  When I asked Health Canada for a comment, I was told:

This method of registration has shown to be successful given the steady increase of registered trials since 2007. Please be assured that we continue to assess regulatory options to mandate clinical trial registration.

An increase in registered trials is not the same as all trials being registered.  It isn’t even the same as an increase in the proportion of trials being registered.  Clearly the Canadian Government has no idea how many trials are currently not being reported and is relying on an honour system involving pharmaceutical companies whose honesty has been called into question.

Fortunately, there has been some progress.  The “Tricouncil” (CIHR, NSERC and SSHRC, the three major research granting bodies in Canada) has made the registration of clinical trials mandatory under Article 11.3 of the new 2010 Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans: “All clinical trials shall be registered before recruitment of the first trial participant in a recognized and easily web-accessible public registry.”  This is good news, but a quick search through www.clinicaltrials.gov suggests that of 112,102 clinical trials that had been registerd, 44,950 (40%) were registered by industry, 19,137 (17%) were registered by the US National Institute for Heath, 2,918 (2.6%) were registered by other US Federal agencies, and 45,097 (40%) were registered by “all others (individuals, universities, organisations…)”.  If the figures are representative of the situation in Canada, there is still a sizeable proportion of trials that do not have to be registered.

Clearly more needs to be done to ensure that the Canadian people are safe from the problems of dangerous or ineffective medicines.  Given that there is already a scheme that would contribute towards solving the problem in the US, why don’t we have it here?

_______________________________________________________________________________

References

Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., Kerekes, M.F., Gerken, M. & Wieseler, B. (2010) Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials, BMJ, doi:10.1136/bmj.c4737.

Turner, E.H., Matthews, A.M., Linardatos, E., Tell, R.A. & Rosenthal, R. (2008) Selective publication of antidepressant trials and its influence on apparent efficacy, New England Journal of Medicine, 358: 252-260.