Effectiveness of antidepressants research and journal publications

[lallieth]

Effectiveness of antidepressants exaggerated
17/01/2008
CTV.ca

A new study finds antidepressants may be less effective than doctors have been led to believe. That's because it appears that studies that have found the drugs do little to help with depression are not being published.

And if studies that have found less-than-positive results have been published, they have been written as if they were positive.

The finding comes from a new analysis in the New England Journal of Medicine.

Dr. Erick Turner, an assistant professor of psychiatry at Oregon Health and Science University led a team who reviewed studies that the U.S. Food and Drug Administration has done on 12 widely prescribed antidepressant drugs.

His team looked to see whether results of these studies had been published in medical journals. They then compared the published version of the results with the FDA version of the results.

Turner's team found that nearly all the studies (94 per cent) that were published in journals suggested positive treatment results. But FDA data showed that in fact only 51 per cent of the studies were positive.

Of the 74 studies conducted on the 12 antidepressants, 38 produced positive results for the drug. All but one of those studies were published. But only three of the 36 studies with negative or questionable results were published. Another 11 were written as if the drug were effective.

Turner says this is just another example of "selective reporting," in which good news about a drug is made public but the bad news isn't.

"Selective publication can lead doctors and patients to believe drugs are more effective than they really are, which can influence prescribing decisions," said Turner.

Turner doesn't believe that the antidepressants studied are ineffective. In fact, his team found through an analysis of all the studies done on each drug that the medications were superior to treatment with placebos (sugar pills).

But after reviewing the FDA data on the drugs, they found that the published literature often exaggerated the drugs' effectiveness.

Turner said that he didn't know whether study sponsors chose not to submit manuscripts for studies that had less-than-positive results, or whether the journal editors and reviewers rejected the studies.

"Regardless, doctors and patients must have access to evidence that is complete and unbiased when they are weighing the risks and benefits of treatment," he emphasized.

 

[David Baxter PhD]

Re: Effectiveness of antidepressants exaggerated

This study has been widely publicized on the internet and in various newspapers, and I meant to post it myself with some commentary when I had more time.

The bottom line is I think it's misleading on a few levels:

1. Yes, there is a danger that any scientific journal may publish data that has been intentionally altered. There is a lot of pressure to publish among academics and researchers, and that pressure to publish has significant career implications for many. That is why the good journals are peer reviewed. It is also why replication is important and why no one should draw seriuous conclusions on the results of any single study. This is simply how science proceeds when it works.

2. The author of this article makes a lot about the fact that there are negative findings that do not get published and calls this "selective publication". In a way, that's true. But there's a very good reason for it: The easiest way to NOT get an expected finding or to fail to replicate someone else's finding is sloppy research. It's called proving the null hypothesis - a well-designed experiment is set up to disprove the null hypothesis, in this case to try to disprove the hypothesis that the medication is useless. Now, if I do such an experiment and my results prove the null hypothesis, i.e., show that the medication doesn't work, what does that mean? Well, of course, it might mean that the medication is worthless. But it might mean that my experiment is worthless.

As an example, my Ph.D. thesis ended up showing the results of five experiments. I actually did quite a few more than five but five was eniough to make the point. What actually happened was that for several years I completed experiments designed to test certain predictions from accepted theory at the time about types of learning and how we learn. Repeatedly, I failed to get the expected findings. Now the question was why. Again, the best and most likely explanation was that I was doing something wrong, that either my experimental design or mthods were flawed. That's why I persisted, trying different variations, for about 10 completed studies if memory serves and a few more partial studies.

After a while, I went back to the theory and started looking at whether there was something I was missing. And what I realized was that the theory I was testing was wrong. By adjusting the theory, I was able to make new predictions or hypotheses about the conditions under which one should expect to get "positive" or "negative" results. I then tested the new revised theory and was able to demonstrate that my revised theory was supported by data. And that of course turned out to be my thesis.

So getting back to this article:

1. Does it mean that antidepressants don't work? No, clearly not. The author of the study himself acknowledges that.

2. Does it mean that there is some sort of bias against negative findings? In a way, yes, but that is true in all areas of science for the reasons I've mentioned. Negative findings need to be replicated repeatedly in the absence of positive findings in order to be published because it is well understood that it is easy to get negative findings by simply conducting a badly designed or badly executed experiment. Does this mean there is some sort of conspiracy to support pharmaceutical drugs and deceive consumers? Absolutely not. Indeed, if you think about how many people would need to be involved and the likelihood of a public leak, it is difficult to imagine this being feasible.

3. Do we want to know about studies that show certain drugs to be ineffective? Yes and no. If it's a bad study, no. It tells us nothing. If it's a failure to replicate other research, and it has adequate design and methods, and it is possible to pinpoint how it differs from positive studies, then yes - but it's not enough to publish the negative finding on its own - it's also important to examine when and where and with whom the drug works or doesn't work, similar to my Ph.D. thesis research.

The bottom line is that if a drug is truly ineffective, it will be exposed. Eventually. As more evidence accumulates. Government departments such as the FDA and Health Canada are charged with examining the efficacy of any new medication in treating what it claims to treat, and getting approval for a new drug is neither easy nor quick. So once the drug is released to the public, it has already undergone considerable scrutiny, not only as to effectiveness but also as to safety and risks.

I think these articles are sensationalist and misguided, to be frank. I think they will suggest to the consumer a conspiracy that doesn't exist.

And I'd also point out that the willingness of the New England Journal of Medicine to publish this article is evidence that the implications against the sort of conspiracy that some journalists are trying to suggest exists.