Psychotropic medication efficacy and publication bias

April 3, 2012 5:47 AM

Antipsychotics: "The magnitude of publication bias found for antipsychotics was less than that found previously for antidepressants, possibly because antipsychotics demonstrate superiority to placebo more consistently."
Antidepressants: "We found a bias toward the publication of positive results. Not only were positive results more likely to be published, but studies that were not positive, in our opinion, were often published in a way that conveyed a positive outcome. [...] Using both approaches, we found that the efficacy of this drug class is less than would be gleaned from an examination of the published literature alone. According to the published literature, the results of nearly all of the trials of antidepressants were positive. In contrast, FDA analysis of the trial data showed that roughly half of the trials had positive results." Previously

Previous news about antipsychotic efficacy from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study.

posted by OmieWise (34 comments total)

When I was doing science for a living, it always seemed that it was more difficult to publish "negative or questionable results." It'd be nice if things had changed with the development of newer online publishing models.
posted by exogenous at 5:52 AM on April 3, 2012 [1 favorite]

Well, maybe the tendency to publish only positive results in antidepressant studies is an attempt to present a positive face on treatment, reducing the anxiety and negative feelings of patients. Yeah, that must be it.

More seriously, nothing I read about the pharmaceutical industry makes me feel good about the medication I take. No wonder people end up drinking bleach and wearing copper bracelets rather than getting medical help.....
posted by GenjiandProust at 5:59 AM on April 3, 2012

I recently had a conversation with someone in the biochemistry department at Cambridge. They flatly stated that if you want continued funding results had better be positive, and that elimination of possibilities was not viewed as a useful result so much as cause for suggestion that more impressive results might be required if people wanted to move from PhD student to post-doc.
posted by jaduncan at 6:08 AM on April 3, 2012 [1 favorite]

yeah, just like in every other industry. try going to upper management with poor stress testing results on a new software app when the VP of Sales has already started selling it.
posted by spicynuts at 6:25 AM on April 3, 2012 [2 favorites]

try going to upper management with poor stress testing results on a new software app when the VP of Sales has already started selling it.

I've done that more than once. The results have always been pretty good (i.e. delayed shipping to fix the problem). If they hadn't been, I'd have started looking for a new job. Not in a "I'm out of here!" the next day way. Just...I wouldn't be happy at that workplace anymore and I'd start checking want ads.
posted by DU at 6:28 AM on April 3, 2012

The real scandal with antipsychotics is not so much the efficacy, but the horrendous, often life-shortening side-effects.
posted by the young rope-rider at 6:31 AM on April 3, 2012 [6 favorites]

Without knowing anything about chemistry or neurotransmitters, I can attest that with both antipsychotics and antidepressants, there tends to be a lot of switching between medications for specific patients before finding something that's a good match for that individual. Testing for efficacy of a single drug would seem to be pretty difficult.
posted by shakespeherian at 6:35 AM on April 3, 2012 [2 favorites]

Don't want to derail, DU, so I leave it at 'you checked the want ads lately?'
posted by spicynuts at 6:35 AM on April 3, 2012

I would posit that depression is a harder nut to crack chemically, because it's not just a chemical problem in many sufferers. You are going to get population results with larger standard deviations, and some populations, when summed together, are going to show no better than placebo, because while the drug might be great for 10% of the participants, their success can't outweigh the lack of success with the other 70%. (The remaining 20% being the placebo effect people.)

That doesn't mean the drugs don't work, it just means they don't work for everyone, and the goal should be to figure out why so that patients can be matched with the most effective drug.

And yes, there are structural issues in the "science industry" that don't always make this possible.

The real scandal with antipsychotics is not so much the efficacy, but the horrendous, often life-shortening side-effects.

Yeah, that's a big thing. The newer antidepressants are like the newer antihistamines. We already have drugs that work pretty well, but the side effects are often intolerable. So newer drugs are designed not to be more effective, but to try to eliminate side effects while retaining some efficacy.
posted by gjc at 6:37 AM on April 3, 2012 [2 favorites]

Thus the "not the next day".
posted by DU at 6:37 AM on April 3, 2012

I used to do drug trials when I first came back to Toronto, in order to make some extra dough, and I can tell you that the particular company I was at was completely unscrupulous. The studies done there were almost always to learn the effects of various pharmaceuticals on casual (illegal) drug users. We'd go in for two nights each week, and get paid about $600 tax free per visit, for between eight to ten weeks. They'd feed us good food and often, great drugs, to produce the results that the drug company wanted to see. Prior to the study's commencement, there would by 'tryouts' for potential participants. Since they needed a certain amount of people for the studies, and they were looking for people who were already drug users, it was often tough to find good candidates (people who could stay off the drugs for the duration of the study). This meant that during the preliminary weeding out process, nurses would stand behind us and coax us to answer questions in a way that would get us into the study.

Basically, we'd go in on the first night, and not be allowed to eat anything after midnight. Then we'd wake up, and almost immediately sit in a room together where they'd either drug us or placebo us about ten minutes apart. We'd have to sit in these chairs all day, and depending on what you received, enjoy your time doing sweet fuck all, or read a book, bored out of your skull. The nurses would have to do testing on us once an hour (if I remember correctly). This meant taking blood, checking blood pressure, etc. These tests were often missed, yet results were always marked down to fit what the customer (drug company) wanted them to be. If blood pressure was taken and it wasn't within the range they needed, they'd take it again five minutes later, and write it down as if it were done on time. Thing is, when you're not allowed to eat anything for fourteen hours, you already have a weird buzz going on, so the answers we'd be giving on their tests as to the effect of the drug/placebo, were often not valid.

We'd have to give a urine test on the first evening in order to see if we had any (illegal) drugs in our system. People would often fail, and be sent home. Based on the agreements we signed, that was supposed to ban them not only from the current study, but all future studies. This rule was always waived on a first offence, and the subject would be allowed back the following week. They had an extra week added on at the end of the study for all those who failed during the study. They'd then back date every subsequent week after failure for these people, so that the study didn't look fucked up. Those who failed, would receive advice from them on how to pass urinalysis if they "fucked up again".

Oh yeah, and one last thing I remember, is the fact that when I was applying for my first study, and I wrote down that I had had malaria on the questionnaire, they asked me to reconsider my answer. Corrupt fuckers.
posted by gman at 6:40 AM on April 3, 2012 [14 favorites]

I don't think antipsychotics are going to help me with my irritable bowel syndrome though.*

*Only partially trying to be humorous here.
posted by The 10th Regiment of Foot at 6:42 AM on April 3, 2012

I find this article depressing. Fortunately, I can always take... oh, no!
posted by dances_with_sneetches at 6:50 AM on April 3, 2012 [3 favorites]

shakespeherian: Medication switching is definitely established clinical wisdom, not only for efficacy but also for side-effect tolerance. However, the largest (by a long-shot) controlled trial on the topic of guided switching and antidepressant efficacy in the community has yet to produce significant findings concerning how psychiatrists should switch patients, or even technically speaking whether switching "helps" over and above remoralization, common factors of supportive clinical amangement, and time (as it notably lacked a placebo arm, which really, really, really sucks). I'm not aware of any placebo-controlled research on switching, and I'd be really rather interested to hear about it!
posted by Keter at 6:51 AM on April 3, 2012 [1 favorite]

We found a bias toward the publication of positive results

I'm shocked. Shocked.

Your winnings, sir.
posted by erniepan at 6:59 AM on April 3, 2012 [2 favorites]

Ahead of the "Well antidepressants are BULLSHIT and don't work so no wonder you have to dress them up in publications" comments, let me just say that for me, for whatever reason, the right antidepressant totally fucking works (and the wrong ones totally don't), so just sayin'.
posted by Rykey at 7:06 AM on April 3, 2012 [9 favorites]

For an extra helping of depression about, um, depression: the effects of psychotherapies for depression may also be overestimated due to publication bias. Though this article uses theoretical measures of publication bias based on knowledge of effect size distributions rather than actual found data like the studies above.
posted by Keter at 7:10 AM on April 3, 2012

It took about eight years trying almost every psychoactive drug that exists to find a combination of drugs that control my bipolar disorder and other problems fairly well without debilitating side-effects. The patient-to-patient variation for these medications is wild. Zyprexa and Prozac had few side effects but did little for me while Effexor made me want to tear my skin off. My sister almost committed suicide when she started Prozac for depression but thinks Effexor is a godsend.

It would be wonderful if there was a test to somehow map a persons neurotransmitter/neurochemical makeup to give them the right medication(s) right away. Not everyone has good insurance, access to a good shrink or the ability to comply with and tolerate numerous medication changes to find the right combination. This in addition to the fact that there is an effectively infinite number of combinations and that for some people, there might not even be a right combination yet. For those of us with difficult to treat issues, the odds are not stacked in our favor.

I used to work for Big Pharma and yes, they can be evil when it comes to market manipulation. Even so, I don't see how it would be possible to devise a meaningful study about the efficacy of any of these drugs, given huge variation of reactions among the population, even if the given drug company didn't have its thumb on the scale.

I still think it's better for me to take a handful of pills that aren't even close to perfect but allow me to live a close-to-normal and productive life than the old system that would have thrown me into a mental institution to rot and thrown away the key.
posted by double block and bleed at 7:39 AM on April 3, 2012 [11 favorites]

I don't think antipsychotics are going to help me with my irritable bowel syndrome though.

Given that blood pressure medicine has been used to effectively treat baldness and ADHD, that's further from a sure thing than you might think.
posted by Kid Charlemagne at 7:50 AM on April 3, 2012

I find myself in the awkward position of regularly needing to defend science, as a whole, against the paranoia of an acquaintance who's been convinced by quacks that he can cure things like cancer and pneumonia on his own with quack medicine, and yet also holding a doctorate which got me close enough to the publishing/grantmaking/tenure processes to want people to know about the potential bias of studies which hit the news regularly. It's frustrating. I feel like if more people were exposed to science as an ongoing, living, collectively-created thing from an early age, it would be easier for all of us as a society to collectively correct for these kinds of biases.
posted by gusandrews at 8:19 AM on April 3, 2012 [5 favorites]

Kid Charlemagne: "I don't think antipsychotics are going to help me with my irritable bowel syndrome though.

Given that blood pressure medicine has been used to effectively treat baldness and ADHD, that's further from a sure thing than you might think."

Clonidine is another hypertension drug with great (for me) psychoactive qualities. Too bad it sends me to sleepyland.
posted by double block and bleed at 8:20 AM on April 3, 2012

Can I just say that whoever invented Zyprexa, I'd like to shake that dude by the hand.
posted by colie at 10:22 AM on April 3, 2012 [3 favorites]

Every field needs a Rejecta Mathematica to shine light on dead ends and false leads.
posted by Slackermagee at 10:54 AM on April 3, 2012 [1 favorite]

Yeah, having helped treat a lot of people who suffered from depression, various forms of psychosis, and both (comorbidity was the rule where I worked), I found this to be the case, strictly anecdotally at least.

I agree with others that depression is a more squirrelly condition (and diagnosis), with many social and emotional components. I found that with psychotic patients, much of their social and emotional problems descended from the fact that they were totally unable to operate in the normal world.

I've seen changes so profound that antipsychotics are my go-to item when it comes to discussions of medication and such. We're talking babbling, hallucinating, and violent to lucid, calm, and polite over the course of a couple days. That's a real goddamn accomplishment of modern medicine. But not every mental illness is as susceptible to chemical disruption.
posted by BlackLeotardFront at 11:03 AM on April 3, 2012 [2 favorites]

I can attest that with both antipsychotics and antidepressants, there tends to be a lot of switching between medications for specific patients before finding something that's a good match for that individual.

Needing to find the right drug for the individual patient might account for the low efficacy of individual antidepressants. However, since depression is usually a temporary condition, it's possible that it some (or many?) cases, physicians are simply switching drugs until the patient experiences spontaneous remission.

Depression is usually attributable to problems in the patient's life (which they have an emotional reaction to). So we can imagine this scenario: Person is unhappy with their job. The situation seems hopeless. They lose motivation and energy. Their unhappiness and lack of energy interfere with their ability to enjoy other aspects of their life. In other words, they get depressed. They go to their doctor, who tries various antidepressants. None of them seem to work. The patient gets another job, and starts feeling much better. The last antidepressant they were on is Paxil, so Paxil cured their depression and turned out to be right drug for this particular patient. Unfortunately, if both the patient and their physician buy the idea that 'depression is a chemical imbalance in the brain' (without getting that the chemical imbalance is usually caused by something), they may believe that the patient needs to stay on Paxil indefinitely (or whatever the last drug was) to 'make sure the chemical imbalance doesn't come back'.

It would be possible to test this, for antidepressants as a class, by giving some patients the typical regimen of trying different drugs and dosages until something works, other patients a randomly selected antidepressant (without changing it), and third group a placebo. It would also be possible, and probably a better experimental design, to include placebos in a blinded and randomly selected sequence of drugs to try, and see to what extent placebos are selected as the 'drug that worked' versus the real drugs. As far as I know, nothing like this has been tried.
posted by nangar at 11:29 AM on April 3, 2012 [2 favorites]

http://www.chinapost.com.tw/health/cancer/2012/04/02/336431/Unduplicable-cancer.htm

http://www.reuters.com/article/2012/03/28/us-science-cancer-idUSBRE82R12P20120328

A lot of science uses the wrong method in pursuit of profit and fame and pursuit of tenure.
posted by sensi63 at 11:44 AM on April 3, 2012

I do think there are cases in which a person is not sane enough to make medical decisions, but I also think that when clients want to create a non-pharmaceutical health plan that we should have better clinical research on non-pharmaceutical solutions.

One of which being social support, which is an aspect of treatment that seems so glaringly obvious and yet is all but absent in mental health treatment.

You have a orphan that just got dumped out of foster care and they are feeling "psychotic" and you think the most important and only thing they need is a med? No consideration of normal response to abnormal circumstance?

And I'm not talking about therapy. In order to feel like society has it's doors open to you, you need to feel like you are more to society than a patient. You need actually human beings to be involved in the support.

"You have to have love, to help people."

Also there's this:
"they found that in the three developing countries-India, Colombia, and Nigeria-only 16% of the patients were regularly maintained on antipsychotics, compared to 61% of the patients in the developed countries. Outcomes were better in countries where patients weren't regularly maintained on the drugs."

There are a lot of debates about why this is, one of which being the impact of medication itself. However another concern is that in the west our goal is to have a person working and living independently, when people with schizophrenia and severe mental illness may in fact have positive low symptoms lives when socially supported living with others and being involved and accepted socially. I hope there's a lot more research done on what is going on with these outcomes and various measurements of what is a positive outcome.

Another example is a study a little birdy told me was done by employees of Lilly-- that found developed nations had better "functional" outcomes--- people were more symptomatic and worse accross many quality of life variables but they were employed and living indipendantly which is a western centric view of being "recovered"
study-- sorry pay wall
posted by xarnop at 11:45 AM on April 3, 2012

Depression is usually attributable to problems in the patient's life

I don't really think this is true for clinical depression. It certainly isn't true for any of the many people I've known with depression. Do you have a cite for that?
posted by shakespeherian at 12:10 PM on April 3, 2012 [7 favorites]

Would that someone, somewhere were brave enough to get funded and undertake a large enough, long enough randomized-placebo trial of psychotropic therapy for a psychiatric condition. I really want to know whether any of these drugs actually work. And if they could throw in a talk therapy arm, that'd be great. kthxbai
posted by Mental Wimp at 12:28 PM on April 3, 2012

"Depression is usually attributable to problems in the patient's life"

I also would like to see some citations for this. Suffering from profound feelings of sadness and worthlessness for NO REASON is one of the classic symptoms of depression.
posted by antiquated at 2:45 PM on April 3, 2012 [1 favorite]

And, of the "positive" studies that are published, the data is massaged if not outright invented (fraud). From time to time a "major, respected journal" has to formally retract a study, but only after it becomes 100% and 1000% certain that the study is a fraud.

(Gee this reminds me of something similar, what is it, what is it, hmm, I feel like I'm getting warmer trying to remember, hmm)
posted by caclwmr4 at 5:01 PM on April 3, 2012 [1 favorite]

Does it remind you of how you don't believe in science?
posted by shakespeherian at 6:48 PM on April 3, 2012

Do you have a cite for that?

I was relying on the distinction between reactive and endogenous depression. It turns out this classification is regarded as outmoded. (I was an undergraduate psych major in the early 90s.)

Anyway, Wikipedia has a brief explanation of the endogenous/reactive thing.
posted by nangar at 5:15 AM on April 4, 2012

FTA: The relationship between trial outcome and publication status, shown in Figure 1, did not reach statistical significance (Fisher's exact test p = 0.09).

They are being really good about this description. If this paper weren't being written by statisticians, I bet you $20SAIT that this would have been called "a strongly suggestive trend." (Especially since Fisher's test can be a little conservative, and there were only 4 unpublished studies.)

The typical vs. atypical antipsychotics thing is interesting too. I thought the major controversy was that the atypicals didn't actually have fewer side effects or didn't actually work better, not necessarily that they didn't work period. And actually, in the one comparison included in this analysis (haloperidol vs. ziprasidone), the typical AP came out on top. I'd love to see more of that type of comparison.

It is at least somewhat reassuring that antipsychotics seem to work more consistently than antidepressants, since psychosis is such a totally disabling condition. I would totally want to be put on an antipsychotic if I ever had a psychotic break (though of course I might not feel that way at the time).
posted by en forme de poire at 8:32 AM on April 4, 2012