Crowd Sourced Efficacy Response in Depression Treatments

October 7, 2010 11:10 AM

An interesting graph based on the results of an informal user poll as to the response/efficacy to various treatments for depression. "Fish oil, also popular, showed up as much less effective than [...] expected."

Neither scientific nor exhaustive (from an ongoing response base of 944) the graph gives a clear representation of what people have found to be effective in their own treatment. Here's a more descriptive version of the graph that labels each plot point. Fill out the survey to widen the sample base.

From the CureTogether Blog - "...patients around the world coming together to share quantitative information on over 500 medical conditions."

posted by gallois (43 comments total)

> "Fish oil, also popular, showed up as much less effective than [...] expected."

Few things on this site have pissed me off more than when I see some dope tell someone asking about a depression/mental health issue on AskMe say "oh, just take some fish oil".

Also, it's good to see that (according to the graph) good old-fashioned getting-off-your-ass-and-moving is still tops.
posted by Burhanistan at 11:18 AM on October 7, 2010

Neither scientific nor exhaustive

I'm glad you know that. I hope everybody else who sees it does too.
posted by Horace Rumpole at 11:19 AM on October 7, 2010 [8 favorites]

I'm going to start a blog so I can crowd-source confusion. I'll name it "Confustion" which is an amalgam of confusion and combustion.
posted by kuatto at 11:20 AM on October 7, 2010 [2 favorites]

Yep, exercise'll do it.

Assuming you can get out of bed, and stomach enough breakfast to get you going for the session, and don't just find it all too bloody hard half way through, causing you to go home to take your totally bloody useless fish oil and then lie on the couch eating mars bars and watching simpsons reruns..

Bloody scienticians with their bloody fancy graphs.
posted by Ahab at 11:22 AM on October 7, 2010 [15 favorites]

I saw this yesterday, and the graph itself (both the original the hackjob floating around) irritated me to no end. Basically, it's a poor "study" made even worse by putting together a poor graph with an arbitrarily chosen baseline.
People looking at this chart and the breathless verbiage will just come away re-enforcing their own beliefs.
posted by Old'n'Busted at 11:24 AM on October 7, 2010 [2 favorites]

Wow, exercise is really an outlier there. All the science I've read (directed at laypeople) puts it about on par with SSRIs and talk therapy, not well above it.

Hard to know what the self-reporting biases are, of course.
posted by callmejay at 11:24 AM on October 7, 2010 [1 favorite]

Stop ragging on my days offs, Ahab.
posted by ZaneJ. at 11:24 AM on October 7, 2010 [1 favorite]

> Assuming you can get out of bed, and stomach enough breakfast to get you going for the session

Yeah, there definitely can be a cart-before-the-horse thing going on with exercise in many cases.
posted by Burhanistan at 11:25 AM on October 7, 2010 [4 favorites]

Not worth the paper it's not printed on.
posted by 2bucksplus at 11:25 AM on October 7, 2010 [1 favorite]

Yeah, the methodology basically renders this worthless. Bad data presented in a spiffy graph is still bad data.
posted by ixohoxi at 11:29 AM on October 7, 2010 [4 favorites]

Exercise fixes .7 depressions per unit!!
posted by Babblesort at 11:32 AM on October 7, 2010

This is rubbish, for a lot of reasons, but I'll focus on just one: it mixes up radically different types of interventions. Among the interventions are "Cold Shower," "Vitamin D3 400 IU Daily," "Dialectical Behavioral Therapy," "Caffeine," and a number of pharmaceuticals, including Prozac, Wellbutrin, and Celexa.

There is absolutely no standardization between these kinds of interventions. Does "Cold Shower" mean that if you take one cold shower you feel less depressed for the hour after the shower? For a day? For a week? Or does it mean that if you take cold showers every day for a few weeks your feel less depressed overall? there is no controlled "dosage" of cold showwers in the same way that there is of pharmaceuticals. No one claims that pharmaceuticals produce an immediate effect, and most of them require a few weeks to show any effect at all. Behavioral and other talk therapies also require consistent exposure to produce an effect.

Similarly, what does "caffeine" mean? A cup of coffee every once in a while, or 10 cups a day for weeks? What does "exercise" mean - going for a jog makes me feel good, or daily exercise reduces the symptoms of more severe depression?

Finally, who is doing this? Someone who is severely depressed may not even be able to exercise, so the "efficacy" may be the product of the fact that comparatively 'healthier' people are the ones who exercise, while the 'sicker' people are the ones who try flaxseed oil.

Look, I'm a fan of crowdsourcing, I love infographics, and I'm a big believer that effective presentation of data is useful and important. But in this case, the nifty graphic conceals information that not only fails to be scientific or exhaustive, but also fails to demonstrate anything at all.
posted by googly at 11:43 AM on October 7, 2010 [15 favorites]

An infographic is just an anecdote dressed in a suit.
posted by TwelveTwo at 11:49 AM on October 7, 2010 [6 favorites]

"Fish oil, also popular, showed up as much less effective than [...] expected."

They should have tried snake oil. I hear that's much more effective ...
posted by moonbiter at 12:05 PM on October 7, 2010 [1 favorite]

As far as I know, the recommended dosage on most bottles of fish oil is a ridiculously small fraction of the dosage that has been (anecdotally, sometimes) shown to be effective in treating depression.

ANECDOTALLY I have personally encountered many people taking one or two capsules per day (~500 mg) and expecting to see a difference, when the proper dosage (if you think fish oil might be useful and you want to give it a try) should be in the high single digits of grams.

That said, I enjoyed this anecdotal infographic. It's a nice change from the OKCupid infographics.
posted by telegraph at 12:11 PM on October 7, 2010

Observations in no particular order

* eyeballing the graph, effectiveness is apparently positively correlated with popularity - doesn't that merit some reflection?

* mindfulness meditation is distinguished from plain old 'meditation'; for that matter, CBT, DBT, psychotherapy, and talk therapy are four distinct data points, though what it's not clear what distinction this implies

* Zoloft differs from SSRIs by a large margin - but Zoloft IS an SSRI!

* on a related note, it makes no sense to say 'Zoloft's effectiveness is 0.4' - how much Zoloft, how often? Does that include everyone who stopped taking it? What about side effects?

* the y-axis is meaningless absent specific information about how 'effectiveness' was operationalized - I gather that it's some flavor of patient rating, but I'd like to know more

* while I realize that this is largely a promotional tool, it's disappointing that there is no self-awareness about the limitations of this survey. Despite its rather obvious lack of rigor, recommendations are breezily tossed out: "Those in the lower-right quadrant have above-average usage but below-average effectiveness, so presumably fewer people should be using these." But is the difference between e.g. DBT (~.38) significantly different than Wellbutrin (~.32)? Enough to recommend that more people dump their pills and take up group therapy? My kingdom for some inferential statistics!

* on the topic of recommendations, if the only significant difference between e.g. fish oil and magnesium is the former's higher popularity, why is that a reason to recommend that people quit using fish oil? Why shouldn't - or should, for that matter - people stop trying magnesium?

* what would possess anyone to breathlessly include 'cold shower', 'housekeeping', and 'caffeine' on a chart labeled "Depression Treatments"? If I'm depressed, does anything I do possibly count as a treatment?

Good luck to anyone trying to figure out how to treat your own depression - you're going to need it.
posted by mister-o at 12:16 PM on October 7, 2010 [1 favorite]

It's a nice change from the OKCupid infographics.

Yes, it's even more useless.
posted by Old'n'Busted at 12:22 PM on October 7, 2010

Yep, exercise'll do it.

In Dolly Freed's Possum Living, she describes how she'll occasionally hear a doom-and-gloomer on the radio and think "What wouldn't a 5 mile run do for this bastard?"
posted by Joe Beese at 12:53 PM on October 7, 2010

Well, god forbid that people suffering from depression self-report what actually works or doesn't work for them, in a less-than-scientific study.

Let's get back to totally disinterested big pharma shills crunching the data instead - that way all the pharmaceuticals bunched down in the bottom left quadrant can take their rightful place above what people say actually works for them: exercise, adequate sleep, talk therapy, meditation, spending time with a pet, psychotherapy, CBT, yoga, relaxation, housecleaning, regular eating times, talking with family & friends, regular meals, and avoiding intoxicants.

There's too much anecdotalism here. We need more lies, damned lies and statistics.
posted by UbuRoivas at 12:59 PM on October 7, 2010 [5 favorites]

Exercise, adequate sleep, meditation, yoga, housecleaning, and talking with family & friends all got a hell of a lot easier to do when I started taking antidepressants.
posted by brookedel at 1:19 PM on October 7, 2010 [4 favorites]

Every time someone mentions fish oil I think of this guy and I hear him saying "Boomerang fish oil" in my head. That's remarkably effective in diminishing my depression - for at least long enough that I can have a good chortle, anyway.
posted by Greg_Ace at 1:27 PM on October 7, 2010

all got a hell of a lot easier to do when I started taking antidepressants.

To be fair, SSRIs were up there in the top RH quadrant, and Zoloft just sneaked in.

You can actually see what all the dots represent by hovering in the live-updated, interactive version of the infographic.
posted by UbuRoivas at 1:31 PM on October 7, 2010

Well, god forbid that people suffering from depression self-report what actually works or doesn't work for them, in a less-than-scientific study.

The second you aggregate data from a non-scientific study is the second you create a number that means nothing. Leave the self reports as reports.
posted by smackfu at 1:39 PM on October 7, 2010 [3 favorites]

The self-reports are important in the sense that different treatments work for different individuals, but that doesn't make the resulting graph anything like useful data.
posted by immlass at 2:29 PM on October 7, 2010 [1 favorite]

The graph says nothing more than that the frequency of fish oil was mentioned more among those who self-selected into the survey. "The people" aren't telling us anything, unfortunately.

The Internet's generating a lot of data for researchers, but we still have to deal with the fact that the data is wrought with selection problems. Figuring out how to map this stuff onto the population is probably impossible.
posted by scunning at 2:46 PM on October 7, 2010

Unsupported scientific-ish nonsense is both 'Popular' and 'Effective'.
posted by kuujjuarapik at 2:53 PM on October 7, 2010 [1 favorite]

above what people say actually works for them: exercise, adequate sleep, talk therapy, meditation, spending time with a pet, psychotherapy, CBT, yoga, relaxation, housecleaning, regular eating times, talking with family & friends, regular meals, and avoiding intoxicants.

Yes, because people's anecdotes about what they happened to be doing when they started feeling better are totally useful! Who needs controlled experiments? The other day I saw a squirrel and _also_ that day I felt better.... squirrels are the answer!

People may or may not be right about those things, but without some kind of controlled experiments their data really is worthless.
posted by wildcrdj at 3:58 PM on October 7, 2010

I added to the survey results, but I feel some misgivings about the graph and the methodology as well.

If I'm self-reporting, and I've been lying on my ass in the dark eating oatmeal cookies because I'm feeling depressed, and the survey asks, "Does exercise help?" I'm not going to say, "I dunno. Pass me another cookie."

I'm going to say, "Yes, exercise works!", implying to the invisible internet guardians that I am actually out running marathons.

So, yeah, self-reporting can be unreliable.

Even though, yes, when I DO exercise, it helps.
posted by misha at 4:06 PM on October 7, 2010

Self-reporting is like self-medicating. Sometimes it works like it should, other times it doesn't work as it ought, and it just might work when it shouldn't, but it always doesn't work when it doesn't!
posted by TwelveTwo at 4:54 PM on October 7, 2010

without some kind of controlled experiments their data really is worthless.

I'm looking forward to somebody getting approval for an experiment in which people cut off contact with their family & friends, stop doing housework, and let their pets die from neglect - because who could tell what effect that would have on the subjects' mood without rigorous methodology & peer reviews?
posted by UbuRoivas at 5:08 PM on October 7, 2010

in which people cut off contact with their family & friends, stop doing housework, and let their pets die from neglect

Well, I, for one, would get a lot more done in a day.
posted by TwelveTwo at 5:21 PM on October 7, 2010

In Dolly Freed's Possum Living, she describes how she'll occasionally hear a doom-and-gloomer on the radio and think "What wouldn't a 5 mile run do for this bastard?"

Odd, because I hear this and think, "What would a knife fight do for this self-righteous bitch?"

Uh, Sicilian Heritage month!
posted by Kid Charlemagne at 5:34 PM on October 7, 2010

I'm looking forward to somebody getting approval for an experiment in which people cut off contact with their family & friends, stop doing housework, and let their pets die from neglect - because who could tell what effect that would have on the subjects' mood without rigorous methodology & peer reviews?

Well, this certainly is an issue with mental health research, yes. I don't think the answer is just to collect what people claim on the Internet and present that as someone useful or meaningful. It's completely worthless.

Believe it or not some scientists do spend a lot of time thinking about how to do these studies. And they don't all work for pharmaceutical companies. But given the nature of mental health, yes it will be harder than studies for treating cancer or something.
posted by wildcrdj at 5:44 PM on October 7, 2010 [1 favorite]

Few things on this site have pissed me off more than..."oh, just take some fish oil"

...totally bloody useless fish oil...

Given that placebos are extremely effective against depression, getting upset at people who recommend trying something like fish oil seems extremely wrongheaded.
posted by straight at 7:28 PM on October 7, 2010 [2 favorites]

Well, god forbid that people suffering from depression self-report what actually works or doesn't work for them, in a less-than-scientific study.

People self-reporting what works for them is not the problem.

Counting a bunch of self reports and putting the totals on a sciencey-looking chart is the problem.
posted by straight at 7:34 PM on October 7, 2010 [2 favorites]

"Fish oil, also popular, showed up as much less effective than [...] expected."

Ur doin it rong
posted by Jimmy Havok at 2:07 AM on October 8, 2010

> Given that placebos are extremely effective against depression, getting upset at people who recommend trying something like fish oil seems extremely wrongheaded.

Well, for someone who is knocking the methodology of this chart (and rightfully so), the context of my disliking fish oil recommendation was strangers on the internet recommending it to other strangers, so calling that "extremely wrongheaded" is missing the point, and a bit hyperbolic to boot.
posted by Burhanistan at 7:53 AM on October 8, 2010

Burhanistan, I don't understand. Do you have some reason to think using fish oil would be less effective than a placebo?

Fish oil's not very expensive, may have other health benefits related to heart disease, and has no real side effects. Current medical consensus seems to be, "It might help, give it a try if you want." Unless someone is saying "Don't bother with exercise, don't talk to a doctor, don't let them put you on medication, fish oil is all you need," I don't see the problem.
posted by straight at 9:26 AM on October 8, 2010

Heh. I know someone who spent a year in several-times-a-week therapy and taking antidepressants. She started out the year never leaving the house unless someone took her, and never doing anything at all but net surfing. After that time, she moved to a different city, got a job, and lived independently (paying her own way) for the first time ever.

And she swears up and down that the antidepressants and the therapy didn't do anything for her. She's convinced they didn't help--but as her primary caretaker, I can tell you there wasn't anything else she was doing that would help, either. Self-reporting the results of various therapies for depression...yeah, no.
posted by galadriel at 7:01 PM on October 8, 2010

she moved to a different city, got a job, and lived independently (paying her own way) for the first time ever.

Helped my depression too. It only lasts a couple of years, though, then you have to move to another city. Hello drifter lifestyle!
posted by Jimmy Havok at 9:02 PM on October 8, 2010

It's interesting and exciting and horrible all at once. I agree with the basic implications but the graph is squishy- unit free and nearly useless. It's like taking an opinion and rendering it as a graph. Like this:

S|
C|
I|
E|
N|
C|                   x (this study)
E+---------------------                   
     PRETTY

posted by chairface at 9:05 AM on October 9, 2010 [3 favorites]

Helped my depression too.

So you, what, completely missed the fact that before the year of antidepressants and therapy, she couldn't even leave the house? Being independent was great for her, sure, but there's no way she could have accomplished that before getting help.
posted by galadriel at 12:18 PM on October 9, 2010

I'm just saying that getting away from your old environment has a stimulating effect on most people. Maybe it was the therapy and antidepressants, but maybe it was kicking over the traces and getting away from everything that had been dragging at her as well.
posted by Jimmy Havok at 1:08 PM on October 9, 2010