Research methods: the heart and soul of knowledge
November 5, 2014 2:50 PM Subscribe
Hollaback and Why Everyone Needs Better Research Methods (And Why All Data Needs Theory), by Zeynep Tufekci:
I’ve taught "introduction to research methods" to undergraduate students for many years, and they would sometimes ask me why they should care about all this "method stuff", besides having a required class for a sociology major out of the way. I would always tell them, without understanding research methods, you cannot understand how to judge what you see.
The Hollaback video shows us exactly why.
The Hollaback video also shows why "data" without theory can be so misleading — and how the same data can fit multiple theories. Since all data collection involves some form of data selection (even the biggest dataset has selection going into what gets included, from what source), and since data selection is always a research method, there is always a need for understanding methods.Previously
Yeah, that is excellent. Thanks for posting it.
posted by languagehat at 3:02 PM on November 5, 2014 [1 favorite]
posted by languagehat at 3:02 PM on November 5, 2014 [1 favorite]
It's meant as an activist campaign, so the evaluation criteria are different.
Perhaps from the point of view of the activists. However, I'm assuming that the article was making the point that it's important that people are literate in research methods so they can figure out whether activists are trying to sucker us by overstating their case.
posted by PeterMcDermott at 3:12 PM on November 5, 2014 [16 favorites]
Perhaps from the point of view of the activists. However, I'm assuming that the article was making the point that it's important that people are literate in research methods so they can figure out whether activists are trying to sucker us by overstating their case.
posted by PeterMcDermott at 3:12 PM on November 5, 2014 [16 favorites]
good article, but I would object that the Hollaback video is not an academic or scientific research project. It's meant as an activist campaign, so the evaluation criteria are different.
Would it be alright for the New York Times to run a story indicating that Obama is actually Muslim, based on reports from Republicans? They're not an academic or scientific outfit either, after all.
posted by clockzero at 3:14 PM on November 5, 2014 [1 favorite]
Would it be alright for the New York Times to run a story indicating that Obama is actually Muslim, based on reports from Republicans? They're not an academic or scientific outfit either, after all.
posted by clockzero at 3:14 PM on November 5, 2014 [1 favorite]
good article, but I would object that the Hollaback video is not an academic or scientific research project.
I was thinking of posting the same thing, but it occurs to me that the point of the article is illustrative, rather than critical.
Meaning, a teacher is trying to demonstrate the impact that anecdotal data can have on our understanding of the world when theory isn't well applied. So he's choosing a well known example of something that is anecdotal and inadvertently caused a stir due to poor theory and implementation. The idea isn't to say "these people are bad social scientists," the idea is to say "good social science doesn't look like [anecodotal non-scientific thing] because, as we can see, the actual findings get lost in the many interpretations that data can fit."
posted by shmegegge at 3:17 PM on November 5, 2014 [7 favorites]
I was thinking of posting the same thing, but it occurs to me that the point of the article is illustrative, rather than critical.
Meaning, a teacher is trying to demonstrate the impact that anecdotal data can have on our understanding of the world when theory isn't well applied. So he's choosing a well known example of something that is anecdotal and inadvertently caused a stir due to poor theory and implementation. The idea isn't to say "these people are bad social scientists," the idea is to say "good social science doesn't look like [anecodotal non-scientific thing] because, as we can see, the actual findings get lost in the many interpretations that data can fit."
posted by shmegegge at 3:17 PM on November 5, 2014 [7 favorites]
Would it be alright for the New York Times to run a story indicating that Obama is actually Muslim, based on reports from Republicans? They're not an academic or scientific outfit either, after all.
What? It's not like the NYT holds itself up as an activist organization; they pride themselves (rightfully or not) on reporting and research, not advocacy. I feel like you're asking why apples would be elephants.
posted by rtha at 3:39 PM on November 5, 2014 [2 favorites]
What? It's not like the NYT holds itself up as an activist organization; they pride themselves (rightfully or not) on reporting and research, not advocacy. I feel like you're asking why apples would be elephants.
posted by rtha at 3:39 PM on November 5, 2014 [2 favorites]
I am unsure as to whether or not this shit is bananas.
posted by delfin at 3:42 PM on November 5, 2014 [9 favorites]
posted by delfin at 3:42 PM on November 5, 2014 [9 favorites]
I am unsure as to whether or not this shit is bananas.
This is an opportunity to think deeply about possible hypotheses regarding the nature of this shit, and what properties of the genus Musa you expect to be present in it...
posted by selenized at 3:56 PM on November 5, 2014
This is an opportunity to think deeply about possible hypotheses regarding the nature of this shit, and what properties of the genus Musa you expect to be present in it...
posted by selenized at 3:56 PM on November 5, 2014
good article, but I would object that the Hollaback video is not an academic or scientific research project. It's meant as an activist campaign, so the evaluation criteria are different.
Fair enough. But the exact same argument could be made about James O'Keefe's videos. Seems like the message here should be to never take videos at face value, no matter how closely the hew to prior beliefs/experiences.
posted by graphnerd at 4:09 PM on November 5, 2014 [2 favorites]
Fair enough. But the exact same argument could be made about James O'Keefe's videos. Seems like the message here should be to never take videos at face value, no matter how closely the hew to prior beliefs/experiences.
posted by graphnerd at 4:09 PM on November 5, 2014 [2 favorites]
The video wasn't intended as any kind of rigorous study, but this article does a great job of taking ideas about rigorous study and showing how they apply to something outside the realm of research/academia. More rigorous thinking about random content, please!
posted by rmd1023 at 4:15 PM on November 5, 2014 [10 favorites]
posted by rmd1023 at 4:15 PM on November 5, 2014 [10 favorites]
Even though the video wasn't a 'rigorous study,' I think the article makes some excellent points on the problems inherent in it (i.e., the strange disconnect between their statement that harassers came from "all backgrounds" and the video's singular focus on black men).
So I'm torn. The video has triggered some great discussions (at least, here), but my rage-o-meter has shifted from an initial harassment is horrible and must be stopped and this is the perfect video to demonstrate why (though I still believe the first part) to why is Hollaback playing into fears of black men in order to get attention?
Research protocol does matter, even if it's for an activist video.
posted by kanewai at 4:30 PM on November 5, 2014 [8 favorites]
So I'm torn. The video has triggered some great discussions (at least, here), but my rage-o-meter has shifted from an initial harassment is horrible and must be stopped and this is the perfect video to demonstrate why (though I still believe the first part) to why is Hollaback playing into fears of black men in order to get attention?
Research protocol does matter, even if it's for an activist video.
posted by kanewai at 4:30 PM on November 5, 2014 [8 favorites]
I would object that the Hollaback video is not an academic or scientific research project
I think this not so much for the Hollaback people as for every pundit beginning an essay, "As the Hollaback video shows..."
posted by straight at 4:38 PM on November 5, 2014 [2 favorites]
I think this not so much for the Hollaback people as for every pundit beginning an essay, "As the Hollaback video shows..."
posted by straight at 4:38 PM on November 5, 2014 [2 favorites]
They didn't have to do it the right way, but if they had, it would have been an epic work. Wouldn't it have been better if it were bulletproof and not open to valid criticism?
posted by bleep at 5:07 PM on November 5, 2014 [4 favorites]
posted by bleep at 5:07 PM on November 5, 2014 [4 favorites]
why is Hollaback playing into fears of black men in order to get attention?
I've felt progressively more queasy about Hollaback as they've grown from a grass-rootsy set of websites into an NGO that seems more concerned with visibility/controversy. I liked the original idea of giving people who've been harassed a safe place to respond, connect about their experiences, and strategize/organize. But the organization now seems uninterested in discussing harassment with any nuance (for ex., the varied cultural frameworks that normalize and perpetuate harassment). This video really crystalizes what I've felt uneasy about--it's like they're so eager to "raise awareness" that they were willing to demonize men of color. Like that's an acceptable trade.
posted by helpthebear at 5:10 PM on November 5, 2014 [4 favorites]
I've felt progressively more queasy about Hollaback as they've grown from a grass-rootsy set of websites into an NGO that seems more concerned with visibility/controversy. I liked the original idea of giving people who've been harassed a safe place to respond, connect about their experiences, and strategize/organize. But the organization now seems uninterested in discussing harassment with any nuance (for ex., the varied cultural frameworks that normalize and perpetuate harassment). This video really crystalizes what I've felt uneasy about--it's like they're so eager to "raise awareness" that they were willing to demonize men of color. Like that's an acceptable trade.
posted by helpthebear at 5:10 PM on November 5, 2014 [4 favorites]
Research study or not, the immediate thought should be "Show me the data."
posted by CrowGoat at 5:37 PM on November 5, 2014 [2 favorites]
posted by CrowGoat at 5:37 PM on November 5, 2014 [2 favorites]
I am unsure as to whether or not this shit is bananas.
Sigh. My first thought before clicking the link was that it was referring to the 2003/4/5? Gwen Stefani song/music video and now it's in my head B-A-N-A-N-A-S and I will never sleep.
posted by discopolo at 9:43 PM on November 5, 2014 [2 favorites]
Sigh. My first thought before clicking the link was that it was referring to the 2003/4/5? Gwen Stefani song/music video and now it's in my head B-A-N-A-N-A-S and I will never sleep.
posted by discopolo at 9:43 PM on November 5, 2014 [2 favorites]
I get the larger point the author is trying to make in this passage (when presented with a large confounding variable a good researcher redoes the study to minimize its effect):
posted by Ham Snadwich at 6:05 AM on November 6, 2014
The only neutral explanation is that there is a lot of construction, ambulances and sirens going on in more white parts of New York, and somehow they just cannot catch a catcalling white guy.But that seems far from the only neutral explanation.
posted by Ham Snadwich at 6:05 AM on November 6, 2014
I would object that the Hollaback video is not an academic or scientific research project. It's meant as an activist campaign
But the way minorities were overrepresented in the video, and the criticism they received for it, distracted from their message. Assuming they were acting in good faith and didn't intend to overrepresent minorities, being more rigorous would have insulated themselves from that criticism and avoided the resulting distraction from the message they want to send.
If memory serves me correctly, they claimed to have hundreds of incidents of catcalling from all over the city that they edited down to just a couple of minutes. Imagine how much more effective the message would have been if they were able to say "We had so much harassment on tape, we were able to present it in exactly the same racial percentage that New York actually has."
posted by Gelatin at 6:17 AM on November 6, 2014 [3 favorites]
But the way minorities were overrepresented in the video, and the criticism they received for it, distracted from their message. Assuming they were acting in good faith and didn't intend to overrepresent minorities, being more rigorous would have insulated themselves from that criticism and avoided the resulting distraction from the message they want to send.
If memory serves me correctly, they claimed to have hundreds of incidents of catcalling from all over the city that they edited down to just a couple of minutes. Imagine how much more effective the message would have been if they were able to say "We had so much harassment on tape, we were able to present it in exactly the same racial percentage that New York actually has."
posted by Gelatin at 6:17 AM on November 6, 2014 [3 favorites]
(Psst, schmegegge: "He" is a she. Completely agreed, otherwise.)
posted by seyirci at 10:16 AM on November 6, 2014 [1 favorite]
posted by seyirci at 10:16 AM on November 6, 2014 [1 favorite]
May be worth noting that Hollaback didn't make or commission this video. Rob Bliss, the director, filmed and edited it, and then approached Hollaback, who agreed to attach their name to it.
I've felt progressively more queasy about Hollaback as they've grown from a grass-rootsy set of websites into an NGO that seems more concerned with visibility/controversy. I liked the original idea of giving people who've been harassed a safe place to respond, connect about their experiences, and strategize/organize. But the organization now seems uninterested in discussing harassment with any nuance (for ex., the varied cultural frameworks that normalize and perpetuate harassment). This video really crystalizes what I've felt uneasy about--it's like they're so eager to "raise awareness" that they were willing to demonize men of color. Like that's an acceptable trade.
Unfortunately their statements haven't gone as viral as the video, but they may help with your concerns. They were e-mailed to all subscribers and are at the top of the front page on their site.
Statement 1, 10/30
Statement 2, 11/4
posted by booksandlibretti at 10:39 AM on November 6, 2014 [1 favorite]
I've felt progressively more queasy about Hollaback as they've grown from a grass-rootsy set of websites into an NGO that seems more concerned with visibility/controversy. I liked the original idea of giving people who've been harassed a safe place to respond, connect about their experiences, and strategize/organize. But the organization now seems uninterested in discussing harassment with any nuance (for ex., the varied cultural frameworks that normalize and perpetuate harassment). This video really crystalizes what I've felt uneasy about--it's like they're so eager to "raise awareness" that they were willing to demonize men of color. Like that's an acceptable trade.
Unfortunately their statements haven't gone as viral as the video, but they may help with your concerns. They were e-mailed to all subscribers and are at the top of the front page on their site.
Statement 1, 10/30
Statement 2, 11/4
posted by booksandlibretti at 10:39 AM on November 6, 2014 [1 favorite]
Assuming they were acting in good faith and didn't intend to overrepresent minorities, being more rigorous
It would be interesting for them to _actually_ be rigorous rather than _pretend_ to be rigorous by trying to balance the video.
For example, what were the actual counts per comment? How did they break down, not just by race but by location?
It would actually be kind of interesting to see actual collected data and not be so up in arms about the perceived racism of a video.
posted by rr at 11:20 AM on November 6, 2014 [1 favorite]
It would be interesting for them to _actually_ be rigorous rather than _pretend_ to be rigorous by trying to balance the video.
For example, what were the actual counts per comment? How did they break down, not just by race but by location?
It would actually be kind of interesting to see actual collected data and not be so up in arms about the perceived racism of a video.
posted by rr at 11:20 AM on November 6, 2014 [1 favorite]
"I am unsure as to whether or not this shit is bananas."
In other news, studies say eating bananas prevents cancer!
posted by sudon't at 12:08 PM on November 6, 2014
In other news, studies say eating bananas prevents cancer!
posted by sudon't at 12:08 PM on November 6, 2014
« Older NO ELEPHANTS, NO CASTLES, ALL LIES | "...no one talked about the real issue, the... Newer »
This thread has been archived and is closed to new comments
posted by blahblahblah at 2:58 PM on November 5, 2014 [5 favorites]