“You Can Hear The Suffering”

November 29, 2019 7:52 AM

“Nearly every Revver who spoke with The Verge said they were exposed to graphic or troubling material on multiple occasions with no warning. This includes recordings of physical and verbal abuse between intimate partners, graphic descriptions of sexual assault, amateur porn, violent footage from police body cameras, a transphobic rant, and, in one instance, “a breast augmentation filmed by a physician’s cell phone, being performed on a patient who was under sedation.” Transcribers for the gig economy service Rev hate the recently slashed rates, but the disturbing content they deal with is even worse.
posted by The Whelk (31 comments total)

1. This is precisely the kind of stuff that AI should be doing.
2. It is horrifying that this is the work available to people who are already in precarious, stressful and unpredictable situations.
posted by grumpybear69 at 8:07 AM on November 29, 2019 [5 favorites]

Given that the best-case speed for human transcription is 4:1, or 15 minutes of audio per working hour, even the former rate of 45 cents per minute was well below federal minimum wage. The new rate of 30 cents per minute is just USD 4.50 per hour -- again, under best-case conditions.

It is illuminating that AB5 has successfully driven Rev out of California (per the Verge piece). This definitely highlights the need for a federally-imposed ABC test for independent contractor status.
posted by Not A Thing at 8:26 AM on November 29, 2019 [23 favorites]

1. This is precisely the kind of stuff that AI should be doing.

AI is horrendously bad at this, and I've spent hundreds of hours correcting bad captioning.

It's upsetting to hear Rev is so shitty about these things. We use them at work basically because they're fast and much better quality than similar services.
posted by Fish Sauce at 8:32 AM on November 29, 2019 [6 favorites]

Given that the best-case speed for human transcription is 4:1, or 15 minutes of audio per working hour, even the former rate of 45 cents per minute was well below federal minimum wage.

I know a lot of Revvers transcribe much faster than that (myself included). I worked mostly on their captioning side and would be able to transcribe and sync all the captions at about a 3:1, or 20 minutes of audio per working hour. Syncing for me involved going back from the top of the video and telling the software interface to put the captions where I wanted.

Even at that speed, the pay was pitiful.
posted by astapasta24 at 8:46 AM on November 29, 2019 [7 favorites]

I was in an extended discussion in a F’book group for local women business owners about MLMs and whether folks involved in them should be allowed to promote their “makeup parties” to the group. And I posted a couple articles that listed other “side gigs” and work-from-home things that could be explored. One of them was transcription services. And I’ve seen AI-generated transcription and it’s terrible. I’m vehemently anti-MLM, but companies like Rev are part of why the scam of MLMs continues to proliferate.
posted by amanda at 8:46 AM on November 29, 2019 [5 favorites]

I didn't last through the training at Rev because the pay was unlivable before the rate cuts. I do the same thing but paid well (well, relatively, and they cut pay recently, too) at a different company and sometimes run into and transcribe content like this. I think, but can't find it now, that contractors asked the company about providing trigger warnings on files and were told that it was just logistically unfeasible, which I get, although it'd be nice if clients provided them, even inconsistently. But it's wild that it's against Rev's TOS but there's nothing revvers can do about it except return it midway through and go unpaid for what they completed, is how I've heard it works.

On a related forum a revver said it's weird/annoying that the media focus quickly left the horrible pay issue to focus on the graphic content. A little bit that's my reaction, too, because I've seen tons of rev complaints for years but not about the content, and it just seems to me like it's the handling that's bad. But that also might be me used to being one of these precarious people and being grateful for work if it's not the actual worst.
posted by gaybobbie at 8:51 AM on November 29, 2019 [6 favorites]

Having also worked with cleaning up AI transcripts, I've noticed that they are absolutely terrible if there are any of the following. If there's a combination of any of these, the AI transcripts can become absolutely non-sensical.

1) women speaking
2) any non-American accents
3) multiple speakers, and they talk over each other
4) too much background noise, like wind/coughing/ambient shifting in seats
5) background music that's too loud (sometimes AI will pick up lyrics from music and intersperse it with what the speaker's talking about)
6) poorly miked speakers
7) technical jargon of any sort (programming, medical, scientific, art)
8) proper names
--
AI transcripts also struggle with when sentences start/stop and punctuation.
posted by astapasta24 at 8:59 AM on November 29, 2019 [4 favorites]

Why spend the money for AI when there's always a steady stream of desperate meatsacks with CPUs to do it cheaper?

If you could solve this problem cheaply/reliably/quickly, someone would write you a check with 8 or 9 zeros on it. Hell, they might just leave it blank and let you fill out whatever you want.
posted by sideshow at 9:03 AM on November 29, 2019

In one of those timely coincidences, I just got off the phone with the small transcription company that we use for health care research. They charge almost twice as much as other companies (and 3 times that of Rev)- and it is worth every penny. Their quality is excellent, they are willing to follow our strict privacy rules - and they even give us advice on how to make better recordings (which, of course, improves the quality of the transcript again). We don't usually have disturbing material (not in our subfield of research) - but we would warn them if we did.

It feels like a world away from the model of transcription in these articles. Their transcriptionists also work from home and have flexibility - but are in a really different situation.

And I can't help but think that we - as customers, as a society - are destroying everything in a bid to reduce costs. Some of my colleagues are beginning to use gig-transcribers for non-private description (meetings, etc.), but that's just feeding the monster that is the gig economy. Every time we buy from a gig-economy supported service, we are part of the problem.

It should be illegal to use this kind of gig-work for any transcription involving the law, law enforcement, health, etc. I can't even imagine how they are abiding by privacy regulations (seriously, in a lot of cases, you can't just pop audio files online and have people transcribe them without confidentiality agreements - certainly in research, but I would have thought also in law or any healthcare situation, and that companies would also want privacy.)
posted by jb at 9:39 AM on November 29, 2019 [24 favorites]

Rev said that “many skilled transcriptionists on Rev earn $15+ per hour.

This is an utterly ridiculous wage. An independent transcriptionist I've worked with charged $40/hour (about $80/audio hour - which is a completely reasonable speed and rate) - and this was several years ago.
posted by jb at 9:42 AM on November 29, 2019 [20 favorites]

Isn't this just straight-up sexual harassment? How is Rev not getting sued for sending sexually explicit material to their contractors without a warning?
posted by explosion at 10:40 AM on November 29, 2019

You probably can't get to a decent AI/ML transcription engine anytime soon but you could certainly classify the content of the video and apply appropriate metadata flags if it is sexual, graphic, etc. in nature. This has been feasible for a while but with the proliferation of systems like tensorflow it's hard to make a case against doing it. Google literally has an API you could submit a sample of still frames to and get a detailed response about the contents.
posted by feloniousmonk at 11:09 AM on November 29, 2019 [1 favorite]

Yeah. I did Rev for a few months before they suspended my account randomly despite having above average ratings as was strangely common at the time. I definitely saw police cam footage, heard violent domestic fights, and at one point a conversation between a customer and an insurance adjuster that contained the client’s social security number, address, date of birth, etc. I was kinda perplexed by the lack of professionalism. But then I got to transcribe really cool and bizarre things, like entire episodes of Chopped, interviews with top tier game developers, lectures from astrophysicists and, my favorite, what seemed to be thousands of recordings from a robot McDonald’s drive through. It was wild but I fortunately never experienced anything too disturbing.
posted by Young Kullervo at 11:42 AM on November 29, 2019 [14 favorites]

Also I sort of gamed the system by using my phone/Dragon to transcribe the audio and then correcting its errors.
posted by Young Kullervo at 11:45 AM on November 29, 2019 [12 favorites]

> and at one point a conversation between a customer and an insurance adjuster that contained the client’s social security number, address, date of birth, etc.

so this is where i suddenly realize that there is potentially a ton of money in working for gig economy transcription services.
posted by Reclusive Novelist Thomas Pynchon at 12:23 PM on November 29, 2019 [24 favorites]

So potentially and phone conversation with a bank / insurance company / medical professional will be listened to and transcribed by a rando who may or may not keep a copy of that information. Amazeballs.
posted by grumpybear69 at 12:44 PM on November 29, 2019 [3 favorites]

So that's the "quality assurance" reason for recording calls that I'm always hearing so much about.
posted by bleep at 1:29 PM on November 29, 2019 [4 favorites]

> So potentially and phone conversation with a bank / insurance company / medical professional will be listened to and transcribed by a rando who may or may not keep a copy of that information. Amazeballs.

god though i want to be that rando

this is how we fund the revolution, comrades. gig-economy transcription services are the new poorly-guarded georgian banks. a+ praxis here
posted by Reclusive Novelist Thomas Pynchon at 1:30 PM on November 29, 2019 [6 favorites]

I have had to resort to Rev work in dire circumstances, and their captioning work is currently the only means of income for my friend, who lives with me and I pay most bills for. the other day I was laying in my bed, and he was working at my desk. he was doing this pretty funny, relatively easy video of a guy playfully narrating some Minecraft shenanigans. This is the kind of project that's perfect - there's only one speaker, a native English speaker, and professionally recorded audio. Even still, he spoke so fast (as youtubers typically do), that it took quite a but of time to get it all written out. My friend isn't the fastest, but even if he was, it would have easily been below minimum wage.
Directly after that, it was a cell phone video in the middle of a brutal act of police violence. He grew up to drug-dependent parents, was orphaned, and homeless in Detroit. This video could have easily triggered any number of diagnosed mental illnesses he has. Thankfully, it did not. But there's no way to know what the video is until you hit play.
posted by FirstMateKate at 2:28 PM on November 29, 2019 [9 favorites]

My mom did transcription of FBI wiretaps while she was pregnant with me in the 1970s. They were apparently mildly surprised that she typed out the swear words rather than typing, e.g., f***.
posted by heatherlogan at 3:53 PM on November 29, 2019 [4 favorites]

Yeah. I thought to myself “Wow I could steal this person’s identity and it would never be traced back to this transcript job.” That is particularly true if you don’t actually finish or get paid for it because it just goes back into the queue (although they do keep a record of anyone who accepted the job and the reason they couldn’t finish it but that could be dozens of other people). They make you agree to confidentially but I mean if I was of a criminal mind why would I care? They don’t require a background check so they are trusting randos off the Internet streets with very sensitive data. It blows my mind how this is legal.
posted by Young Kullervo at 4:07 PM on November 29, 2019 [5 favorites]

Also I sort of gamed the system by using my phone/Dragon to transcribe the audio and then correcting its errors.

Surely the most efficient way to do transcription is to talk into a speech recognition program that is trained on your voice, repeat what you hear in real time like a simultaneous translator, only pausing to correct mistakes?

Are there people who can type faster than they can talk?
posted by straight at 5:52 PM on November 29, 2019 [3 favorites]

Surely the most efficient way to do transcription is to talk into a speech recognition program that is trained on your voice, repeat what you hear in real time like a simultaneous translator, only pausing to correct mistakes?

straight, I worked for a bit in phone call captioning for people who are hard of hearing and that's exactly what we did in real-time!
posted by augustimagination at 6:11 PM on November 29, 2019 [1 favorite]

Oh damn that’s even better. But Dragon worked fine, and I could step away and do something while it was transcribing.
posted by Young Kullervo at 6:34 PM on November 29, 2019

Straight, there are some Revvers who do exactly, mainly due to physical limitation on high-speed typing, from what I gather. But you still have to go back, proofread, grammar-check, format correctly, etc. So for many people, yeah, typing is faster.
posted by quinndexter at 7:14 PM on November 29, 2019

Are there people who can type faster than they can talk?

I used to type 125 words per minute on an IBM Selectric. I’m probably faster than that on a computer keyboard, though I’ve never taken a test.

I transcribed interviews for documentaries and telecourses in the 1980s-90s and mostly enjoyed it. I never heard anything upsetting. Rev sounds absolutely awful.
posted by elphaba at 7:19 PM on November 29, 2019

I did some work with Rev when I was desperate. I occasionally ran into disturbing content but that wasn't common, and definitely wasn't why I quit working for them. There are two main problems with Rev: the low pay rates to begin with, and the fact that it's really hard to find "quality" content.

Rev doesn't assign things to you, you pick them yourself out of the list on their website. You can preview and nope out if anything sounds offensive or triggering, and quite often they're already flagged.

My main problem was that 95% of their material was recorded by a 12-year-old cell phone held 30 feet away from a group of 20 people who are all talking at once.

My only real success was working during the Australian day and grabbing TV show episodes to transcribe since I'm good at understanding non-US accents.
posted by mmoncur at 1:58 AM on November 30, 2019 [4 favorites]

Isn't this just straight-up sexual harassment? How is Rev not getting sued for sending sexually explicit material to their contractors without a warning?

Contractors have no legal protections against sexual harassment (or discrimination, or almost anything).
posted by cushie at 10:46 AM on November 30, 2019

Contractors have no legal protections against sexual harassment (or discrimination, or almost anything).

This is not always necessarily true -- certain states do extend sexual harassment protection to independent contractors. And there are a few applicable federal protections, e.g. 42 USC 1981 which protects against racial discrimination in contracting. Of course, even if any of those were applicable here, it would be difficult to get around the abusive forced-arbitration clause Rev has shielded itself with.

If somebody could get into court, though, I'm not sure Rev's "independent contractor" claim would hold water even under the FLSA's employer-friendly "economic realities" test. Clearly Rev itself recognizes that it couldn't pass the increasingly common and much more straightforward ABC test.
posted by Not A Thing at 12:35 PM on November 30, 2019 [1 favorite]

I have friends who do live captioning for universities and public events (remote work, done using a microphone and special audio transcription software they’ve been trained to operate). Those jobs pay extremely well- sounds like an order of magnitude better than Rev. So it looks like if you “crowdsource” to untrained temp labor you can cut wages by 90-95%. It’s a brave new world.
posted by q*ben at 2:41 PM on November 30, 2019 [2 favorites]

Me too, augustimagination. That's where I got the idea.
posted by straight at 4:50 PM on November 30, 2019