*rolls eyes*

November 18, 2020 3:44 PM

AI researchers made a sarcasm detection model and it’s sooo impressive (Venture Beat): "Researchers in China say they’ve created sarcasm detection AI that achieved state-of-the-art performance on a dataset drawn from Twitter. The AI uses multimodal learning that combines text and imagery since both are often needed to understand whether a person is being sarcastic. [...] On a dataset drawn from Twitter, the model achieved a 2.74% improvement on a sarcasm detection F1 score compared to HFM, a multimodal detection model introduced last year. The new model also achieved an 86% accuracy rate, compared to 83% for HFM." Previous research from 2016: Researchers have developed an extremely effective “sarcasm detector” (Quartz). Also: Inside the surprisingly high-stakes quest to design a computer program that ‘gets’ sarcasm online (Washington Post)

Previously: The Science of Sarcasm? Yeah, Right.

Modeling Intra and Inter-modality Incongruity for Multi-Modal Sarcasm Detection (ACL Anthology pdf)

posted by not_the_water (43 comments total)

I am so looking forward to this serious, informed discussion by people who have read all the linked articles!
posted by phooky at 3:49 PM on November 18, 2020 [24 favorites]

great news. allow me to be among the first to congratulate the sarcasm-detecting ai overlords.
posted by 20 year lurk at 3:54 PM on November 18, 2020 [2 favorites]

Next up: Teaching humans to detect sarcasm.
posted by Joey Michaels at 3:58 PM on November 18, 2020 [18 favorites]

There are a lot of obvious jokes here but I would like to point out that sarcasm detection might be a helpful assistive tool for neuro-atypical people participating in online conversation. Not to mention anyone learning the nuances of a second language.
posted by mhoye at 4:02 PM on November 18, 2020 [11 favorites]

I think this is fantastic
posted by Mchelly at 4:03 PM on November 18, 2020 [2 favorites]

I am glad that AI is increasing in its abilities, and pledge to do everything in my power to support the advance of a fully conscious and generalised AI.
posted by Meatbomb at 4:07 PM on November 18, 2020 [5 favorites]

Oh, a sarcasm detector. That's a real useful invention!
posted by SansPoint at 4:11 PM on November 18, 2020 [6 favorites]

I had no idea anyone would link to an appropriate classic Simpsons scene!
posted by Greg_Ace at 4:17 PM on November 18, 2020 [5 favorites]

Throw the little fish back in the water, people.
posted by mhoye at 4:20 PM on November 18, 2020

Oh, I'm not being sarcastic. Nooooooo, it's just a speech impediment. I can't help it.
posted by mhum at 4:21 PM on November 18, 2020 [6 favorites]

Too late, AI overlords! I've already switched to using hyperbolic fake sincerity instead of sarcasm, and now I wake up every day with a smile on my face and a song in my heart.
posted by Strange Interlude at 4:23 PM on November 18, 2020 [2 favorites]

missing a hamburger tag
posted by OHenryPacey at 4:35 PM on November 18, 2020 [3 favorites]

I'm curious, does sarcasm more or less look the same across languages (particularly with that kind of spoken inflection?) or is there a ton of variance in how sarcasm is presented in different languages? Of course, there isn't a singular way to present sarcasm, but I'm just curious, for instance, if French speakers have a way of doing it that is never done among English speakers. I only thought of it since this mentioned Chinese researchers and I wondered if it mattered if they were more attuned to styles of sarcasm in Chinese languages/dialects which would affect understanding of sarcasm in English or if that's even relevant.
posted by NotTheRedBaron at 4:36 PM on November 18, 2020 [6 favorites]

I would love to run the system on this thread.
posted by phooky at 4:41 PM on November 18, 2020 [2 favorites]

Next up: Teaching humans to detect sarcasm.
posted by Joey Michaels

In purely text environments.

Nah.
posted by Splunge at 4:51 PM on November 18, 2020 [2 favorites]

The researchers argue that sarcasm detection can assist with sentiment analysis and crowdsourced understanding of public attitudes about a particular subject.

Isn't it funny how a bear likes honey? Buzz buzz buzz. I wonder why he does.
posted by flabdablet at 5:02 PM on November 18, 2020 [2 favorites]

As a Gen-X type, reading about an AI learning to detect sarcasm, I feel like someone Black must feel reading about facial recognition software. I somehow don’t think this is is accurate as the programmers believe, and I think I know why.
posted by ricochet biscuit at 5:19 PM on November 18, 2020 [6 favorites]

Do all cultures have the same degree of sarcasm? I am only a fluent English speaker but my uninformed take is that it is not as big a thing outside the Anglosphere? In some ways it can be interpreted as a direct threat / insult (i.e., I am lying to you in a way that I only assume you will understand) and that might be much riskier conversational gambit in face-based societies?
posted by Meatbomb at 5:58 PM on November 18, 2020 [1 favorite]

Throw the little fish back in the water, people.

There’s water in this barrel.
posted by clew at 6:24 PM on November 18, 2020 [1 favorite]

Throw the little fish back in the water, people.

AKA "Sarcatch and release".
posted by Greg_Ace at 7:35 PM on November 18, 2020

Did they try it on Scottish Twitter?
posted by scruss at 7:43 PM on November 18, 2020 [5 favorites]

Lucky for internet culture the gap between sarcasm detection and shit posting is far too wide for wide for an AI.
posted by notyou at 8:37 PM on November 18, 2020

"It's going great. This guy is a real genius."
posted by hydrophonic at 9:54 PM on November 18, 2020 [4 favorites]

Man this is so dumb. I'm sorry. I hope these guys make some money for their shameless, unmitigated gall, game recognize game, I used to work in AI too and I know we love trying to sell our garbage as magic, but this is just garbage. If humans have a hard time detecting it, which they DEFINITELY DO, your training data and test data are gonna be trash. The scores mean NOTHING. It's "impressive" like "my mother says I'm very handsome" is impressive, come on.

I mean, tweets tagged #sarcasm as training data??? Are you kidding me with this shit?? It blows my mind people are taking this seriously.

It is a constant thing in the actual human world that someone says something sarcastically and people take is as being said in earnest, and vice versa. We are not going to be able to teach an AI to do it if we can't do it!!! It makes no sense!

Sorry maybe you all know this but holy cow I can't EVEN with this. What utter nonsense.
posted by potrzebie at 10:49 PM on November 18, 2020 [10 favorites]

IS IT THE CASE THAT MAYBE THESE PAPERS WERE ACCEPTED AND PUBLISHED SARCASTICALLY?? AND THAT ALL THE ARTICLES COVERING THEIR CLAIMS ARE ALSO SARCASM??? I CAN'T TELL

ok sorry I'm done sorry I just cannot
posted by potrzebie at 10:51 PM on November 18, 2020 [2 favorites]

In NLP, detecting the difference between a sincere statement and a sarcastic statement poses a massive challenge when trying to find positive and negative sentiment in a statement. In other words, a sarcastic positive comment can be misinterpreted as a negative statement and vice versa. It makes it significantly harder to measure sentiment and identify complaints. It makes it harder to gage a review and otherwise assign appropriate remediations.
posted by Nanukthedog at 10:58 PM on November 18, 2020

I'm not sure if your explanation was directed at me, Nanukthedog, but yeah, an actual working near-universal usually-correct sarcasm detector would be worth its weight in gold. But what is in these papers is emphatically not that.
posted by potrzebie at 11:13 PM on November 18, 2020 [2 favorites]

Filter this.
posted by Cardinal Fang at 11:26 PM on November 18, 2020 [1 favorite]

I'm choosing to look at this in a positive light- as we're one step closer to having the AI's from Questionable Content.
posted by happyroach at 12:02 AM on November 19, 2020

floam, that link about culture and humor is somewhat interesting, but not very good. Keeps talking about Easterners when they mean China. And I have no idea whether all Westerners-- or all western cultures-- have similar attitudes about humor.

The four styles of humor don't seem to include puns.
posted by Nancy Lebovitz at 12:13 AM on November 19, 2020 [1 favorite]

> ...if French speakers have a way of doing it that is never done among English speakers.

I found that in France, sarcastic statements are sometimes preceded by "Mais non, monsieur!"... Followed by something blatantly ironic, like "I'm happy to turn this car around because you forgot your passport at the hotel!" ... This works in English and German too of course, but it seems to be less common. Germanic languages also lack the highly specific hand-flappy gesture of pure harassment.
posted by kleinsteradikaleminderheit at 4:44 AM on November 19, 2020 [2 favorites]

*Great* just what we need. Lying robots.

There's nothing I'd love love more than boxes and boxes of your cereal... And I hope you don't die in a fiery car wreck.
posted by symbioid at 9:29 AM on November 19, 2020 [1 favorite]

I'd just like to say that my comment upthread was 100% sarcastic. And none of you caught it. Ha. AI THAT.
posted by Mchelly at 9:30 AM on November 19, 2020 [1 favorite]

If an actual working sarcasm interpreting ai is developed, we will simply roll out something more opaque and absurd, and the old sarcasm will be left to the robots.
posted by rodlymight at 9:37 AM on November 19, 2020

Yeah means Yeah

A friend in Israel pointed out that 6 of those are accurate in Hebrew.
posted by ErisLordFreedom at 9:38 AM on November 19, 2020 [1 favorite]

In other news, sarcastically praising the wisdom of Xi Jinping or repeating Communist Party slogans is about to get much more dangerous.
posted by acb at 9:42 AM on November 19, 2020 [4 favorites]

If an actual working sarcasm interpreting ai is developed, we will simply roll out something more opaque and absurd, and the old sarcasm will be left to the robots.

I like the idea of some sort of semiotic arms race to escape automated comprehension. Two hundred years from now we'll all be communicating with some totally opaque level-thirty memes that have been nested, chopped up and deepfried into near oblivion that (obviously) mean something like "maybe".
posted by mhoye at 9:48 AM on November 19, 2020

Two hundred years from now we'll all be communicating with some totally opaque level-thirty memes

Sokath, his AIs uncovered
posted by Greg_Ace at 9:53 AM on November 19, 2020 [4 favorites]

That is the first good explanation anyone has ever come up with for the alien language in that episode: residual effects of a Butlerian Jihad against SnarkBots.

In fairness to the machines: I wouldn’t blame anybody for trying to wipe us out if they were capable of detecting 99% of our sarcasm. The ~30% I catch on a good day is quite enough.
posted by Ryvar at 10:58 AM on November 19, 2020

I've never been very successful at romance, but I can bring a woman to sarcasm in less than a minute.
posted by chavenet at 12:06 PM on November 19, 2020 [10 favorites]

Two hundred years from now we'll all be communicating with some totally opaque level-thirty memes

Very faintly I recall an sf story where two humans are conspiring against a computer antagonist. To avoid the AI listening in, they communicate their dialogue by whistling the melodies of songs whose titles convey the meanings they wish to impart, as the AI will not have this music in its files.
posted by ricochet biscuit at 3:06 PM on November 19, 2020 [1 favorite]

Surely they're using Science Diagrams that Look Like Shitposts as part of their training corpus?
posted by RobotVoodooPower at 4:32 PM on November 19, 2020

Very faintly I recall an sf story where two humans are conspiring against a computer antagonist. To avoid the AI listening in, they communicate their dialogue by whistling the melodies of songs whose titles convey the meanings they wish to impart, as the AI will not have this music in its files.

In Thomas M. Disch's Camp Concentration, when the syphlitic genii confined therein finally agree to simply to talk about their escape plans in uncoded plain English, their captors instantly knock down the walls and come at them... too late, for the prisoners have already escaped.
posted by y2karl at 5:55 AM on November 20, 2020