“Sometimes, you need a little help from a higher algorithm.”
June 13, 2023 9:44 AM Subscribe
‘Mrs. Davis’ Versus AI: Here’s What Happened When ChatGPT Interviewed Damon Lindelof, Tara Hernandez and Betty Gilpin
Mrs. Davis is a TV show about nuns, magic, free will, the Catholic Church, the nature of God and an artificial intelligence systems that runs the world. The show has has been discussed in FanFare.
In the main link, Variety reporter Michael Schneider decided to see if ChatGPT could conduct an interview with the show's star and its creators. The results were mixed at best. The article includes commentary from all parties about what ChatGPT does well and does poorly.
Spoiler warning: If you don't want to find out too much about how the show wraps up (at least that's my interpretation, and I'm only halfway through the episodes) stop reading once you hit the paragraph What ChatGPT also failed to do was ask any pertinent questions about Gilpin (or any of the show’s other actors, for that matter). When I asked Gilpin about preparing for “Mrs. Davis,” she said two things were important: “I wanted to honor nuns and I wanted to honor magicians.” She pauses. “I have never said that out loud.”
Mrs. Davis is a TV show about nuns, magic, free will, the Catholic Church, the nature of God and an artificial intelligence systems that runs the world. The show has has been discussed in FanFare.
In the main link, Variety reporter Michael Schneider decided to see if ChatGPT could conduct an interview with the show's star and its creators. The results were mixed at best. The article includes commentary from all parties about what ChatGPT does well and does poorly.
Spoiler warning: If you don't want to find out too much about how the show wraps up (at least that's my interpretation, and I'm only halfway through the episodes) stop reading once you hit the paragraph What ChatGPT also failed to do was ask any pertinent questions about Gilpin (or any of the show’s other actors, for that matter). When I asked Gilpin about preparing for “Mrs. Davis,” she said two things were important: “I wanted to honor nuns and I wanted to honor magicians.” She pauses. “I have never said that out loud.”
I've got to say, I strongly disagree with you The Pluto Gangsta.
I've read all (or at least most) of the LLM threads. A lot of them have been really redundant--at least from my point of view--constantly tripping over "we can't call this AI" even though that's what broader society has settled upon as the way to refer to this technology. To me, that's neither here nor there. That's boring to me.
This is actually an exercise that is of interest to me in that the technology has been applied to one very specific job, interviewing and writing, and the results are being discussed. This isn't a theoretical discussion. This is a real-world application that affects people who make their living as writers and reporters and marketers and journalists--those same classes of people who keep getting told they're going to be replaced by the machine. And yes, they're those same people whose jobs are already at risk due to a wide variety of factors.
A lot of what I've seen posted is "this is what AI did when I told it to write stuff" but I haven't seen very much of the other side of the equation: get AI to ask the questions. That's why I posted this and that's the angle that is of the most interest to me.
Look, I don't believe AI or LLMs or whatever we decide to call it on MeFi can or will replace journalists. I do believe, however, that media organizations will certainly do their best to see if it can. Anything that improves the bottom line, and all that.
The piece is clearly labelled as AI, so that way anybody who wants to avoid the AI-generated output can do so. The "more inside" description says what the show is about because I personally hate mystery meat posts and I also don't assume that everybody lives in the US and is totally up-to-date with every piece of pop culture that country produces, because I certainly don't and am not.
If this isn't a conversation you're interested in having, no skin off my back, but there's also no need to try to shut it down from the start, no disrespect intended, of course.
posted by sardonyx at 10:26 AM on June 13, 2023 [11 favorites]
I've read all (or at least most) of the LLM threads. A lot of them have been really redundant--at least from my point of view--constantly tripping over "we can't call this AI" even though that's what broader society has settled upon as the way to refer to this technology. To me, that's neither here nor there. That's boring to me.
This is actually an exercise that is of interest to me in that the technology has been applied to one very specific job, interviewing and writing, and the results are being discussed. This isn't a theoretical discussion. This is a real-world application that affects people who make their living as writers and reporters and marketers and journalists--those same classes of people who keep getting told they're going to be replaced by the machine. And yes, they're those same people whose jobs are already at risk due to a wide variety of factors.
A lot of what I've seen posted is "this is what AI did when I told it to write stuff" but I haven't seen very much of the other side of the equation: get AI to ask the questions. That's why I posted this and that's the angle that is of the most interest to me.
Look, I don't believe AI or LLMs or whatever we decide to call it on MeFi can or will replace journalists. I do believe, however, that media organizations will certainly do their best to see if it can. Anything that improves the bottom line, and all that.
The piece is clearly labelled as AI, so that way anybody who wants to avoid the AI-generated output can do so. The "more inside" description says what the show is about because I personally hate mystery meat posts and I also don't assume that everybody lives in the US and is totally up-to-date with every piece of pop culture that country produces, because I certainly don't and am not.
If this isn't a conversation you're interested in having, no skin off my back, but there's also no need to try to shut it down from the start, no disrespect intended, of course.
posted by sardonyx at 10:26 AM on June 13, 2023 [11 favorites]
My understanding was that AI generated comments were especially frowned upon, some people wanted clearer indications of when a link might expose them to AI interactions, but that posts concerning AI remained worthy of consideration.
posted by otsebyatina at 10:26 AM on June 13, 2023 [2 favorites]
posted by otsebyatina at 10:26 AM on June 13, 2023 [2 favorites]
Is there a link for that discussion? Without weighing in on this particular article, which I haven't finished, it seems that a blanket ban on content that contains anything generated by LLMs is too far.
posted by justkevin at 10:33 AM on June 13, 2023 [1 favorite]
posted by justkevin at 10:33 AM on June 13, 2023 [1 favorite]
No such ban exists; see the conversation here.
Generating Mefi content with AI is the main hard no, linking to things generated by or running on AI without making it explicit is rude, and articles discussing AI in general are fine, is where I wound up after the thread.
posted by sagc at 10:41 AM on June 13, 2023 [5 favorites]
Generating Mefi content with AI is the main hard no, linking to things generated by or running on AI without making it explicit is rude, and articles discussing AI in general are fine, is where I wound up after the thread.
posted by sagc at 10:41 AM on June 13, 2023 [5 favorites]
Then I was definitely mistaken, at least as far as the site's position on articles about AI-generated content. Again, I didn't intend any insult and I apologize.
I do feel like this is the thin end of the wedge, though.
posted by The Pluto Gangsta at 11:04 AM on June 13, 2023 [1 favorite]
I do feel like this is the thin end of the wedge, though.
posted by The Pluto Gangsta at 11:04 AM on June 13, 2023 [1 favorite]
Would it be better if it were a third party writing about another publication using ChatGPT to run an interview? At least here, it's done in a somewhat-experimentally-responsible way.
posted by sagc at 11:12 AM on June 13, 2023 [1 favorite]
posted by sagc at 11:12 AM on June 13, 2023 [1 favorite]
I enjoyed the couple times the interviewees just didn't take the bait. "No I'm not going to waste my time writing a coherent answer to your nonsense question."
Especially the question about whether the portrayal of AI in the show is "ethical" - I appreciated that they didn't bother trying to answer that.
posted by subdee at 12:16 PM on June 13, 2023 [1 favorite]
Especially the question about whether the portrayal of AI in the show is "ethical" - I appreciated that they didn't bother trying to answer that.
posted by subdee at 12:16 PM on June 13, 2023 [1 favorite]
What was especially telling, at least to me, was that with one good question and some personal interaction, the interviewer got an revealing answer from Gilpin (one that surprised even her), which is something that no automated too will ever be able to do. It's so easy for people to think that interviewing is so easy, that anybody can do it, and that of course machines/systems/AI/etc. will be able to replace people with actual skills in interviewing other people.
Humans can follow up. Humans can read cues (verbal, physical, etc.) Humans can be sympathetic or antagonistic or curious (all to varying degrees) and can channel those feelings and use them to coax responses out of their subjects.
I mean, nothing in this article came as a revelation to me (although, like the reporter, I'm kind of impressed at some of the tag lines that were generated), but I do think the exercise was truly a worthwhile one to conduct at this time, if only to have something concrete to point all the bean counters and tech bros to so that we can say "this is why we need people in newsrooms."
I also really like celebrating clever approaches to newsgathering, and I think Schneider did a nice job of picking the subjects for his experiment and setting both them and himself up for success.
posted by sardonyx at 1:09 PM on June 13, 2023
Humans can follow up. Humans can read cues (verbal, physical, etc.) Humans can be sympathetic or antagonistic or curious (all to varying degrees) and can channel those feelings and use them to coax responses out of their subjects.
I mean, nothing in this article came as a revelation to me (although, like the reporter, I'm kind of impressed at some of the tag lines that were generated), but I do think the exercise was truly a worthwhile one to conduct at this time, if only to have something concrete to point all the bean counters and tech bros to so that we can say "this is why we need people in newsrooms."
I also really like celebrating clever approaches to newsgathering, and I think Schneider did a nice job of picking the subjects for his experiment and setting both them and himself up for success.
posted by sardonyx at 1:09 PM on June 13, 2023
While I totally agree on discouragement/ban on robot comments, this is a show explicitly about the issues of AI. The conceit of the creatives of such a show interviewed by a "real" AI is just too cute and certainly appropriate.
Full disclosure: I secretly sorta want Mrs Davis whispering in my ear sometimes.
posted by sammyo at 3:47 PM on June 13, 2023 [3 favorites]
Full disclosure: I secretly sorta want Mrs Davis whispering in my ear sometimes.
posted by sammyo at 3:47 PM on June 13, 2023 [3 favorites]
ask chatgpt for something mundane (a recipe, say, or some interview questions) then ask it to repeat the response with every word replaced by two copies of the word “indeed.” if there are words other than “indeed,” repeat the process. once all the text is repetitions of “indeed,” have it continue generating text until something interesting happens.
posted by bombastic lowercase pronouncements at 4:12 PM on June 13, 2023
posted by bombastic lowercase pronouncements at 4:12 PM on June 13, 2023
There's a recent Planet Money episode which explorers some similar territory here. They were rather more successful, but I think were more iterative/collaborative in their approach.
posted by kaibutsu at 9:32 PM on June 13, 2023 [1 favorite]
posted by kaibutsu at 9:32 PM on June 13, 2023 [1 favorite]
ChatGPT cannot conduct an interview. It has no will, no understanding of anything, no internal model of other people, no beliefs, no desires, no drives. It is autocomplete with a lot of data. Stop anthropomorphizing these tools. It's bad for everyone.
Once you remove this ridiculous conceit of pretending this hammer is alive and can think, there is no story here.
posted by abucci at 12:05 PM on June 14, 2023 [2 favorites]
Once you remove this ridiculous conceit of pretending this hammer is alive and can think, there is no story here.
posted by abucci at 12:05 PM on June 14, 2023 [2 favorites]
I'd love to say I agree with you, abucci, but do you have any idea how a lot of interviews are conducted these days? They're done over email. Sometimes that's at the request of the reporter, but a lot of times, that's the way the corporate source insists the communication flow, so the message can be controlled.
Can I envision a time where some "progressive" newsroom manager (with the CFO barking in his ear) suggests just getting "AI" to generate questions for emailing and then handing those off to some intern/assistant/person-getting-paid-way-too-little? You bet your booties I can. Are we there yet? No, or at least I hope we're not. Is this a good approach? Hell no, but when has that ever stopped business from trying something?
posted by sardonyx at 12:44 PM on June 14, 2023
Can I envision a time where some "progressive" newsroom manager (with the CFO barking in his ear) suggests just getting "AI" to generate questions for emailing and then handing those off to some intern/assistant/person-getting-paid-way-too-little? You bet your booties I can. Are we there yet? No, or at least I hope we're not. Is this a good approach? Hell no, but when has that ever stopped business from trying something?
posted by sardonyx at 12:44 PM on June 14, 2023
sardonyx, the fact that we live in a late-stage capitalist hellscape in which the media has been thoroughly gutted and purchased by billionaires does not imply we should all acquiesce to making it even worse on purpose.
We should resist the downward spiral by, for instance, rejecting that a non-sentient computer program is capable of doing what human beings do. No one but billionaires benefits if we all give up and pretend this is OK.
posted by abucci at 6:30 PM on June 14, 2023 [1 favorite]
We should resist the downward spiral by, for instance, rejecting that a non-sentient computer program is capable of doing what human beings do. No one but billionaires benefits if we all give up and pretend this is OK.
posted by abucci at 6:30 PM on June 14, 2023 [1 favorite]
See my comments above. I don't support the use of AI in this fashion. I stated unequivocally that computers can't do journalism. But in the end, my opinion means squat. I'm not in charge. I don't have the money or the power to make decisions.
As much as I may not like something, that doesn't mean that if I ignore it, it will go away. It's important to know what's out there and what technology is capable of and understand how it can be used--even if it's in an "know your enemy" kind of way.
That is the appeal of both the article I posted and the podcast kaibutsu posted (thanks for that). These are real reporters looking at their profession and their future and saying we think we still have value and we believe in what we do, but it's also the nature of our profession (and likely the mindset of people who become reporters) to look at what's out there, see what it's all about. This is them exploring the technology and contemplating the future in a way that puts things into stark perspective for both them and for the audience--which is what good journalism is supposed to do.
posted by sardonyx at 6:47 PM on June 14, 2023 [1 favorite]
As much as I may not like something, that doesn't mean that if I ignore it, it will go away. It's important to know what's out there and what technology is capable of and understand how it can be used--even if it's in an "know your enemy" kind of way.
That is the appeal of both the article I posted and the podcast kaibutsu posted (thanks for that). These are real reporters looking at their profession and their future and saying we think we still have value and we believe in what we do, but it's also the nature of our profession (and likely the mindset of people who become reporters) to look at what's out there, see what it's all about. This is them exploring the technology and contemplating the future in a way that puts things into stark perspective for both them and for the audience--which is what good journalism is supposed to do.
posted by sardonyx at 6:47 PM on June 14, 2023 [1 favorite]
Very well said, sardonyx.
posted by blue shadows at 11:02 PM on June 14, 2023 [1 favorite]
posted by blue shadows at 11:02 PM on June 14, 2023 [1 favorite]
« Older A Temporal Experience of Indefinite Detention | Another stroll through the Google Graveyard Newer »
This thread has been archived and is closed to new comments
Articles like this are the thin end of a wedge, because sure, when you discuss this show with these people, it seems OK. But we already have the FanFare page to discuss the show itself, so discussions here would necessarily center on "Look at this stuff ChatGPT wrote, isn't it coherent/weird/lifelike etc.?" and lead us down the path of "what can/can't/should/shouldn't LLMs do in replacing human writers" again.
posted by The Pluto Gangsta at 9:57 AM on June 13, 2023 [5 favorites]