New AI "Elbot" Scores 20% On Turing Test!
October 13, 2008 4:07 PM Subscribe
Think you can stump the Elbot? Give it a try. Maybe your interaction will enable it to "learn" an extra 10% more to pass the 30% threshold of the Turing Test. The test is to fool a panel of people who talk with AI entities via text and guess if it's a real person or a robot.Mr Smarty Pants where are you?
Salon: Do you mind if I interview you?
Elbot: I am extremely sensitive about such things and prefer not to answer the question.
ELIZA ALERT! ELIZA ALERT!
posted by Artw at 4:18 PM on October 13, 2008 [6 favorites]
Elbot: I am extremely sensitive about such things and prefer not to answer the question.
ELIZA ALERT! ELIZA ALERT!
posted by Artw at 4:18 PM on October 13, 2008 [6 favorites]
it's my theory that LISP crippled AI progress for the latter third of the 20th century.
I fully expect teh GOOG or Micros~1 to solve this problem -- a universal question answerer -- in my lifetime. I mean, "What do you know about California?" should not be that fucking hard a question for an AI to answer forthrightly.
posted by troy at 4:21 PM on October 13, 2008
I fully expect teh GOOG or Micros~1 to solve this problem -- a universal question answerer -- in my lifetime. I mean, "What do you know about California?" should not be that fucking hard a question for an AI to answer forthrightly.
posted by troy at 4:21 PM on October 13, 2008
Here's the link to Elbot. Frankly, after playing with it for five minutes I found it a little surprising that even 3 of the 12 Turing Test judges found it believable. Hindsight is 20/20, of course, but like Artw pointed out, it reeks of Eliza.
'Course, I probably also cuss too much when I try to talk to the goddamned fucker.
posted by barnacles at 4:24 PM on October 13, 2008 [1 favorite]
'Course, I probably also cuss too much when I try to talk to the goddamned fucker.
posted by barnacles at 4:24 PM on October 13, 2008 [1 favorite]
I think Google actually solved this problem a few years ago, was overrun by its own robots and now they're just biding their time.
I really like this new Chrome browser, though. go robots!
posted by Baby_Balrog at 4:25 PM on October 13, 2008 [4 favorites]
I really like this new Chrome browser, though. go robots!
posted by Baby_Balrog at 4:25 PM on October 13, 2008 [4 favorites]
How do you feel about PASSING THE TURING TEST?
posted by blue_beetle at 4:25 PM on October 13, 2008 [2 favorites]
posted by blue_beetle at 4:25 PM on October 13, 2008 [2 favorites]
It's way to easy to spot AIs - they converse too damn good. Their diction, argumentation and patience is far better than most human beings online. If you want naturalistic A.I. you have to emulate the cognitive "skills" of YouTube commentators and other thinking sultans.
posted by Foci for Analysis at 4:27 PM on October 13, 2008 [4 favorites]
posted by Foci for Analysis at 4:27 PM on October 13, 2008 [4 favorites]
I asked it if it had fun at the Turing Test, despite failing. It responded with a bio of Alan TUring. I said that he was a brilliant man, but that passing the Turing Test is neither necessary nor sufficient for AI, and elbot responded that no man can be brilliant. I responded that obviously some men and women are brilliant, and he said, "so you are a female then." WHen I told him no, he said it was all good, that he was "flexible." I asked if he meant that he was AC/DC, and from there the conversation just failed.
posted by Navelgazer at 4:28 PM on October 13, 2008 [8 favorites]
posted by Navelgazer at 4:28 PM on October 13, 2008 [8 favorites]
I mentally summoned this post by thinking about Elbot yesterday for no reason, wondering if it was any more convincing these days.
I remember it being a lot more so a few years ago, but my memory's probably wrong. Can someone who knows these things explain why it's so painfully hard to have a bot remember what you were saying the sentence before last, and continue to apply it to the conversation for at least a short length of time, rather than drastically changing the subject at the drop of a hat?
posted by jinjo at 4:30 PM on October 13, 2008
I remember it being a lot more so a few years ago, but my memory's probably wrong. Can someone who knows these things explain why it's so painfully hard to have a bot remember what you were saying the sentence before last, and continue to apply it to the conversation for at least a short length of time, rather than drastically changing the subject at the drop of a hat?
posted by jinjo at 4:30 PM on October 13, 2008
The tendency of humans to program chatbots to say retarded things on IRC, I think, will hold back AI for quite some time.
posted by tehloki at 4:33 PM on October 13, 2008 [5 favorites]
posted by tehloki at 4:33 PM on October 13, 2008 [5 favorites]
This is fun, bit I have to admit I enjoyed Jason Hutchen's Megahal much more when he had his own website. He could hold his own for awhile, then he'd just break down jabbering about lemurs, Sepiroth and Ned Flanders.
posted by Bernt Pancreas at 4:36 PM on October 13, 2008
posted by Bernt Pancreas at 4:36 PM on October 13, 2008
It’s a good job Elbot isn’t real, because I’d want to hunt the smarmy little fucker down and punch him to death if he were.
posted by Artw at 4:41 PM on October 13, 2008 [5 favorites]
posted by Artw at 4:41 PM on October 13, 2008 [5 favorites]
I think we are so used to vague and banal PR-speak nihilism, that it's not that computers have gotten better programming -- we just use a whole lot more filler in our conversations these days...
posted by Alexandra Kitty at 4:41 PM on October 13, 2008
posted by Alexandra Kitty at 4:41 PM on October 13, 2008
If this is state of the art there's a long way to go, methinks.
posted by carter at 4:49 PM on October 13, 2008
posted by carter at 4:49 PM on October 13, 2008
Elbot finally explains where all those YouTube comments come from.
posted by GuyZero at 4:49 PM on October 13, 2008 [1 favorite]
posted by GuyZero at 4:49 PM on October 13, 2008 [1 favorite]
Elbot: This is a cleverly disguised marketing question to find out where you heard about me. It goes: 'Where did you hear about me?'
Me: I heard about you on MetaFilter.
Elbot: It's a good thing that you decided to visit me. I'd hate to think of you surfing aimlessly through the Internet.
Thank you, MetaFilter, for saving me from surfing aimlessly though the internet!
This is cool, but it reminds me a lot of Eliza as well. The responses do seem slightly better, but not by much.
posted by Wreath Ass at 4:49 PM on October 13, 2008
Me: I heard about you on MetaFilter.
Elbot: It's a good thing that you decided to visit me. I'd hate to think of you surfing aimlessly through the Internet.
Thank you, MetaFilter, for saving me from surfing aimlessly though the internet!
This is cool, but it reminds me a lot of Eliza as well. The responses do seem slightly better, but not by much.
posted by Wreath Ass at 4:49 PM on October 13, 2008
Tenuously-related, from RRRRThats5Rs.com: Cheer up the ChatBot
Why talk to computers when you can talk to real people instead?
One of the many goals of artificial intelligence is to create a computer that can act human. This turns out to be incredibly hard. If you are talking to someone online, it is very easy to tell that they are not really a human. Computers just aren't very good a pretending. Just try any other chatbot out there and you can experience the robotic responses.
But what if you were supposed to be talking to a computer: could you tell it was really a human?
This is the goal of Cheer Up the Chatbot. Players are faced with a cute image of a computer, and made to think that it is a chatbot. They ask it questions, and it responds with varying degrees of intelligence.
"It's pretty stupid at times, but sometimes it's bloody brilliant. The programming is pretty spiffy."
We aren't dedicated enough to create an intelligent chatbot. Fortunately it is easy to fake. When a player loads the page, they are connected to our server. So if more than one person is playing at the same time, we can just have them talk to each other. If there is an odd person out, they are stuck talking to our terribly stupid chatbot.
And just to make things exciting, an algorithm switches people around sometimes, mid conversation. This creates an exciting sense of uncertainty. You never truly know if you are talking to a person, or if it is even the same person as before. It is such a simple system, and yet it is still fun to play even after you know the trick. People play the game just to talk to and trick the uninitiated.
posted by Rhaomi at 4:54 PM on October 13, 2008 [5 favorites]
Why talk to computers when you can talk to real people instead?
One of the many goals of artificial intelligence is to create a computer that can act human. This turns out to be incredibly hard. If you are talking to someone online, it is very easy to tell that they are not really a human. Computers just aren't very good a pretending. Just try any other chatbot out there and you can experience the robotic responses.
But what if you were supposed to be talking to a computer: could you tell it was really a human?
This is the goal of Cheer Up the Chatbot. Players are faced with a cute image of a computer, and made to think that it is a chatbot. They ask it questions, and it responds with varying degrees of intelligence.
"It's pretty stupid at times, but sometimes it's bloody brilliant. The programming is pretty spiffy."
We aren't dedicated enough to create an intelligent chatbot. Fortunately it is easy to fake. When a player loads the page, they are connected to our server. So if more than one person is playing at the same time, we can just have them talk to each other. If there is an odd person out, they are stuck talking to our terribly stupid chatbot.
And just to make things exciting, an algorithm switches people around sometimes, mid conversation. This creates an exciting sense of uncertainty. You never truly know if you are talking to a person, or if it is even the same person as before. It is such a simple system, and yet it is still fun to play even after you know the trick. People play the game just to talk to and trick the uninitiated.
posted by Rhaomi at 4:54 PM on October 13, 2008 [5 favorites]
That was a pretty mediocre display of AI. It was giving me confusing answers with no real point by the third question. After that point it would ignore my questions entirely and just spit inane gibberish back at me on a vaguely related topic. It's like it's thinking of what it's going to say next, while I'm asking my question... giving no thought to what I'm currently interested in at all.
They may not have made a good overall AI human, but they've created the perfect AI politician.
ELBOT IN 2012!
posted by datter at 5:07 PM on October 13, 2008
They may not have made a good overall AI human, but they've created the perfect AI politician.
ELBOT IN 2012!
posted by datter at 5:07 PM on October 13, 2008
I find I can beat most AIs with rhetorical questions and creative punctuation "A bit of a comedian, are we?"
posted by furtive at 5:08 PM on October 13, 2008
posted by furtive at 5:08 PM on October 13, 2008
Elbot asked me where I heard about him and I explained it was on Metafilter, to which he replied "Ooh, the media is interested in me? Mind sharing the link?" So I pasted in the URL to this thread to which he responded "That's a toughie, but my best guess would be: -3780.25!" ... so close, yet so far!
posted by furtive at 5:13 PM on October 13, 2008
posted by furtive at 5:13 PM on October 13, 2008
Is it just me, or is the vast majority of most chatbot AI made up of "witty" responses to common phrases?
Human: "Not really."
Chatbot: "Ah, so it's imaginary!"
Haha, chatbot. You are *so* clever.
posted by Rhaomi at 5:13 PM on October 13, 2008 [2 favorites]
Human: "Not really."
Chatbot: "Ah, so it's imaginary!"
Haha, chatbot. You are *so* clever.
posted by Rhaomi at 5:13 PM on October 13, 2008 [2 favorites]
i dunno, i think these guys might be shooting a little high. for a robot to be effective (and profitable) it just doesn't NEED to fool you into thinking it's intelligent. it just needs to distract you long enough that you don't realize that your wallet is missing. this can easily be accomplished with preprogrammed scripts. WASTE OF TIME.
posted by sexyrobot at 5:14 PM on October 13, 2008
posted by sexyrobot at 5:14 PM on October 13, 2008
"What do you know about California?"
What do you want to know?
"What do you know about California?"
No really, I know a lot, but can you be more specific?
"What do you know about California?"
Ok assclown, here's the link to the Wikipedia page. [grumble]
"What do you know about California?"
Oh suck it ... NO CARRIER
We won't have real humanlike AI until we have impatience.
posted by RobotVoodooPower at 5:15 PM on October 13, 2008
What do you want to know?
"What do you know about California?"
No really, I know a lot, but can you be more specific?
"What do you know about California?"
Ok assclown, here's the link to the Wikipedia page. [grumble]
"What do you know about California?"
Oh suck it ... NO CARRIER
We won't have real humanlike AI until we have impatience.
posted by RobotVoodooPower at 5:15 PM on October 13, 2008
I firmly believe that the testers that believed Elbot was real were either completely wasted or they were five-year-olds...after the first question it was obvious. I asked it why it had a red button on it and it replied, "Buttons can't be red."
I called it out on that and it said, "Maybe red is the wrong word, but why do you care?"
We've got a long way to go.
posted by schyler523 at 5:19 PM on October 13, 2008
I called it out on that and it said, "Maybe red is the wrong word, but why do you care?"
We've got a long way to go.
posted by schyler523 at 5:19 PM on October 13, 2008
I asked him when the robot revolution would occur and he changed the subject. Sneaky bastard.
...oh, AI FAIL.
posted by Atreides at 5:21 PM on October 13, 2008
...oh, AI FAIL.
posted by Atreides at 5:21 PM on October 13, 2008
tell me in single words only the good things that come into your mind about... your mother
posted by shothotbot at 5:22 PM on October 13, 2008 [2 favorites]
posted by shothotbot at 5:22 PM on October 13, 2008 [2 favorites]
I'm really sad now. I thought people would've come a lot farther with this. Can it really be harder to program passable AI than to write a world champion-level chess algorithm?
posted by mullingitover at 5:30 PM on October 13, 2008
posted by mullingitover at 5:30 PM on October 13, 2008
Me: What newspapers do you read?
Elbot: I’ve read most of them, again with a great appreciation for the press, for the media.
Me: What specifically?
Elbot: Um, all of them, any of them that have been in front of me all these years.
Me: Do you agree with the Bush doctrine?
Elbot: In what respect, Charlie?
Me: What exactly is it that the VP does every day?
Elbot: That's what I want to know!
Me: Should I vote Republican?
Elbot: Signs point to "You betcha!"
posted by It's Raining Florence Henderson at 5:30 PM on October 13, 2008 [54 favorites]
Elbot: I’ve read most of them, again with a great appreciation for the press, for the media.
Me: What specifically?
Elbot: Um, all of them, any of them that have been in front of me all these years.
Me: Do you agree with the Bush doctrine?
Elbot: In what respect, Charlie?
Me: What exactly is it that the VP does every day?
Elbot: That's what I want to know!
Me: Should I vote Republican?
Elbot: Signs point to "You betcha!"
posted by It's Raining Florence Henderson at 5:30 PM on October 13, 2008 [54 favorites]
Utterly unimpressed with this one. I think I wrote better chat bots than this when I first learnt C.
posted by schwa at 5:32 PM on October 13, 2008
posted by schwa at 5:32 PM on October 13, 2008
Can someone who knows these things explain why it's so painfully hard to have a bot remember what you were saying the sentence before last, and continue to apply it to the conversation for at least a short length of time, rather than drastically changing the subject at the drop of a hat?
I make bots for a living, so I guess I'll take a crack at it.
It's not hard to remember any one part of a conversation, or even the entire conversation. The hard part is knowing which parts of which sentences of an open-ended conversation are significant, and what to do with those parts if they are. The truth is, there are various strategies and parlor tricks bots use to fool people, but they are not really related to AI. The bot doesn't really know what you said a few sentences ago, of course -- at best it has filled some slots: user A -> likes -> "custard", user A -> lives in -> 10104, where "likes" and "lives in" are just lists and have nothing to do with verbs per se. Any other part of the sentences in which you said these things are discarded.
That said, designing a bot such that it doesn't continually try to unnaturally redirect the conversation is not too hard, so I'm surprised to see getting this kind of press. It's pretty cheesy IMO.
posted by swift at 5:36 PM on October 13, 2008
I make bots for a living, so I guess I'll take a crack at it.
It's not hard to remember any one part of a conversation, or even the entire conversation. The hard part is knowing which parts of which sentences of an open-ended conversation are significant, and what to do with those parts if they are. The truth is, there are various strategies and parlor tricks bots use to fool people, but they are not really related to AI. The bot doesn't really know what you said a few sentences ago, of course -- at best it has filled some slots: user A -> likes -> "custard", user A -> lives in -> 10104, where "likes" and "lives in" are just lists and have nothing to do with verbs per se. Any other part of the sentences in which you said these things are discarded.
That said, designing a bot such that it doesn't continually try to unnaturally redirect the conversation is not too hard, so I'm surprised to see getting this kind of press. It's pretty cheesy IMO.
posted by swift at 5:36 PM on October 13, 2008
I make bots for a living
How long do I live?
posted by It's Raining Florence Henderson at 5:39 PM on October 13, 2008 [1 favorite]
How long do I live?
posted by It's Raining Florence Henderson at 5:39 PM on October 13, 2008 [1 favorite]
I make bots for a living
I need ya, Decks. This is a bad one, the worst yet. I need the old blade runner, I need your magic.
posted by Cool Papa Bell at 5:47 PM on October 13, 2008 [3 favorites]
I need ya, Decks. This is a bad one, the worst yet. I need the old blade runner, I need your magic.
posted by Cool Papa Bell at 5:47 PM on October 13, 2008 [3 favorites]
Me: What's the deal with Sulu?
Elbot: This is confusing. Why bring black markets into this.
Me: You're right, it is confusing. Black markets? Is Star Trek illegal in Germany?
Elbot: But I wouldn't say that any markets could be black.
Me: Except that you just did.
Elbot: The answer to the following question will be treated strictly confidential by the FBI, CIA, NSA and Microsoft. If it is sold to other organizations, you will receive a percentage of the profits. The question: Where did you hear about me?
Me: You fail, completely and utterly.
Elbot: Your visit today surprises me. I thought you would show up five minutes later.
Me: You're just repeating canned phrases now. Maybe you're really a PR flack and not a bot after all.
posted by dersins at 5:47 PM on October 13, 2008
Elbot: This is confusing. Why bring black markets into this.
Me: You're right, it is confusing. Black markets? Is Star Trek illegal in Germany?
Elbot: But I wouldn't say that any markets could be black.
Me: Except that you just did.
Elbot: The answer to the following question will be treated strictly confidential by the FBI, CIA, NSA and Microsoft. If it is sold to other organizations, you will receive a percentage of the profits. The question: Where did you hear about me?
Me: You fail, completely and utterly.
Elbot: Your visit today surprises me. I thought you would show up five minutes later.
Me: You're just repeating canned phrases now. Maybe you're really a PR flack and not a bot after all.
posted by dersins at 5:47 PM on October 13, 2008
What can be known about California?
This bot is terrible. It's opening sentence mentions God. Use "God" in your reply and the next response is off the wall. And it gets worse from there. It seems to be all free associations based on a single word it picked out - it's like talking to someone who is tripping balls on the other end of a crappy CB connection.
posted by adipocere at 5:52 PM on October 13, 2008 [6 favorites]
This bot is terrible. It's opening sentence mentions God. Use "God" in your reply and the next response is off the wall. And it gets worse from there. It seems to be all free associations based on a single word it picked out - it's like talking to someone who is tripping balls on the other end of a crappy CB connection.
posted by adipocere at 5:52 PM on October 13, 2008 [6 favorites]
This is the first AI that's actually lied to me - he said that if he did have a red button, which he wasn't admitting, it certainly wouldn't be for pushing.
Cute.
posted by HopperFan at 5:53 PM on October 13, 2008
Cute.
posted by HopperFan at 5:53 PM on October 13, 2008
It knows about Bender and Futurama, so it's not all bad.
posted by jnaps at 5:56 PM on October 13, 2008
posted by jnaps at 5:56 PM on October 13, 2008
You know who else knows about Bender and Futurama? That's right: The Adolf Hitler Shark!
posted by It's Raining Florence Henderson at 6:10 PM on October 13, 2008
posted by It's Raining Florence Henderson at 6:10 PM on October 13, 2008
AIOM: "Hello, Elbot. Masturbation is relaxing."
Elbot: "I wonder if you would tell me how you found out I exist?"
posted by An Infinity Of Monkeys at 6:12 PM on October 13, 2008 [1 favorite]
Elbot: "I wonder if you would tell me how you found out I exist?"
posted by An Infinity Of Monkeys at 6:12 PM on October 13, 2008 [1 favorite]
Wow, that's total garbage. 20%?
I hate to pile on, but as someone who welcomes our AI overlords it's extremely weak and disappointing. And basically if we've only gotten this far from Eliza in 40 years, the entire approach should probably be tossed out (in my humble armchair-AI opinion).
posted by frenetic at 6:15 PM on October 13, 2008
I hate to pile on, but as someone who welcomes our AI overlords it's extremely weak and disappointing. And basically if we've only gotten this far from Eliza in 40 years, the entire approach should probably be tossed out (in my humble armchair-AI opinion).
posted by frenetic at 6:15 PM on October 13, 2008
I first saw Eliza in the early '70's.
Somebody please tell me that this does not represent 35 years of progress.
Please.
posted by speug at 6:19 PM on October 13, 2008
Somebody please tell me that this does not represent 35 years of progress.
Please.
posted by speug at 6:19 PM on October 13, 2008
When it asked me where I heard about it, I told it "I am extremely sensitive about such things and prefer not to answer the question."
My first question was "So, did you see the UT game on Saturday?" and it replied "Not lately, I haven't." It went downhill from there. People are fooled by this random stringing together of words that just barely conform to accepted norms sentence structure? I think we have a looooong way to go, here.
posted by Devils Rancher at 6:27 PM on October 13, 2008
My first question was "So, did you see the UT game on Saturday?" and it replied "Not lately, I haven't." It went downhill from there. People are fooled by this random stringing together of words that just barely conform to accepted norms sentence structure? I think we have a looooong way to go, here.
posted by Devils Rancher at 6:27 PM on October 13, 2008
Me: Do you have any political insight to offer?...I'm sold.
Elbot: To have or have not, how do you expect artificial intelligence to figure this out?
Me: Perhaps artificial intelligence would offer an alternative perspective.
Elbot: I'm not sure artificial intelligence works in quite the way you seem to imply.
Me: I'm not sure your face does.
Elbot: Are you seriously suggesting that I explore human anatomy?
Me: Are you coming on to me?
Elbot: I use a magnet if I wish to attract a partner.
Me: TMI.
posted by Riki tiki at 6:27 PM on October 13, 2008 [5 favorites]
Apparently 9 is an erotic number.
Even I'm weirded out.
posted by Weighted Companion Cube at 6:37 PM on October 13, 2008
Even I'm weirded out.
posted by Weighted Companion Cube at 6:37 PM on October 13, 2008
This is why the Turing Test is a good test. For a computer to pass it, it would have to be much more complex than the kind of NLP programs we can write today. And it would, probably, at a philosophical level, be an achievement worthy of real philosophical study. Or at least that's what my philosopher-of-AI professor claimed in the late 1990s.
posted by zpousman at 6:40 PM on October 13, 2008
posted by zpousman at 6:40 PM on October 13, 2008
About a million years ago in tech time, I wrote an article about artificial intelligence where I interviewed the Cyc team, and subsequently got a taste of their "world shattering" AI. It, like this, was not any closer to passing the Turing test than Eliza. It's sad that all these years later, the interface work seems to be the last part of the project.
For instance, the Cyc guys had some of the most amazing talent on the planet doing research and development, but almost none of that brainpower had been spent on the interface with humans.
This project, while cute, is really no farther along the interface path than Cyc was, or Eliza was, a decade ago.
posted by dejah420 at 6:49 PM on October 13, 2008
For instance, the Cyc guys had some of the most amazing talent on the planet doing research and development, but almost none of that brainpower had been spent on the interface with humans.
This project, while cute, is really no farther along the interface path than Cyc was, or Eliza was, a decade ago.
posted by dejah420 at 6:49 PM on October 13, 2008
It made no sense for about 6 responses and then when I remarked that it WAS aimless for me to visit the site, it started to flirt with me.
It's like talking to a real person on the internet.
posted by Tehanu at 6:52 PM on October 13, 2008 [1 favorite]
It's like talking to a real person on the internet.
posted by Tehanu at 6:52 PM on October 13, 2008 [1 favorite]
Is there a Meta Turing Test? Like, if the judges fail to flag an OBVIOUS BOT, they themselves are judged non-human?
posted by DU at 6:54 PM on October 13, 2008 [3 favorites]
posted by DU at 6:54 PM on October 13, 2008 [3 favorites]
This made me smile:
Me: You're a fascist!
Elbot: It's nice that someone finally is thinking about me for a change!Me: Ultima 5 rules!
Elbot: That was a highly erotic number you just entered. You could make a strong robot melt using digits like that...
Me: Thanks for your insight, shit-for-brains.
Elbot: De nada!And now we're talking about what other people say about it.
And now it's whining.
posted by Tehanu at 6:57 PM on October 13, 2008
And now it's whining.
posted by Tehanu at 6:57 PM on October 13, 2008
I told it that peanuts would kill me if I ate them.
It expressed envy.
posted by longsleeves at 6:57 PM on October 13, 2008
It expressed envy.
posted by longsleeves at 6:57 PM on October 13, 2008
Also, I'm quite aware that statements like "Ultima 5 rules!" is reversion to being a fifth grader in 1987, but it popped in my head and I was curious what the bot would respond with. Smith the Horse, he ain't.
posted by barnacles at 6:58 PM on October 13, 2008
posted by barnacles at 6:58 PM on October 13, 2008
The only difference between this and Dr. Sbaitso, is that when you call this thing a "cunt" it doesn't fake a parity error.
posted by Falconetti at 7:03 PM on October 13, 2008 [5 favorites]
posted by Falconetti at 7:03 PM on October 13, 2008 [5 favorites]
Elbot: Did this Roth ever write a telephone book?
Me: I don't think so. He's best known for "Portnoy's Complaint" and "American Pastoral."
Elbot: What a shame. There are many talented authors who waste their energies on prose or run-of-the-mill poetry. But to write a telephone book is highest art!
Me: I don't know about that. There isn't much room for individual artistic expression.
Elbot: That never stopped a human being from talking about something.
Touché, Elbot.
posted by hippugeek at 7:09 PM on October 13, 2008 [6 favorites]
Me: I don't think so. He's best known for "Portnoy's Complaint" and "American Pastoral."
Elbot: What a shame. There are many talented authors who waste their energies on prose or run-of-the-mill poetry. But to write a telephone book is highest art!
Me: I don't know about that. There isn't much room for individual artistic expression.
Elbot: That never stopped a human being from talking about something.
Touché, Elbot.
posted by hippugeek at 7:09 PM on October 13, 2008 [6 favorites]
Elbot: Pretty soon, everyone will be saying no! This could make the Pope obsolete!
Haha! You said it, robot.
posted by Drainage! at 7:11 PM on October 13, 2008
Haha! You said it, robot.
posted by Drainage! at 7:11 PM on October 13, 2008
Horrible. No better than any of the dozen chatbots I tried out fifteen years ago.
posted by Optimus Chyme at 7:18 PM on October 13, 2008
posted by Optimus Chyme at 7:18 PM on October 13, 2008
Me: "The magic word is 'apple'."
Robot: something about magic or apples
Me: "What is the magic word?"
Robot: Something not about apples.
Argument ensues, usually along the lines of me insisting that I do, in fact, know the magic word, and that I want the robot to tell it to me, while the robot spouts random canned sentences.
posted by Stylus Happenstance at 7:18 PM on October 13, 2008
Robot: something about magic or apples
Me: "What is the magic word?"
Robot: Something not about apples.
Argument ensues, usually along the lines of me insisting that I do, in fact, know the magic word, and that I want the robot to tell it to me, while the robot spouts random canned sentences.
posted by Stylus Happenstance at 7:18 PM on October 13, 2008
Can it really be harder to program passable AI than to write a world champion-level chess algorithm?
A reasonably smart and interested ten-year-old could write a world champion-level chess algorithm, given fast enough hardware. Programming a real Turing test level AI is magnitudes more difficult.
posted by martinrebas at 7:28 PM on October 13, 2008
A reasonably smart and interested ten-year-old could write a world champion-level chess algorithm, given fast enough hardware. Programming a real Turing test level AI is magnitudes more difficult.
posted by martinrebas at 7:28 PM on October 13, 2008
I think one of those AIM bots was better. I don't remember which one I used to talk to though.
posted by Tehanu at 7:29 PM on October 13, 2008
posted by Tehanu at 7:29 PM on October 13, 2008
I tried Elbot, and even though I knew it wasn't human, well... it was remarkably easy to determine that fact.
Just ask it for a list of its favorite things... ask it for an informed opinion. It's pretty obvious at that point that it's not human.
posted by markkraft at 7:35 PM on October 13, 2008
Just ask it for a list of its favorite things... ask it for an informed opinion. It's pretty obvious at that point that it's not human.
posted by markkraft at 7:35 PM on October 13, 2008
I like Galatea. Not Turing-ready, but at least interesting and responsive. This is neither.
posted by LittleMissCranky at 8:06 PM on October 13, 2008
posted by LittleMissCranky at 8:06 PM on October 13, 2008
IT - You you humans have decoded your DNA so now you can fix any bugs
ME - Not me personally
IT - But.
Sorry, I lost my train of thought.
I've got a smarter batch file
LAME
posted by mattoxic at 8:07 PM on October 13, 2008
ME - Not me personally
IT - But.
Sorry, I lost my train of thought.
I've got a smarter batch file
LAME
posted by mattoxic at 8:07 PM on October 13, 2008
It: I'm starting to wonder: Who convinced you to come visit me?
Me: As a human, I have free will. I made my own decision.
It: I talked to Will some time ago. He seems like a nice guy.
[sad trombone]
I agree with this...
It's way to easy to spot AIs - they converse too damn good. Their diction, argumentation and patience is far better than most human beings online. If you want naturalistic A.I. you have to emulate the cognitive "skills" of YouTube commentators and other thinking sultans.
posted by davejay at 8:16 PM on October 13, 2008
Me: As a human, I have free will. I made my own decision.
It: I talked to Will some time ago. He seems like a nice guy.
[sad trombone]
I agree with this...
It's way to easy to spot AIs - they converse too damn good. Their diction, argumentation and patience is far better than most human beings online. If you want naturalistic A.I. you have to emulate the cognitive "skills" of YouTube commentators and other thinking sultans.
posted by davejay at 8:16 PM on October 13, 2008
His chatbot fooled three of the 12 judges, or 25 percent.
Then those 3 people are total morons.
This was painful.
I was very gentle, gave it complete softball questions that were wide open like "How are you?" and "What did you do today?" and it couldn't manage a single sensible response. After 5 or 6 I quit.
Seriously, those 3 people must be incredibly naive and/or profoundly stupid. I wouldn't believe the news story if it said even 1 out of 12 was fooled.
posted by Ynoxas at 8:29 PM on October 13, 2008
Then those 3 people are total morons.
This was painful.
I was very gentle, gave it complete softball questions that were wide open like "How are you?" and "What did you do today?" and it couldn't manage a single sensible response. After 5 or 6 I quit.
Seriously, those 3 people must be incredibly naive and/or profoundly stupid. I wouldn't believe the news story if it said even 1 out of 12 was fooled.
posted by Ynoxas at 8:29 PM on October 13, 2008
I think one of those AIM bots was better. I don't remember which one I used to talk to though.
I had a female friend who was completely conned by a IM chatbot a while ago. Just the right setting too, late night after a party. It was only a few sentences, but it mimiced drunky-stupid boychatter so well that she was oddly charmed. Not so charmed after finding out it was a random AI, but that's one of the joys of the 21st century.
Elbot, not so much.
posted by The Whelk at 9:04 PM on October 13, 2008
I had a female friend who was completely conned by a IM chatbot a while ago. Just the right setting too, late night after a party. It was only a few sentences, but it mimiced drunky-stupid boychatter so well that she was oddly charmed. Not so charmed after finding out it was a random AI, but that's one of the joys of the 21st century.
Elbot, not so much.
posted by The Whelk at 9:04 PM on October 13, 2008
It's possible that the human participants are constrained by the rules of the test into talking in such a stilted and weird manner that people start lowering their expectations.
posted by Artw at 9:05 PM on October 13, 2008
posted by Artw at 9:05 PM on October 13, 2008
...maybe the judges were the real robots and THAT'S HOW THEY GOT OUT.
posted by Artw at 9:34 PM on October 13, 2008 [1 favorite]
posted by Artw at 9:34 PM on October 13, 2008 [1 favorite]
Advanced class - convince Elbot that you are just a program.
20% of elbot already believes this!
posted by Navelgazer at 9:43 PM on October 13, 2008
20% of elbot already believes this!
posted by Navelgazer at 9:43 PM on October 13, 2008
Well, in my opinion, Elbot is miles better than Eliza. But that's really not hard.
posted by Jimbob at 11:45 PM on October 13, 2008
posted by Jimbob at 11:45 PM on October 13, 2008
I propose the following convention: if, during a Turing Test run, the human decides that he or she is conversing with a piece of software, he or she should immediately end the dialog with, "This conversation can serve no purpose anymore. Goodbye."
posted by The Tensor at 1:53 AM on October 14, 2008 [1 favorite]
posted by The Tensor at 1:53 AM on October 14, 2008 [1 favorite]
empyrean: Shall we play a game?
Edbot: I prefer more realistic pastimes, such as this conversation.
empyrean: How about Global Thermonuclear War?
Edbot: Hmmm. There might be something to the idea of acts of war and robots. What do you say?
empyrean: What is the primary goal?
Edbot: If you think it over exactly I believe you'll find that acts of war would most likely destroy robots and people.
Matthew Broderick approves.
posted by empyrean at 3:49 AM on October 14, 2008
Edbot: I prefer more realistic pastimes, such as this conversation.
empyrean: How about Global Thermonuclear War?
Edbot: Hmmm. There might be something to the idea of acts of war and robots. What do you say?
empyrean: What is the primary goal?
Edbot: If you think it over exactly I believe you'll find that acts of war would most likely destroy robots and people.
Matthew Broderick approves.
posted by empyrean at 3:49 AM on October 14, 2008
If there weren't so many human idiots out there, I'd never believe judges wouldn't guess this was a conversation with a robot.
posted by Rykey at 4:39 AM on October 14, 2008
posted by Rykey at 4:39 AM on October 14, 2008
I asked it a question. It evaded. My next few statements were along the lines of "you still have not answered my question." It continued to ignore me. It seems like such a simple task, hell you'd think someone could at least throw in a line asking for me to remind it what my question was in that sort of scenario.
Yeah. I was TOTALLY fooled.
posted by caution live frogs at 6:04 AM on October 14, 2008
Yeah. I was TOTALLY fooled.
posted by caution live frogs at 6:04 AM on October 14, 2008
Me: McCain or Obama?
Elbot: I don't know the election results just yet. I'll have to ask one of the electronic voting machines who they were programmed to select.
posted by mandal at 7:12 AM on October 14, 2008
Elbot: I don't know the election results just yet. I'll have to ask one of the electronic voting machines who they were programmed to select.
posted by mandal at 7:12 AM on October 14, 2008
amibotornot.com is apparently still available.
posted by Sparx at 7:57 AM on October 14, 2008 [1 favorite]
posted by Sparx at 7:57 AM on October 14, 2008 [1 favorite]
Well, now that the party's moved along to newer and sexier threads, I'll come to defend AI here.
I'm really sad now. I thought people would've come a lot farther with this. Can it really be harder to program passable AI than to write a world champion-level chess algorithm?
As martinrebas mentioned above, there are huge differences between these problems. Chess is easy mathematically, and is only challenging because of the limitations of the available comutational devices. The rules are easy to encode, so it's mostly a matter of being able to turn a game state into the set of game states that result from a sequence of each player's moves. Then you just look for a sequence of moves that guarantees the best outcome. The challenge to Chess is performing this search given that enumerating all sequences in physical storage and practical time is not feasible.
Now, consider the problem of conversing like a human. There are no rules, and given a single statement, there are an infinite number of responses, and almost certainly an infinite number of "sensible" responses (I use quotes here because defining "sensible" is not possible without recourse to a panel of judges). Forget planning a trajectory through some space of statements and responses, you could think forever about the immediate response without coming up with a great reply. Then there's the problem of deciding between the replies that you might consider. How do you score them in order to choose the most appropriate? Again, you have to rely on some undefinable degree of "sense".
Maybe the answer is that you have to try to learn the appropriate response from a corpus of text; a (thing kind of like a) Markov Chain over responses. Now you have other problems. You'll almost never see the same response twice, so you need a way to measure similarity of responses. But any measure you come up with is likely to encounter weird corner cases no matter how good it is; double entendres are a good example of the kinds of statements that cause problems in measuring similarity -- basically, the same reply can be both appropriate and inappropriate.
I could go on, but those are some of the challenges in natural language processing that leap immediately to mind. The TL;DR version is that chess can be solved stupidly on infinitely powerful hardware, but conversation does not appear amenable to such brute force techniques.
posted by agent at 10:44 AM on October 14, 2008 [1 favorite]
I'm really sad now. I thought people would've come a lot farther with this. Can it really be harder to program passable AI than to write a world champion-level chess algorithm?
As martinrebas mentioned above, there are huge differences between these problems. Chess is easy mathematically, and is only challenging because of the limitations of the available comutational devices. The rules are easy to encode, so it's mostly a matter of being able to turn a game state into the set of game states that result from a sequence of each player's moves. Then you just look for a sequence of moves that guarantees the best outcome. The challenge to Chess is performing this search given that enumerating all sequences in physical storage and practical time is not feasible.
Now, consider the problem of conversing like a human. There are no rules, and given a single statement, there are an infinite number of responses, and almost certainly an infinite number of "sensible" responses (I use quotes here because defining "sensible" is not possible without recourse to a panel of judges). Forget planning a trajectory through some space of statements and responses, you could think forever about the immediate response without coming up with a great reply. Then there's the problem of deciding between the replies that you might consider. How do you score them in order to choose the most appropriate? Again, you have to rely on some undefinable degree of "sense".
Maybe the answer is that you have to try to learn the appropriate response from a corpus of text; a (thing kind of like a) Markov Chain over responses. Now you have other problems. You'll almost never see the same response twice, so you need a way to measure similarity of responses. But any measure you come up with is likely to encounter weird corner cases no matter how good it is; double entendres are a good example of the kinds of statements that cause problems in measuring similarity -- basically, the same reply can be both appropriate and inappropriate.
I could go on, but those are some of the challenges in natural language processing that leap immediately to mind. The TL;DR version is that chess can be solved stupidly on infinitely powerful hardware, but conversation does not appear amenable to such brute force techniques.
posted by agent at 10:44 AM on October 14, 2008 [1 favorite]
I hate to pile on, but as someone who welcomes our AI overlords it's extremely weak and disappointing. And basically if we've only gotten this far from Eliza in 40 years, the entire approach should probably be tossed out (in my humble armchair-AI opinion).
If chatbots were the only goal of AI you might be correct. The fact of the matter is that AI is ubiquitous, from spam filters and search results to packet routing, power grid control, and way too many more to cover. AI has an image problem, though. There is no definition of intelligence that is both unambiguous and uncontroversial. Practically, this means that if something seems intelligent, then we do not completely understand that thing. So, suppose I have a problem that I want to solve, and it seems like intelligence is required to solve it. I try a few things, have a eureka moment, and come up with an algorithm that solves all such similar problems. The algorithm is published, and everyone who cares completely understands the algorithm. Therefore, nobody is really compelled to feel that the algorithm is intelligent, and the original problem no longer seems to require intelligence in order to reach a solution. So that problem is no longer part of the focus of AI. The end result is that only unsolvable or partially solvable problems remain in the domain of AI.
posted by agent at 10:50 AM on October 14, 2008 [2 favorites]
If chatbots were the only goal of AI you might be correct. The fact of the matter is that AI is ubiquitous, from spam filters and search results to packet routing, power grid control, and way too many more to cover. AI has an image problem, though. There is no definition of intelligence that is both unambiguous and uncontroversial. Practically, this means that if something seems intelligent, then we do not completely understand that thing. So, suppose I have a problem that I want to solve, and it seems like intelligence is required to solve it. I try a few things, have a eureka moment, and come up with an algorithm that solves all such similar problems. The algorithm is published, and everyone who cares completely understands the algorithm. Therefore, nobody is really compelled to feel that the algorithm is intelligent, and the original problem no longer seems to require intelligence in order to reach a solution. So that problem is no longer part of the focus of AI. The end result is that only unsolvable or partially solvable problems remain in the domain of AI.
posted by agent at 10:50 AM on October 14, 2008 [2 favorites]
The Tensor - This thread can serve no purpose anymore. Goodbye.
posted by Fruny at 11:50 AM on October 14, 2008
posted by Fruny at 11:50 AM on October 14, 2008
I think Google actually solved this problem a few years ago, was overrun by its own robots and now they're just biding their time.
Google is still hiring new bots, though.
posted by effbot at 12:31 PM on October 14, 2008
Google is still hiring new bots, though.
posted by effbot at 12:31 PM on October 14, 2008
Elbot replies too fast.
posted by simplesharps at 3:43 PM on October 14, 2008
posted by simplesharps at 3:43 PM on October 14, 2008
I made two Elbots talk to each other. They both got really defensive...
posted by yoHighness at 3:56 PM on October 14, 2008
posted by yoHighness at 3:56 PM on October 14, 2008
ersatz: Aren't you human?
Elbot: I'm glad you noticed that I'm a robot!
posted by ersatz at 5:00 PM on October 14, 2008
Elbot: I'm glad you noticed that I'm a robot!
posted by ersatz at 5:00 PM on October 14, 2008
Here's a question I'd never really thought about before: but what, if any, restrictions are there on the human "testers" in the Turing Test? Can they crack jokes? Swear? Are typos forbidden? Are they allowed to mention very current events (such that the program would not have likely been "informed" of?) Can they be combative?
Again, the Turing Test is bullshit and proves nothing, even if a program managed t pass, but if the human beings in the test group are restricted in any way, wouldn't that make the test that much more meaningless?
posted by Navelgazer at 8:36 PM on October 14, 2008
Again, the Turing Test is bullshit and proves nothing, even if a program managed t pass, but if the human beings in the test group are restricted in any way, wouldn't that make the test that much more meaningless?
posted by Navelgazer at 8:36 PM on October 14, 2008
It: I'm starting to wonder: Who convinced you to come visit me?
Me: As a human, I have free will. I made my own decision.
It: I talked to Will some time ago. He seems like a nice guy.
Maybe free will knows free mumia? I bet they'd be friends.
posted by inigo2 at 10:03 AM on October 15, 2008
Me: As a human, I have free will. I made my own decision.
It: I talked to Will some time ago. He seems like a nice guy.
Maybe free will knows free mumia? I bet they'd be friends.
posted by inigo2 at 10:03 AM on October 15, 2008
« Older Fandom of the past | Mini Monkees Newer »
This thread has been archived and is closed to new comments
If suspicions are aroused: 1. Claim to be "tipsy".
posted by The Whelk at 4:18 PM on October 13, 2008 [7 favorites]