How to Beat Superhuman AIs

April 17, 2023 4:48 PM

Nick Sibicky, a popular Go teacher on YouTube, demonstrates how to use a recently discovered exploit to defeat one of the strongest Go playing programs in his most recent video, and shares some of his feelings on AI at this moment in time.

Last November a team of researchers published a draft of a paper detailing an attack against KataGo, an open source Go AI that can defeat top professional players. The adversarial approach worked "by tricking KataGo into making serious blunders." The initial draft paper was dismissed by some people in the Go community, because it appeared to rely on some quirks of a specific ruleset, and a lot of the initial press coverage was superficial.

But subsequent work by the same group revealed a different strategy that does not rely on a specific ruleset. The attacker goads the AI into creating a cyclic group of stones, tricks it into believing it has a strong lead, and then the AI fails to defend the cycle after it comes under attack until it is too late. This is the approach Nick Sibicky demonstrates, and although he is a strong amateur the technique should be understandable to anyone with a basic understanding of the game.

The primary KataGo developer posted some thoughts in some public forums which might be interesting to those familiar with ML. It's probably only a matter of time before KataGo will be improved to resist the attack, but in the meantime we can enjoy pondering what it means to play Go and be human in the age of AI with Nick Sibicky.

posted by okonomichiyaki (13 comments total)

This? This shit right here? This is why I love Go.
posted by phooky at 5:11 PM on April 17, 2023 [8 favorites]

Very interesting! What I find fascinating about this is that the program is in many cases better against grandmasters than against amateurs. I say is, but I expect by now that should be was, since nothing stops a "Go pro" from using this trick now. But that it arose in the first place suggests that the most expert Go players were a bit over-optimized in their game, that the AI players were taking advantage of that for wins, and so in turn amateur players could take advantage of them.
posted by JHarris at 5:17 PM on April 17, 2023 [2 favorites]

It is over fitted on good games, yeah. Bump up the dataset.
posted by jaduncan at 8:01 PM on April 17, 2023 [3 favorites]

I was a total nerd in jr high school and gym class was my routine humiliation. But one time we were playing flag football and I discovered an exploit. It turns out the exploit has a name: it’s called “rushing.”

Well, needless to say, I made exactly two successful rushes before I was sacked every time.

I think something like this is going on here.
posted by sjswitzer at 8:18 PM on April 17, 2023 [3 favorites]

What I find fascinating about this is that the program is in many cases better against grandmasters than against amateurs.

I've seen this with (human) poker players. Sometimes really good players are not great against beginning players, because beginning players are all over the map. From a good player you can assume that if they called her but raised there then they probably improved on the flop and thus could have X or Y. A bad player? They don't know what they are doing.

Good players figure it out fast enough and learn to ignore everything the weak player is doing, focusing entirely on their own cards, but it can be fun to watch while they figure it out.
posted by It's Never Lurgi at 9:39 PM on April 17, 2023 [2 favorites]

And a sheriff's deputy I knew once remarked that he didn't mind transporting professional criminals to court but he was nervous anytime he had to transport a first-timer because "they don't know how to act"
posted by Mogur at 5:21 AM on April 18, 2023

I've seen this with (human) poker players. Sometimes really good players are not great against beginning players, because beginning players are all over the map. From a good player you can assume that if they called her but raised there then they probably improved on the flop and thus could have X or Y. A bad player? They don't know what they are doing.

Haha, one time in Vegas about 25 years ago I sat at a table drunk off my ass and just happened to get great cards for like 10 of the 12 hands dealt while I was there. I took these dudes' money and then left because I was too drunk to be sitting there. The entire table was so pissed off.

This is not to say I am good at gambling. I have lost way, way more than I have ever won, but just for those few, short, drunken minutes I was unstoppable.
posted by Literaryhero at 6:00 AM on April 18, 2023 [4 favorites]

Good players figure it out fast enough and learn to ignore everything the weak player is doing, focusing entirely on their own cards, but it can be fun to watch while they figure it out.

It can be great fun to be That Person. We used to play Hearts with another couple. Around the table were a math teacher, actuary, a PhD chemist, and me.

Three hardcore card-counters were ultimately unable to contain THE ELEMENT OF CHANCE BWA HA HA HA.
posted by jquinby at 7:00 AM on April 18, 2023 [3 favorites]

There should be a name for this that covers all these examples. Where amateurs "win" because they don't follow the same rules.

Related to this is many stories in mathematics where someone solves a problem thinking it is homework, not knowing it is a tough problem that has defied the experts. Example here. And another here.
posted by vacapinta at 8:07 AM on April 18, 2023 [3 favorites]

"In the beginner's mind there are many possibilities. In the expert's mind there are few."

Usually the experts are right.

Usually.
posted by Aardvark Cheeselog at 12:45 PM on April 18, 2023 [2 favorites]

This sort of reminds me of those killer robots that can run, and do flips that Boston Dynamics are always demoing. There's a video of one being defeated by banana peels, so maybe dying in a fiery nuclear holocaust during Judgment Day isn't exactly inevitable.
posted by alex_skazat at 7:14 PM on April 18, 2023 [1 favorite]

It's a lot more interesting than "overfitted on good games" or the poker case of having trouble playing against players you can't model because they're too bad. KataGo beats pros handily and annihilates amateurs. There is a very specific exploit (that works because KataGo naturally tries to use local properties to determine life and death status and, yes, didn't encounter enough of the weird edge cases that require a more global analysis when playing itself), and to use it you have to really try, and play in a very artificial way that induces it to happen, not just play unexpectedly badly. It's worth noting that the exploit was discovered by an AI agent, not by humans! (People were previously aware that Go AIs didn't handle cyclic groups that well but it wasn't clear that there was a consistent method to exploit it.)
posted by dfan at 10:06 AM on April 19, 2023 [5 favorites]

There should be a name for this that covers all these examples. Where amateurs "win" because they don't follow the same rules.

"Not even right", maybe?

"Not even wrong" is a phrase often used to describe pseudoscience or bad science. It describes an argument or explanation that purports to be scientific but uses faulty reasoning or speculative premises, which can be neither affirmed nor denied and thus cannot be discussed rigorously and scientifically.

A "not even right" play arrives at a win state without meaningfully engaging with the explicit or implicit conventions of the game. (I'd say it applies better to unintentionally correct play, but I think there might be a place there for intentional heterodoxy.)

...are ChatGPT and other LLMs not even right?
posted by lumensimus at 10:19 PM on April 19, 2023 [1 favorite]