Sumplete
March 7, 2023 8:40 AM Subscribe
Sumplete is a math-logic web browser game where you delete numbers so the rows and columns in the grid correspond to the totals. Sumplete was apparently designed, coded and named by ChatGPT; more in Neowin and Gigazine.
Give the robot credit where it's due--'Sumplete' is a better name than 'Summer.'
posted by box at 8:52 AM on March 7, 2023 [8 favorites]
posted by box at 8:52 AM on March 7, 2023 [8 favorites]
A friend sent this to me yesterday. It is a fun game but I gave up when I saw the 9x9 puzzle. I can manage the 8x8s as long as there are a couple of gimme rows (like where the total is really low or high so that it eliminates a bunch of numbers) but once I have to deal with possibilities and think "well if X is true then what does that mean for the rest of the puzzle" the game becomes work and I lose interest. Sudoku is like that for me too.
posted by any portmanteau in a storm at 9:14 AM on March 7, 2023 [4 favorites]
posted by any portmanteau in a storm at 9:14 AM on March 7, 2023 [4 favorites]
Also, maybe I'm dumb but did it just give me one with two solutions?
Swap the blue and yellow squares and all the sums stay the same?
posted by flamewise at 9:26 AM on March 7, 2023 [2 favorites]
Swap the blue and yellow squares and all the sums stay the same?
posted by flamewise at 9:26 AM on March 7, 2023 [2 favorites]
designed, coded and named by ChatGPT
GPT is doing Last Starfighter testing with this, isn't it.
posted by Servo5678 at 9:35 AM on March 7, 2023 [6 favorites]
GPT is doing Last Starfighter testing with this, isn't it.
posted by Servo5678 at 9:35 AM on March 7, 2023 [6 favorites]
Also, maybe I'm dumb but did it just give me one with two solutions?
Yeah I've had a couple like that. Once that's all I'm left with then I'm confident enough to take the plunge and just pick a pair.
posted by any portmanteau in a storm at 9:37 AM on March 7, 2023
Yeah I've had a couple like that. Once that's all I'm left with then I'm confident enough to take the plunge and just pick a pair.
posted by any portmanteau in a storm at 9:37 AM on March 7, 2023
Holy shit.
I mean I guess when I read that this takes several hours of interaction I temper my enthusiasm somewhat. But supposedly the working prototype was practically instantaneous.
I understand at some level that this is like a collage of publicly available cliches, in game design, presentation, and code. I assume a competent human could do the same work in the same time or quicker. Heck, maybe even I could. But I wouldn't want to.
posted by grobstein at 9:56 AM on March 7, 2023 [2 favorites]
I mean I guess when I read that this takes several hours of interaction I temper my enthusiasm somewhat. But supposedly the working prototype was practically instantaneous.
I understand at some level that this is like a collage of publicly available cliches, in game design, presentation, and code. I assume a competent human could do the same work in the same time or quicker. Heck, maybe even I could. But I wouldn't want to.
posted by grobstein at 9:56 AM on March 7, 2023 [2 favorites]
The programming languages that will succeed are the ones that are easiest to use, regardless of code quality as long as it compiles.
posted by Going To Maine at 10:12 AM on March 7, 2023 [1 favorite]
posted by Going To Maine at 10:12 AM on March 7, 2023 [1 favorite]
"The hottest new programming language is English" - Andrej Karpathy
posted by gwint at 10:31 AM on March 7, 2023 [4 favorites]
posted by gwint at 10:31 AM on March 7, 2023 [4 favorites]
One of my hobbies is messing around with microcontrollers, and chatGPT will absolutely write working arduino programs for you. Tell it what pins everything is on and what you want it to do, and get instant results! The first try will probably not be exact (buttons aren't debounced, your puzzle game sometimes had two correct solutions), but those can be solved by testing and iteration.
When I tried to get it to program a neopixel ring, the answers it gave were about on par with code snippets I could find by googling, but they worked without me understanding what the code did and could be modified without me understanding what the code did.
I am certain that there are people using it right now to cheat on their computer science homework.
posted by surlyben at 10:41 AM on March 7, 2023 [3 favorites]
When I tried to get it to program a neopixel ring, the answers it gave were about on par with code snippets I could find by googling, but they worked without me understanding what the code did and could be modified without me understanding what the code did.
I am certain that there are people using it right now to cheat on their computer science homework.
posted by surlyben at 10:41 AM on March 7, 2023 [3 favorites]
To be clear, a person on the internet is claiming that ChatGPT created and coded this game. As proof he offered several screenshots.
It's already been shown that this is not, in fact, an original game.
Why are we all believing it was done by ChatGPT? I mean, maybe it was, but is this really proof of that?
posted by Frayed Knot at 10:41 AM on March 7, 2023 [7 favorites]
It's already been shown that this is not, in fact, an original game.
Why are we all believing it was done by ChatGPT? I mean, maybe it was, but is this really proof of that?
posted by Frayed Knot at 10:41 AM on March 7, 2023 [7 favorites]
Well I just wasted 30 minutes playing this so who's the fake intelligence now
posted by HeroZero at 10:55 AM on March 7, 2023 [10 favorites]
posted by HeroZero at 10:55 AM on March 7, 2023 [10 favorites]
Aren't the first couple steps (including generating the basic javascript) easily checkable just by attempting the same thing with ChatGPT yourself?
posted by col_pogo at 10:57 AM on March 7, 2023 [2 favorites]
posted by col_pogo at 10:57 AM on March 7, 2023 [2 favorites]
I am certain that there are people using it right now to cheat on their computer science homework
This is 100% correct and I, too, have been using it for periodic coding support, but I do think it’s worth remembering that students could have been using it to cheat for a while now by using GitHub Copilot. Like, I think ChatGPT is amazing, but also that some features are novel to all of us because we haven’t been forced to pay attention by a compelling narrative.
posted by Going To Maine at 11:19 AM on March 7, 2023 [2 favorites]
This is 100% correct and I, too, have been using it for periodic coding support, but I do think it’s worth remembering that students could have been using it to cheat for a while now by using GitHub Copilot. Like, I think ChatGPT is amazing, but also that some features are novel to all of us because we haven’t been forced to pay attention by a compelling narrative.
posted by Going To Maine at 11:19 AM on March 7, 2023 [2 favorites]
Aren't the first couple steps (including generating the basic javascript) easily checkable just by attempting the same thing with ChatGPT yourself?
Absolutely. Works if you want to break ChatGPT, works if you want to test a result.
posted by Going To Maine at 11:25 AM on March 7, 2023
Absolutely. Works if you want to break ChatGPT, works if you want to test a result.
posted by Going To Maine at 11:25 AM on March 7, 2023
Aren't the first couple steps (including generating the basic javascript) easily checkable just by attempting the same thing with ChatGPT yourself?
They are, and I did. The program I got out was buggy and unplayable.
In the blog post, Tait says he got a playable game in 30 seconds. I did not. He then says he spent "the next few hours" iterating with ChatGPT "adding new features and improving the design", none of which he posts to the blog. And this is where my doubt comes in.
If this is true, it's mind blowing. But if, as I strongly suspect, Tait spent that extra time debugging by telling ChatGPT what to fix, it's interesting. But it's pretty damn far away from "AI writes game all by itself."
posted by Frayed Knot at 11:27 AM on March 7, 2023 [7 favorites]
They are, and I did. The program I got out was buggy and unplayable.
In the blog post, Tait says he got a playable game in 30 seconds. I did not. He then says he spent "the next few hours" iterating with ChatGPT "adding new features and improving the design", none of which he posts to the blog. And this is where my doubt comes in.
If this is true, it's mind blowing. But if, as I strongly suspect, Tait spent that extra time debugging by telling ChatGPT what to fix, it's interesting. But it's pretty damn far away from "AI writes game all by itself."
posted by Frayed Knot at 11:27 AM on March 7, 2023 [7 favorites]
I asked it to come up with a Chess/Go-like game, and it kinda did, but the rules were a bit vague. I then asked it for the code, and it pointed me toward a Github repo that didn't exist.
posted by credulous at 11:29 AM on March 7, 2023 [8 favorites]
posted by credulous at 11:29 AM on March 7, 2023 [8 favorites]
I assume a competent human could do the same work in the same time or quicker.
I mean, a competent human could not create a working prototype in minutes, but I am also a little skeptical of that claim. There’s plenty of evidence that these things (Copilot and ChatGPT are branches off the same tree) can be good at self-contained coding tasks, in a way that a competent human can use to accelerate development significantly, however.
posted by atoxyl at 11:51 AM on March 7, 2023
I mean, a competent human could not create a working prototype in minutes, but I am also a little skeptical of that claim. There’s plenty of evidence that these things (Copilot and ChatGPT are branches off the same tree) can be good at self-contained coding tasks, in a way that a competent human can use to accelerate development significantly, however.
posted by atoxyl at 11:51 AM on March 7, 2023
Dumb question but doesn’t ChatGPT suck at arithmetic? I admit I have not tried it, but at first blush it seems less like a game and more like a recipe for winding up in an argument with a language model over the value of 2+7.
posted by eirias at 12:08 PM on March 7, 2023 [1 favorite]
posted by eirias at 12:08 PM on March 7, 2023 [1 favorite]
ChatGPT doesn't have any semantic understanding of what it is saying. So it has no way to know that '2 + 3 = 8' is nonsensiscal.
So, yes, it sucks at math. It doesn't even know what math is!
posted by Frayed Knot at 12:20 PM on March 7, 2023
So, yes, it sucks at math. It doesn't even know what math is!
posted by Frayed Knot at 12:20 PM on March 7, 2023
I admit I have not tried it, but at first blush it seems less like a game and more like a recipe for winding up in an argument with a language model over the value of 2+7.
ChatGPT is terrible at specific things, but decent at generating things that take parameters. Try asking it to count how many words are in a certain sentence, and you may get some hilarious results. (I have, and did.) Ask it to make a program that does something with “numbers” as a general concept, and you might get something that works just fine because it’s generating code that looks like it should work and that it has seen people reference. It might confabulate a library, but it often doesn’t.
posted by Going To Maine at 12:55 PM on March 7, 2023
ChatGPT is terrible at specific things, but decent at generating things that take parameters. Try asking it to count how many words are in a certain sentence, and you may get some hilarious results. (I have, and did.) Ask it to make a program that does something with “numbers” as a general concept, and you might get something that works just fine because it’s generating code that looks like it should work and that it has seen people reference. It might confabulate a library, but it often doesn’t.
posted by Going To Maine at 12:55 PM on March 7, 2023
Try asking it to count how many words are in a certain sentence, and you may get some hilarious results.
"Because I was afraid of worms, Roxanne!"
posted by Etrigan at 1:37 PM on March 7, 2023 [2 favorites]
"Because I was afraid of worms, Roxanne!"
posted by Etrigan at 1:37 PM on March 7, 2023 [2 favorites]
I used it for work, asking for a javasript function that would take a starting cell of an excel sheet as an input, the total number of columns I wanted, and give me a range of the desired width.
It took about 7 tries, mostly me telling it "No, using your code I get A1:A%1. It should be "A1:MT1", then it would say it was sorry and make another mistake, etc, until it got it right.
The code it generated is in production now.
posted by signal at 1:43 PM on March 7, 2023 [4 favorites]
It took about 7 tries, mostly me telling it "No, using your code I get A1:A%1. It should be "A1:MT1", then it would say it was sorry and make another mistake, etc, until it got it right.
The code it generated is in production now.
posted by signal at 1:43 PM on March 7, 2023 [4 favorites]
That sounds a lot like how I can teach my dog to do a trick by filming her not doing it over and over again until she does something that resembles what I was asking, and then Instagramming that. Such a good girl!
posted by grahamparks at 2:30 PM on March 7, 2023 [2 favorites]
posted by grahamparks at 2:30 PM on March 7, 2023 [2 favorites]
Dumb question but doesn’t ChatGPT suck at arithmetic? I admit I have not tried it, but at first blush it seems less like a game and more like a recipe for winding up in an argument with a language model over the value of 2+7.
Sometimes. I think they are bolting on services for it. It may not encode sqrt(50176)/(2**3) = 28 anywhere in the transformer, but it knows that it can python3 -c "import math;print(math.sqrt(50176)/(2**3))" to get the answer. Same for how it gets today's date. When I played with it, it was not very good at some problems requiring multiple steps with units (and inverting some ratios where necessary) without some careful edits to the prompt. It could also be that the internet at large's modal discussion botches unit analysis, and I need to tell it to pretend to be a best answerer before giving it those questions.
posted by a robot made out of meat at 2:32 PM on March 7, 2023 [1 favorite]
Sometimes. I think they are bolting on services for it. It may not encode sqrt(50176)/(2**3) = 28 anywhere in the transformer, but it knows that it can python3 -c "import math;print(math.sqrt(50176)/(2**3))" to get the answer. Same for how it gets today's date. When I played with it, it was not very good at some problems requiring multiple steps with units (and inverting some ratios where necessary) without some careful edits to the prompt. It could also be that the internet at large's modal discussion botches unit analysis, and I need to tell it to pretend to be a best answerer before giving it those questions.
posted by a robot made out of meat at 2:32 PM on March 7, 2023 [1 favorite]
They are, and I did. The program I got out was buggy and unplayable.
I also just tried this and it bailed halfway through the javascript part, so hard to even say. From some googling, it seems to have a character limit on code output. Possibly you could get around with some prompt engineering - or maybe the paid version doesn't have this limit? But anyways, the sumplete code for the OP link would have exceeded the char limit I'm hitting for sure. Makes it all a bit hard to replicate.
I was also playing around a bit with prompts like "can you invent a logic game similar to chess that doesn't currently exist?" It gave me a fun but quite confused sounding game called Chrono Knight, that had four pieces; but one of the various ways in which the generated rules were confused was that they describe only one piece being on the board (the titular chrono knight: "The game begins with both Chrono Knights in the center of the board."). I then asked, "ok, but why are there 4 types of pieces when only one on the board?", and it gave a long response about how it had made a mistake, "...The Squire and the Archer were mistakenly included in my previous description of the game, and I apologize for any confusion that may have caused. The game only includes two types of pieces: the Chrono Knight and the Foot Soldier....". I then asked, "but why isn't the foot soldier on the board?", and it responded with some detail, but, "I apologize for any confusion. The Foot Soldier is indeed on the board, as I previously mentioned." (It had not mentioned, though it was happy to gaslight some new rules into existence following that.) I then asked it to produce some code, which it started to and was vaguely promising, but unsurprisingly it once again hit the word limit on code output.
This was all actually pretty entertaining but I'm no less skeptical about the practical usefulness of chatgpt than when I started out, and I'm now even more skeptical about the reliability of the sumplete "about" narrative, which is saying something.
posted by advil at 2:42 PM on March 7, 2023 [5 favorites]
I also just tried this and it bailed halfway through the javascript part, so hard to even say. From some googling, it seems to have a character limit on code output. Possibly you could get around with some prompt engineering - or maybe the paid version doesn't have this limit? But anyways, the sumplete code for the OP link would have exceeded the char limit I'm hitting for sure. Makes it all a bit hard to replicate.
I was also playing around a bit with prompts like "can you invent a logic game similar to chess that doesn't currently exist?" It gave me a fun but quite confused sounding game called Chrono Knight, that had four pieces; but one of the various ways in which the generated rules were confused was that they describe only one piece being on the board (the titular chrono knight: "The game begins with both Chrono Knights in the center of the board."). I then asked, "ok, but why are there 4 types of pieces when only one on the board?", and it gave a long response about how it had made a mistake, "...The Squire and the Archer were mistakenly included in my previous description of the game, and I apologize for any confusion that may have caused. The game only includes two types of pieces: the Chrono Knight and the Foot Soldier....". I then asked, "but why isn't the foot soldier on the board?", and it responded with some detail, but, "I apologize for any confusion. The Foot Soldier is indeed on the board, as I previously mentioned." (It had not mentioned, though it was happy to gaslight some new rules into existence following that.) I then asked it to produce some code, which it started to and was vaguely promising, but unsurprisingly it once again hit the word limit on code output.
This was all actually pretty entertaining but I'm no less skeptical about the practical usefulness of chatgpt than when I started out, and I'm now even more skeptical about the reliability of the sumplete "about" narrative, which is saying something.
posted by advil at 2:42 PM on March 7, 2023 [5 favorites]
ChatGPT doesn't have any semantic understanding of what it is saying.
Regardless of… whatever it would even mean for it to “understand” things, its model clearly effectively encodes fairly sophisticated semantic relationships between words. It’s pretty decent at answering analogy or antonym questions, for example. But indeed, it’s pretty shaky on math.
posted by atoxyl at 6:04 PM on March 7, 2023 [2 favorites]
Regardless of… whatever it would even mean for it to “understand” things, its model clearly effectively encodes fairly sophisticated semantic relationships between words. It’s pretty decent at answering analogy or antonym questions, for example. But indeed, it’s pretty shaky on math.
posted by atoxyl at 6:04 PM on March 7, 2023 [2 favorites]
Why are we all believing it was done by ChatGPT? I mean, maybe it was, but is this really proof of that?
ChatGPT helped write a basketball physics simulation using Three.js and Cannon.js in less than half an hour. I had to paste things together and make it coherent, but it did the vast majority of the work.
posted by ryoshu at 7:11 PM on March 7, 2023 [1 favorite]
ChatGPT helped write a basketball physics simulation using Three.js and Cannon.js in less than half an hour. I had to paste things together and make it coherent, but it did the vast majority of the work.
posted by ryoshu at 7:11 PM on March 7, 2023 [1 favorite]
ChatGPT is surprisingly good at coding, but the quality of output depends heavily on quality of your input (like everything). If you have a workable overall design, you can describe functions to it and it will spit them out fairly reliably. If you don't give it clear instructions it will not be very effective. If it doesn't work and you give it the error message it can generally fix it. I strongly recommend playing with it. It's also pretty solid at explaining code, and improving code with poor syntax and organization. Same with giving functions and variables better names. Amazing stuff.
posted by lookoutbelow at 10:39 PM on March 7, 2023 [3 favorites]
posted by lookoutbelow at 10:39 PM on March 7, 2023 [3 favorites]
My experiments with ChatGPT have led me to conclude that it is sometimes surprisingly good at producing code IF the problem matches a known one, such as an existing game or well known interview question. It fails spectacularly and will even get somewhat argumentative if tasked to produce anything novel. It is so bad at fairly basic math that I can't understand how anybody could trust it to "write" a program of any complexity.
I like the game... but I'm suspicious the "ChatGPT wrote it" claim could just be cover for "I ripped off this guy's idea. (Be sure to hit Subscribe!)"
posted by dsword at 11:09 PM on March 7, 2023 [4 favorites]
I like the game... but I'm suspicious the "ChatGPT wrote it" claim could just be cover for "I ripped off this guy's idea. (Be sure to hit Subscribe!)"
posted by dsword at 11:09 PM on March 7, 2023 [4 favorites]
Can any of you who have been producing working code explain how to deal with the character limit?
posted by advil at 6:23 AM on March 8, 2023 [1 favorite]
posted by advil at 6:23 AM on March 8, 2023 [1 favorite]
I have a short story I want to submit to a magazine, but it's a bit too short. I asked ChatGPT to extend it to twice its length. It continued the story way past its ending. Fair enough.
I added the specification that it should not change the beggining or ending, just extend it in the middle. It added parts explaining everything that went on. I asked it not to. It moved the final twist that makes the whole story work to the middle of the story. I asked it not to. It repeated just the original text. I scolded it.
It gave me the entire text of Little Red Riding Hood.
I said, "that's Little Red Riding Hood".
It said "sorry," and gave me the Three Little Pigs.
So yes, it's possible to piss an AI off.
posted by signal at 7:53 AM on March 8, 2023 [6 favorites]
I added the specification that it should not change the beggining or ending, just extend it in the middle. It added parts explaining everything that went on. I asked it not to. It moved the final twist that makes the whole story work to the middle of the story. I asked it not to. It repeated just the original text. I scolded it.
It gave me the entire text of Little Red Riding Hood.
I said, "that's Little Red Riding Hood".
It said "sorry," and gave me the Three Little Pigs.
So yes, it's possible to piss an AI off.
posted by signal at 7:53 AM on March 8, 2023 [6 favorites]
It should have given you Goldilocks.
posted by aniola at 10:36 PM on March 8, 2023 [1 favorite]
posted by aniola at 10:36 PM on March 8, 2023 [1 favorite]
« Older The Academy is building a bunker that could... | Frozen choo-choo: Snow (1963) Newer »
This thread has been archived and is closed to new comments
(h/t Adrian Hon)
posted by Going To Maine at 8:49 AM on March 7, 2023 [11 favorites]