A Quick Puzzle to Test Your Problem Solving
July 2, 2015 8:00 AM Subscribe
A short game sheds light on government policy, corporate America and why no one likes to be wrong. [SLNYT]
I went three examples without a no, and then guessed wrong. Womp womp.
posted by roomthreeseventeen at 8:09 AM on July 2, 2015
posted by roomthreeseventeen at 8:09 AM on July 2, 2015
I'm in the 9%. Quick, someone put me in charge of something.
posted by BlueJae at 8:12 AM on July 2, 2015 [10 favorites]
posted by BlueJae at 8:12 AM on July 2, 2015 [10 favorites]
Yeah, I'm thinking that this is more about pump priming than confirmation bias. The fact that they started with powers of 2 did not help.
posted by NoxAeternum at 8:12 AM on July 2, 2015 [12 favorites]
posted by NoxAeternum at 8:12 AM on July 2, 2015 [12 favorites]
(Is this why all my bosses ever complained about my negativity?)
posted by BlueJae at 8:13 AM on July 2, 2015 [2 favorites]
posted by BlueJae at 8:13 AM on July 2, 2015 [2 favorites]
Start big, narrow down. Although I don't know if it actually tried to understand my answer, as monotonically increasing sequence is something that I don't expect the NYT to know. (Although it is the most accurate answer I can think of.)
On the other hand, I saw a video with this a while ago. Finding incorrect sequences is always the best way to determine these things. I don't think that lesson applies quite as broadly as they'd like to suggest. (Let's make a car that no one wants to buy! Ok, they don't like that kind, lets find another they don't like!)
posted by Hactar at 8:13 AM on July 2, 2015 [1 favorite]
On the other hand, I saw a video with this a while ago. Finding incorrect sequences is always the best way to determine these things. I don't think that lesson applies quite as broadly as they'd like to suggest. (Let's make a car that no one wants to buy! Ok, they don't like that kind, lets find another they don't like!)
posted by Hactar at 8:13 AM on July 2, 2015 [1 favorite]
I guessed two examples and got "yes" both times. Then, I typed my description of the rule, clicked on the "I think I know" button and...nothing happened. No response.
Then, I clicked on the "I don't want to play; just tell me the answer." link and...nothing happened.
Then, I reloaded the page and re-entered my examples, and now got "no" on them.
wtf?
posted by Thorzdad at 8:14 AM on July 2, 2015 [3 favorites]
Then, I clicked on the "I don't want to play; just tell me the answer." link and...nothing happened.
Then, I reloaded the page and re-entered my examples, and now got "no" on them.
wtf?
posted by Thorzdad at 8:14 AM on July 2, 2015 [3 favorites]
It got me -- I thought I was being all smart when I DID test my original theory (3rd number is product of 1st and second) and found a counter example, which just led me to complicate my theory more (the second number is larger than the first and the third number is the product of the first and second).
posted by sparklemotion at 8:14 AM on July 2, 2015
posted by sparklemotion at 8:14 AM on July 2, 2015
If you work in a profession or come from a background where you have been trained to try to break stuff (for example, computer programming or science), I think you're more likely to seek out the negatives because they're generative of new information.
posted by overeducated_alligator at 8:15 AM on July 2, 2015 [34 favorites]
posted by overeducated_alligator at 8:15 AM on July 2, 2015 [34 favorites]
The problem presented doesn't actually do a great job of demonstrating the point of the article.
I didn't get a no and also did not describe the rule correctly, but all that says is that my assumption was one of several potentially correct ones based on the available evidence. But many of the examples the article then cites when talking about confirmation bias are not ones where the available evidence was leading to a wrong conclusion, but where the available evidence was/is ignored.
What the article should be saying is that one has to explore all possible conclusions to be derived from the evidence, and be able to disprove other conclusions before deciding what is actual true (or most plausible).
posted by dry white toast at 8:16 AM on July 2, 2015 [5 favorites]
I didn't get a no and also did not describe the rule correctly, but all that says is that my assumption was one of several potentially correct ones based on the available evidence. But many of the examples the article then cites when talking about confirmation bias are not ones where the available evidence was leading to a wrong conclusion, but where the available evidence was/is ignored.
What the article should be saying is that one has to explore all possible conclusions to be derived from the evidence, and be able to disprove other conclusions before deciding what is actual true (or most plausible).
posted by dry white toast at 8:16 AM on July 2, 2015 [5 favorites]
I didn't even test my own rule properly - if I was going to assume it was xn I should have tested for 1 or 0.
You didn't run the right controls, maryr. Bad scientist. Bad!
posted by maryr at 8:16 AM on July 2, 2015 [6 favorites]
You didn't run the right controls, maryr. Bad scientist. Bad!
posted by maryr at 8:16 AM on July 2, 2015 [6 favorites]
I guess I didn't get 3 Nos, but I did get 2 and successfully guessed what was going on, so I still feel like I've been given license to be smug about this.
posted by Copronymus at 8:17 AM on July 2, 2015 [8 favorites]
posted by Copronymus at 8:17 AM on July 2, 2015 [8 favorites]
Yeah, I had that philosophy too, overeducated_alligator. (I had five no's and five yes's before I put in my (correct) description).
But.. then I asked the friend on Facebook who posted it, who like me has a PhD in theoretical physics... and I had to kill that hypothesis. (He only had yes's).
posted by nat at 8:17 AM on July 2, 2015
But.. then I asked the friend on Facebook who posted it, who like me has a PhD in theoretical physics... and I had to kill that hypothesis. (He only had yes's).
posted by nat at 8:17 AM on July 2, 2015
I wonder if this was phrased as "win $10 if you get the right answer" would people be a little more eager to get it right.
posted by Skorgu at 8:18 AM on July 2, 2015 [1 favorite]
posted by Skorgu at 8:18 AM on July 2, 2015 [1 favorite]
I got it. I tried the obvious stuff first, throwing in a few complications of doubling each successive number, and then thought that the trick might be that it just accepts everything, so I threw in something that would HAVE to be wrong if there was any true sequence (1, 1, 1) and it said no. I then just did jumbles on the keyboard a few times, one of which was (8, 589, 212) and that gave it a no. I tried a few jumbles where I increased the place values each time (like 1, 10, 100) and they were all yes. At that point I figured that a simple increase was probably what they were looking for, and went with that.
On preview, what alligator said. I've done programming and technical troubleshooting for a long time professionally in various capacities, and a major aspect of that is trying stuff and seeing if it works, or if it changes the state of the system at all.
posted by codacorolla at 8:19 AM on July 2, 2015 [5 favorites]
On preview, what alligator said. I've done programming and technical troubleshooting for a long time professionally in various capacities, and a major aspect of that is trying stuff and seeing if it works, or if it changes the state of the system at all.
posted by codacorolla at 8:19 AM on July 2, 2015 [5 favorites]
I had the same problem as Thorzdad, because the security controls on my work computer won't let javascript run. What is the explanation?
posted by suelac at 8:19 AM on July 2, 2015
posted by suelac at 8:19 AM on July 2, 2015
I definitely noticed my problem solving strategy kick in. Test for the obvious yes, test for another simple case. Now, look for a no. Ooops, that was a yes as well. Try something else for a no. Got it. Confirm with another no. Yup. Now, test a corner case that should still be a yes. Okay, found my theory.
For people who enjoy this and want to try it in game form I highly recommend Zendo.
posted by meinvt at 8:19 AM on July 2, 2015 [6 favorites]
For people who enjoy this and want to try it in game form I highly recommend Zendo.
posted by meinvt at 8:19 AM on July 2, 2015 [6 favorites]
Also I don't think "monotonically increasing" is correct. Isn't a constant function technically monotonic? You want "strictly increasing".
posted by nat at 8:20 AM on July 2, 2015 [3 favorites]
posted by nat at 8:20 AM on July 2, 2015 [3 favorites]
I guess I didn't get 3 Nos, but I did get 2 and successfully guessed what was going on, so I still feel like I've been given license to be smug about this.
posted by Copronymus at 11:17 AM on July 2 [+] [!]
Yeah but you'd be smug anyway.
posted by Mrs. Pterodactyl at 8:21 AM on July 2, 2015 [1 favorite]
posted by Copronymus at 11:17 AM on July 2 [+] [!]
Yeah but you'd be smug anyway.
posted by Mrs. Pterodactyl at 8:21 AM on July 2, 2015 [1 favorite]
Evidence: Every time I test to see if you are smug, the answer comes back yes. Problem solving win!
posted by Mrs. Pterodactyl at 8:21 AM on July 2, 2015 [6 favorites]
posted by Mrs. Pterodactyl at 8:21 AM on July 2, 2015 [6 favorites]
I wonder if this was phrased as "win $10 if you get the right answer" would people be a little more eager to get it right.
They need stakes. Without a reason to solve the puzzle, the vast majority are just going to try it a few times, and make a guess. Any stat gleaned from this exercise should probably take that into account.
I, of course, tested a number of answers and guessed correctly, so I obviously know what I'm talking about.
posted by graventy at 8:22 AM on July 2, 2015
They need stakes. Without a reason to solve the puzzle, the vast majority are just going to try it a few times, and make a guess. Any stat gleaned from this exercise should probably take that into account.
I, of course, tested a number of answers and guessed correctly, so I obviously know what I'm talking about.
posted by graventy at 8:22 AM on July 2, 2015
> Finding incorrect sequences is always the best way to determine these things
Isn't the article pointing out that this isn't always true? If it's easy to generate and test new hypotheses then it is the "best", but put this in a situation where there's a big reward for hypotheses that get a "Yes" and big penalties for ones that get "No" and you have to start asking whether your goal is better understanding or a higher score.
posted by benito.strauss at 8:23 AM on July 2, 2015 [1 favorite]
Isn't the article pointing out that this isn't always true? If it's easy to generate and test new hypotheses then it is the "best", but put this in a situation where there's a big reward for hypotheses that get a "Yes" and big penalties for ones that get "No" and you have to start asking whether your goal is better understanding or a higher score.
posted by benito.strauss at 8:23 AM on July 2, 2015 [1 favorite]
When I get the "right" answer to something like this, but I'm still pretty sure I'm an idiot, I just get really sad about everything.
posted by wakannai at 8:23 AM on July 2, 2015 [2 favorites]
posted by wakannai at 8:23 AM on July 2, 2015 [2 favorites]
Also, dry white toast, I actually think it isn't a question of available evidence-- it's a question of seeking out further evidence. Part of confirmation bias is not bothering to seek out evidence which might contradict you. (I expect in principle there's an infinite amount of available evidence here.. you could check for any numerical input. You didn't, just like I don't go out of my way to read Free Republic.)
posted by nat at 8:23 AM on July 2, 2015 [3 favorites]
posted by nat at 8:23 AM on July 2, 2015 [3 favorites]
This is why successful people always give advice along the lines of "don't be afraid of failure," right?
posted by infinitewindow at 8:24 AM on July 2, 2015
posted by infinitewindow at 8:24 AM on July 2, 2015
(I am totally leaving myself open for someone to tell me "no" with that comment)
posted by infinitewindow at 8:25 AM on July 2, 2015 [2 favorites]
posted by infinitewindow at 8:25 AM on July 2, 2015 [2 favorites]
right benito.strauss, externalities can definitely pollute the process. The most information is generated not from a yes or a no, but from a test where you have a clearly expected hypothesis of getting one of those answers and the actual answer is the opposite. In any other situation you have only learned that "what I thought was true before may still be true", while in the differing response situation you heave learned "what I thought before was definitely not true". The absolute is more informative than the possible.
I agree that the article isn't exactly clear on the psychology of this. Because, you aren't only looking for yes instead of no, you also have to actively look for situations where what you think might be true is most likely to be proven wrong.
posted by meinvt at 8:26 AM on July 2, 2015 [1 favorite]
I agree that the article isn't exactly clear on the psychology of this. Because, you aren't only looking for yes instead of no, you also have to actively look for situations where what you think might be true is most likely to be proven wrong.
posted by meinvt at 8:26 AM on July 2, 2015 [1 favorite]
The academic psychologist who first researched this was Peter Wason. He coined the phrase "confirmation bias."
posted by obscure simpsons reference at 8:26 AM on July 2, 2015 [3 favorites]
posted by obscure simpsons reference at 8:26 AM on July 2, 2015 [3 favorites]
I got it right only because I've encountered the Wason experiment before.
posted by ogooglebar at 8:28 AM on July 2, 2015 [1 favorite]
posted by ogooglebar at 8:28 AM on July 2, 2015 [1 favorite]
Aw crap, I thought there would be more than one question so I rushed through the first one thinking it would be the easy one, and didn't consider I should give it a whole lot of thought. Aren't most people who've had any kind of academic experience in their background socialized to approach these kinds of numeric sequence problems in a particular way? I'm not sure it's really fair or valid to generalize all the way to people's broader attitudes and general approaches to problem-solving from a single trick question...
posted by saulgoodman at 8:28 AM on July 2, 2015 [4 favorites]
posted by saulgoodman at 8:28 AM on July 2, 2015 [4 favorites]
Yeah, they got me. I assumed it was doubling numbers, got three confirmations. Didn't test to see if I could produce a no by entering a wrong value. I wonder to what extent that's laziness (I got it right; why try to get it wrong on purpose) and to what extent it's not wanting to hear "no."
As much as I'd like to lay this one on psychology, I'm betting it's laziness.
posted by Mooski at 8:31 AM on July 2, 2015 [2 favorites]
As much as I'd like to lay this one on psychology, I'm betting it's laziness.
posted by Mooski at 8:31 AM on July 2, 2015 [2 favorites]
If you work in a profession or come from a background where you have been trained to try to break stuff (for example, computer programming or science), I think you're more likely to seek out the negatives because they're generative of new information.
I remember hearing a quote from somebody to the effect of, "If your [software] testing didn't find any bugs, your test is bad."
I ran twelve cases and received five nos. I did first fall in to the trap that they laid (each number is 2x more than the previous), but the second test disproved that. I even tried non-integers, negative numbers, and zero.
posted by backseatpilot at 8:32 AM on July 2, 2015 [2 favorites]
I remember hearing a quote from somebody to the effect of, "If your [software] testing didn't find any bugs, your test is bad."
I ran twelve cases and received five nos. I did first fall in to the trap that they laid (each number is 2x more than the previous), but the second test disproved that. I even tried non-integers, negative numbers, and zero.
posted by backseatpilot at 8:32 AM on July 2, 2015 [2 favorites]
I was frustrated by not getting a "No", and I generated four "No"s and two more "Yes"s before I answered. And now *I* feel smug too!! Hurray for being a good scientist!
This confirms my theory that we should have a technocracy instead of a democracy and that those educated in investigation and critical reasoning are just inherently superior as leaders <HAMBURGER>
posted by Made of Star Stuff at 8:33 AM on July 2, 2015
This confirms my theory that we should have a technocracy instead of a democracy and that those educated in investigation and critical reasoning are just inherently superior as leaders <HAMBURGER>
posted by Made of Star Stuff at 8:33 AM on July 2, 2015
I'm curious how they're parsing the answers people give.
posted by LobsterMitten at 8:43 AM on July 2, 2015 [2 favorites]
posted by LobsterMitten at 8:43 AM on July 2, 2015 [2 favorites]
I guessed incorrectly on purpose to see what color the no graphic was.
posted by the webmistress at 8:45 AM on July 2, 2015 [2 favorites]
posted by the webmistress at 8:45 AM on July 2, 2015 [2 favorites]
I hope I'm not the only one who thought that the given sequence was just an example/warm-up and consequently didn't spend too much time thinking about it before submitting a guess, operating under the assumption that there'd be a button at the bottom of the page labeled "Take the Quiz".
Cuz it'd be pretty embarrassing if I'm the only one.
posted by lord_wolf at 8:46 AM on July 2, 2015 [8 favorites]
Cuz it'd be pretty embarrassing if I'm the only one.
posted by lord_wolf at 8:46 AM on July 2, 2015 [8 favorites]
Yeah, I was like 'This is the easiest Zendo puzzle I've seen yet!'
posted by corb at 8:49 AM on July 2, 2015
posted by corb at 8:49 AM on July 2, 2015
Reading "Harry Potter and the Methods of Rationality" is what primed me to get this right. Thanks, fanfic!
posted by Solon and Thanks at 8:49 AM on July 2, 2015 [8 favorites]
posted by Solon and Thanks at 8:49 AM on July 2, 2015 [8 favorites]
I thought "Aha! One of those experiments where the spelling of the number is key!"
So: one, nine, seven. No.
One, four, seven. Yes. I was right! The rule is: Each number has to be higher AND have one additional letter.
Oh.
posted by kozad at 8:51 AM on July 2, 2015
So: one, nine, seven. No.
One, four, seven. Yes. I was right! The rule is: Each number has to be higher AND have one additional letter.
Oh.
posted by kozad at 8:51 AM on July 2, 2015
If you work in a profession or come from a background where you have been trained to try to break stuff (for example, computer programming or science), I think you're more likely to seek out the negatives because they're generative of new information.
There's an irony in this statement for me. Scientists aregood better than most at spotting confirmation bias, but the scientific community as a whole still suffers from the larger problem that is being discussed here: We all want the positive result. Careers are made on having a positive result, which may explain why so many positive results turn out to be irreproducible.
It is difficult to look at things as they are when it's not in your DNA to do so. Our pattern recognition capabilities are so good that we see things even when they're not there. And we all want to bask in the adulation that comes with delivering something positive. Cynics and problem solvers might be good at pointing out the flaws, but such acts can be socially jarring, and nobody wants to be the negative asshole all the time. This all conspires to make the confirmation bias all the more ubiquitous in society.
posted by kisch mokusch at 8:52 AM on July 2, 2015 [3 favorites]
There's an irony in this statement for me. Scientists are
It is difficult to look at things as they are when it's not in your DNA to do so. Our pattern recognition capabilities are so good that we see things even when they're not there. And we all want to bask in the adulation that comes with delivering something positive. Cynics and problem solvers might be good at pointing out the flaws, but such acts can be socially jarring, and nobody wants to be the negative asshole all the time. This all conspires to make the confirmation bias all the more ubiquitous in society.
posted by kisch mokusch at 8:52 AM on July 2, 2015 [3 favorites]
Count me in as one of those who, short on time as I have to go back to work soon, thought the first test was just a warm-up, and guessed "doubled." Didn't have time to work on more examples, so what does that say about (1) me, (2) the test, and (3) the NYT theories.
posted by marienbad at 9:00 AM on July 2, 2015
posted by marienbad at 9:00 AM on July 2, 2015
Actually we're all wrong, and so are they.
Their number checker has a finite ability to tell differences. It sees 1 and 1.00000000000000000000000000000000000001 as the same number and marks the sequence wrong.
So.. it's "increases by more than x" where x is some small but nonzero number that I don't have the time to find because I gotta do some real work.
posted by nat at 9:01 AM on July 2, 2015 [21 favorites]
Their number checker has a finite ability to tell differences. It sees 1 and 1.00000000000000000000000000000000000001 as the same number and marks the sequence wrong.
So.. it's "increases by more than x" where x is some small but nonzero number that I don't have the time to find because I gotta do some real work.
posted by nat at 9:01 AM on July 2, 2015 [21 favorites]
I found that almost ridiculously fascinating -- about as good an illustration of confirmation bias as I've ever seen. (Yep, I'll brag -- I got five "no" answers, and guessed the rule right after the fifth "no." I rendered my guess of the rule as "Three real numbers in ascending order, i.e., from lowest to highest." FWIW, I'm not a computer programmer or scientist.)
posted by holborne at 9:03 AM on July 2, 2015 [1 favorite]
posted by holborne at 9:03 AM on July 2, 2015 [1 favorite]
I guessed two examples and got "yes" both times. Then, I typed my description of the rule, clicked on the "I think I know" button and...nothing happened. No response.
Then, I clicked on the "I don't want to play; just tell me the answer." link and...nothing happened.
Then, I reloaded the page and re-entered my examples, and now got "no" on them.
To be fair to the FPP's phrasing, this does seem like a lot of my dealings with government policy and corporate America.
posted by ricochet biscuit at 9:04 AM on July 2, 2015 [9 favorites]
Then, I clicked on the "I don't want to play; just tell me the answer." link and...nothing happened.
Then, I reloaded the page and re-entered my examples, and now got "no" on them.
To be fair to the FPP's phrasing, this does seem like a lot of my dealings with government policy and corporate America.
posted by ricochet biscuit at 9:04 AM on July 2, 2015 [9 favorites]
Yay. I'm happy being told no.
If I ever have to conduct job interviews, I'll probably use this problem.
posted by amtho at 9:05 AM on July 2, 2015 [2 favorites]
If I ever have to conduct job interviews, I'll probably use this problem.
posted by amtho at 9:05 AM on July 2, 2015 [2 favorites]
So.. it's "increases by more than x" where x is some small but nonzero number that I don't have the time to find because I gotta do some real work.
Probably Number.EPSILON, which is approximately 2.22x10-16. The difference you used is much smaller than that.
posted by jedicus at 9:06 AM on July 2, 2015 [2 favorites]
Probably Number.EPSILON, which is approximately 2.22x10-16. The difference you used is much smaller than that.
posted by jedicus at 9:06 AM on July 2, 2015 [2 favorites]
amtho: "If I ever have to conduct job interviews, I'll probably use this problem."
But you should hire at least one person who gets the problem wrong, or you'll never know for certain if it's a good way to choose a hire.
posted by RobotHero at 9:09 AM on July 2, 2015 [22 favorites]
But you should hire at least one person who gets the problem wrong, or you'll never know for certain if it's a good way to choose a hire.
posted by RobotHero at 9:09 AM on July 2, 2015 [22 favorites]
Then, I typed my description of the rule, clicked on the "I think I know" button and...nothing happened. No response.
I got the same result when I played. Thought it was a browser issue, switched to IE, same result. Tried it again rephrasing my result from a wordy phrase like "The number on the left has to be smaller than the number in the middle and the number in the middle has to be smaller than the number on the right" to "Left < Middle < Right" and it worked. Maybe there's a character limit on the answer field?
posted by JDHarper at 9:16 AM on July 2, 2015 [1 favorite]
I got the same result when I played. Thought it was a browser issue, switched to IE, same result. Tried it again rephrasing my result from a wordy phrase like "The number on the left has to be smaller than the number in the middle and the number in the middle has to be smaller than the number on the right" to "Left < Middle < Right" and it worked. Maybe there's a character limit on the answer field?
posted by JDHarper at 9:16 AM on July 2, 2015 [1 favorite]
Of course, I recognized the problem so I was already expecting a certain rule, and in a way, both the "Yes" and "No" answers were confirming that I was right.
From a meta perspective, maybe I could have worked harder to prove my theory wrong, which wouldn't be a question of getting "no" answers, but getting answers that aren't predicted by my theory. Except I had no good alternate theories.
posted by RobotHero at 9:20 AM on July 2, 2015
From a meta perspective, maybe I could have worked harder to prove my theory wrong, which wouldn't be a question of getting "no" answers, but getting answers that aren't predicted by my theory. Except I had no good alternate theories.
posted by RobotHero at 9:20 AM on July 2, 2015
Confusion: how many petals on the rose?
posted by mikurski at 9:22 AM on July 2, 2015 [1 favorite]
posted by mikurski at 9:22 AM on July 2, 2015 [1 favorite]
This is the part where I get to feel smug, right?
I was kind of spoiled for this problem, actually, since it appeared a while ago in this chapter of Harry Potter and the Methods of Rationality, Harry explaining smugly to Hermione that she's experiencing confirmation bias and all.
posted by brecc at 9:25 AM on July 2, 2015 [1 favorite]
I was kind of spoiled for this problem, actually, since it appeared a while ago in this chapter of Harry Potter and the Methods of Rationality, Harry explaining smugly to Hermione that she's experiencing confirmation bias and all.
posted by brecc at 9:25 AM on July 2, 2015 [1 favorite]
Guessed it in one. Eliminate the real easy possibilities first, right?
posted by Sing Or Swim at 9:28 AM on July 2, 2015
posted by Sing Or Swim at 9:28 AM on July 2, 2015
As a computer programmer formerly specializing in debugging things that mystify other programmers, my first guess was '1,2,3'. My second guess suggested I was right to guess the rule it turned out they actually wanted. My next two guesses ruled out some other obvious possibilities. And then I thought for a moment about all the non-obvious possible rules that could also match my guesses, decided it was too much like work, and gave up.
posted by sfenders at 9:43 AM on July 2, 2015 [1 favorite]
posted by sfenders at 9:43 AM on July 2, 2015 [1 favorite]
I got the same result when I played.
Yeah, me too. Maybe too many people are playing at one time, or something? In any case, chalk me up as a non-science, non-tech person who was perfectly capable of realizing that I couldn't possibly deduce the rule without generating some "no" responses (otherwise the rule could simply be: "put something in each box"). I got the rule worked out in about 6 guesses.
posted by yoink at 9:47 AM on July 2, 2015 [3 favorites]
Yeah, me too. Maybe too many people are playing at one time, or something? In any case, chalk me up as a non-science, non-tech person who was perfectly capable of realizing that I couldn't possibly deduce the rule without generating some "no" responses (otherwise the rule could simply be: "put something in each box"). I got the rule worked out in about 6 guesses.
posted by yoink at 9:47 AM on July 2, 2015 [3 favorites]
From the results page: Most of us can quickly come up with other forms of confirmation bias — and yet the examples we prefer tend to be, themselves, examples of confirmation bias. If you’re politically liberal, maybe you’re thinking of the way that many conservatives ignore strong evidence of global warming and its consequences and instead glom onto weaker contrary evidence. Liberals are less likely to recall the many incorrect predictions over the decades, often strident and often from the left, that population growth would create widespread food shortages. It hasn’t.
Isn't this super-stretch to come to a cliched 'both sides do it' conclusion itself an example of strong confirmation bias?
posted by One Second Before Awakening at 9:54 AM on July 2, 2015 [4 favorites]
Isn't this super-stretch to come to a cliched 'both sides do it' conclusion itself an example of strong confirmation bias?
posted by One Second Before Awakening at 9:54 AM on July 2, 2015 [4 favorites]
I got it right, am in the 9% that tried a bunch of things to test my hypothesis, and also think the tie to confirmation bias is a little shaky.
posted by sweetkid at 9:58 AM on July 2, 2015 [1 favorite]
posted by sweetkid at 9:58 AM on July 2, 2015 [1 favorite]
I'm finally reading "The Black Swan", and just finished the chapter were this very test was cited. I'll probably always be a bias confirming son of a gun, but at least I passed an internet quiz in one try.
posted by klarck at 10:12 AM on July 2, 2015
posted by klarck at 10:12 AM on July 2, 2015
I probably would have gotten it wrong if I hadn't picked up the hints of "the answer is something obvious and you'll feel dumb for getting it wrong" from the framing of the problem.
posted by zixyer at 10:25 AM on July 2, 2015
posted by zixyer at 10:25 AM on July 2, 2015
Actually we're all wrong, and so are they.
Their number checker has a finite ability to tell differences. It sees 1 and 1.00000000000000000000000000000000000001 as the same number and marks the sequence wrong.
So.. it's "increases by more than x" where x is some small but nonzero number that I don't have the time to find because I gotta do some real work.
Yes, it doesn't work on numbers greater than about 1000000000000000000000 or less than about -1000000000000000000000, either.
So, in short, the rule seems to be that the sequence must be strictly increasing and also within a certain range defined by whatever programming language and data types they are using to determine the greater/lesser than relation.
Of course, there might be other constraints or exceptions depending on exact details. Floating point math implementations in both software and hardware have a fairly amazing amount of quirks. So it's hard to say without examining a *lot* more examples and/or knowing the details of the code & system involved what other exceptions or limitations there might be.
I hate to get all technical on them, but they DID ask the question and it was pretty easy to find examples that didn't follow their rule without even thinking too hard about it. The fact that you have verified something for whole numbers less than 1000 or even 1,000,000 hardly means that any given rule is truly verified.
posted by flug at 10:43 AM on July 2, 2015 [3 favorites]
Their number checker has a finite ability to tell differences. It sees 1 and 1.00000000000000000000000000000000000001 as the same number and marks the sequence wrong.
So.. it's "increases by more than x" where x is some small but nonzero number that I don't have the time to find because I gotta do some real work.
Yes, it doesn't work on numbers greater than about 1000000000000000000000 or less than about -1000000000000000000000, either.
So, in short, the rule seems to be that the sequence must be strictly increasing and also within a certain range defined by whatever programming language and data types they are using to determine the greater/lesser than relation.
Of course, there might be other constraints or exceptions depending on exact details. Floating point math implementations in both software and hardware have a fairly amazing amount of quirks. So it's hard to say without examining a *lot* more examples and/or knowing the details of the code & system involved what other exceptions or limitations there might be.
I hate to get all technical on them, but they DID ask the question and it was pretty easy to find examples that didn't follow their rule without even thinking too hard about it. The fact that you have verified something for whole numbers less than 1000 or even 1,000,000 hardly means that any given rule is truly verified.
posted by flug at 10:43 AM on July 2, 2015 [3 favorites]
The delta doesn't work the way I thought, either. The difference it can detect between one and 1+x appears to be infact x> Number.EPSILON/2.
But if you detect between 0 and 0+x, it can be smaller than Number.EPSILON/2.
And if you detect between 2 and 2+x, x has to be even bigger (I didn't check how much).
And thanks for the floating point link, interesting :-)
posted by nat at 10:51 AM on July 2, 2015
But if you detect between 0 and 0+x, it can be smaller than Number.EPSILON/2.
And if you detect between 2 and 2+x, x has to be even bigger (I didn't check how much).
And thanks for the floating point link, interesting :-)
posted by nat at 10:51 AM on July 2, 2015
Oh man, this is so timely. Lately, my SO and I have been having very heated arguments about controversial subjects and our method of dialogue has been so frustrating to me:
SO: All XYZs are bad.
Me: Well, actually that's not true. Some XYZs are bad, but I don't think it's fair to lump them all into one category.
SO: *insert "fact" that conflates correlation with causation or relies on some flimsy logical fallacy*
Me: Well that's not really true and here's evidence from reputable outside sources to suggest otherwise.
SO: *disregards evidence, ramps up XYZ attack*
This happened just last night and it's been driving me crazy because he's a very intelligent person, but I don't understand why he refuses to acknowledge data that could possibly contradict his world view. This article made sense to me and helped me understand why we view the world differently and why our hot-topic conversations end up on a downward spiral.
To me, it doesn't make sense to disregard or ignore information just because it doesn't support your proconceived notion. To him (I'm assuming), it doesn't make sense to intentionally seek out information that contradicts your world view if you already "know" it's right.
For the record, I guessed the rule correctly, but that's how I operate in my daily life. When I learn about something new and/or complex (racism, transgender issues, economic crises, whatevs), I'm constantly playing devil's advocate. Not because I want to be an annoying twat, but because I want to gain as much information as possible to support and contradict my belief. I love being right, but I also love being wrong because that gives me the opportunity to learn more.
posted by chara at 11:00 AM on July 2, 2015 [4 favorites]
SO: All XYZs are bad.
Me: Well, actually that's not true. Some XYZs are bad, but I don't think it's fair to lump them all into one category.
SO: *insert "fact" that conflates correlation with causation or relies on some flimsy logical fallacy*
Me: Well that's not really true and here's evidence from reputable outside sources to suggest otherwise.
SO: *disregards evidence, ramps up XYZ attack*
This happened just last night and it's been driving me crazy because he's a very intelligent person, but I don't understand why he refuses to acknowledge data that could possibly contradict his world view. This article made sense to me and helped me understand why we view the world differently and why our hot-topic conversations end up on a downward spiral.
To me, it doesn't make sense to disregard or ignore information just because it doesn't support your proconceived notion. To him (I'm assuming), it doesn't make sense to intentionally seek out information that contradicts your world view if you already "know" it's right.
For the record, I guessed the rule correctly, but that's how I operate in my daily life. When I learn about something new and/or complex (racism, transgender issues, economic crises, whatevs), I'm constantly playing devil's advocate. Not because I want to be an annoying twat, but because I want to gain as much information as possible to support and contradict my belief. I love being right, but I also love being wrong because that gives me the opportunity to learn more.
posted by chara at 11:00 AM on July 2, 2015 [4 favorites]
chara's SO is obviously a conservative homo sapiens.
posted by yoink at 11:12 AM on July 2, 2015 [3 favorites]
posted by yoink at 11:12 AM on July 2, 2015 [3 favorites]
[THIS COMMENT CONTAINS SPOILERS]
LobsterMitten: "I'm curious how they're parsing the answers people give."
I was wondering this, too! I thought it might be some neat natural-language processing, but it's literally just this Javascript function, checking for the presence of specific words:
This function is in "build.js", within the graphics8.nytimes.com folder.)
posted by omnomnOMINOUS at 11:45 AM on July 2, 2015 [12 favorites]
LobsterMitten: "I'm curious how they're parsing the answers people give."
I was wondering this, too! I thought it might be some neat natural-language processing, but it's literally just this Javascript function, checking for the presence of specific words:
function judgeSentence(sentence, numNo) {
sentence = sentence.toLowerCase();
// no nos -> wrong.
if (numNo === 0 || sentence == "") {
return false;
}
// if have any fancy words -> wrong.
var probablyWrong = ["doubl", "expon", "multipl", "^", "**", "power", "two", "2", "twice", "as big", "nth", "rais"];
if (hasAny(probablyWrong, sentence)) {
return false;
}
// if you have the right words, and no buts.
var seemsRight = ["larger", "increas", "greater", "small", "less", "big", ">", "<", "go up", "ascending"],
weaselWords = ['but ', 'not ', 'odd'];
if (hasAny(seemsRight, sentence) & !hasAny(weaselWords, sentence)) {
return true;
}
// // no nouns, verbs or adjectives in your sentence -> wrong.
// var s = nlp.pos(sentence).sentences[0],
// verbs = s.verbs().map(getWords),
// nouns = s.nouns().map(getWords),
// adj = s.adjectives().map(getWords),
// numWords = verbs.length + nouns.length + adj.length;
// if (numWords === 0) {
// return false;
// }
return false;
}
(You can find this by right-clicking anywhere on the page, clicking "Inspect Element", and going to the "Sources" tab.This function is in "build.js", within the graphics8.nytimes.com folder.)
posted by omnomnOMINOUS at 11:45 AM on July 2, 2015 [12 favorites]
This demonstrated to me something I already know about myself, which is that I'm really interested in failure states, as several people noted above. I'm totally fascinated by the marginal cases on the edges of rules where EVERYTHING GOES BATSHIT WONKY BIZARRE. People always get annoyed with me in management/governance positions because I always want to talk about the 1% of cases that our rule doesn't properly cover and what will happen then ... but I always turn out to look psychic when exactly that thing happens. (Because if I know one thing about life, it's that no matter how hard you try to idiot-proof the rules, someone will build a better idiot.) I don't have any particular gift in programming/numbers, but in rules as they apply to people, or logic games, I looooooooove finding the fail states.
(I got it right, and I tested 9 combinations and got 4 nos. I started testing evens, then odds, then repeating numbers (no); three same (no); mixed odds and evens (yes); triple-digit numbers (yes); "1,2,3" (yes); "3,2,1" (no). So my first instinct was even/oddness, which I had to test and then exclude; then sameness; then largeness; and only then did I think about ascending/descending. Throughout I was playing with different "spacing" between the numbers but quickly concluded it didn't matter as long as they weren't the same.)
Sort-of a variant form of fermi-pico-bagel, and I always liked that game.
posted by Eyebrows McGee at 12:03 PM on July 2, 2015 [4 favorites]
(I got it right, and I tested 9 combinations and got 4 nos. I started testing evens, then odds, then repeating numbers (no); three same (no); mixed odds and evens (yes); triple-digit numbers (yes); "1,2,3" (yes); "3,2,1" (no). So my first instinct was even/oddness, which I had to test and then exclude; then sameness; then largeness; and only then did I think about ascending/descending. Throughout I was playing with different "spacing" between the numbers but quickly concluded it didn't matter as long as they weren't the same.)
Sort-of a variant form of fermi-pico-bagel, and I always liked that game.
posted by Eyebrows McGee at 12:03 PM on July 2, 2015 [4 favorites]
I love when I get asked a question and then I'm told I'm wrong. It's like "okay goodbye"
"NO BUT READ WHY YOU WERE WRONG ITS V. IMPORTANT"
"nah"
posted by Cyclopsis Raptor at 12:13 PM on July 2, 2015 [1 favorite]
"NO BUT READ WHY YOU WERE WRONG ITS V. IMPORTANT"
"nah"
posted by Cyclopsis Raptor at 12:13 PM on July 2, 2015 [1 favorite]
Programmer here, also feeling smug. You can't really know how something works till you know how to break it.
posted by the_blizz at 12:16 PM on July 2, 2015 [1 favorite]
posted by the_blizz at 12:16 PM on July 2, 2015 [1 favorite]
Cyclopsis Raptor: "I love when I get asked a question and then I'm told I'm wrong. It's like "okay goodbye"
"NO BUT READ WHY YOU WERE WRONG ITS V. IMPORTANT"
"nah""
Well the question is, if they told you that you were right would you have read it?
posted by RobotHero at 12:17 PM on July 2, 2015 [1 favorite]
"NO BUT READ WHY YOU WERE WRONG ITS V. IMPORTANT"
"nah""
Well the question is, if they told you that you were right would you have read it?
posted by RobotHero at 12:17 PM on July 2, 2015 [1 favorite]
No.
posted by Cyclopsis Raptor at 12:24 PM on July 2, 2015
posted by Cyclopsis Raptor at 12:24 PM on July 2, 2015
He could not count the polar bears around the ice hole.
posted by mikurski at 12:35 PM on July 2, 2015
posted by mikurski at 12:35 PM on July 2, 2015
Also a quote from the article:
posted by RobotHero at 12:39 PM on July 2, 2015 [2 favorites]
One of the best-selling business books in history — about negotiation strategy — is “Getting to Yes.” But the more important advice for us may instead be to go out of our way to get to no.Except that negotiation has a completely different goal, so it's not really comparable, is it?
posted by RobotHero at 12:39 PM on July 2, 2015 [2 favorites]
I had used 0, .99999999999999999, 1 which breaks the intended pattern.
But the subtle thing is, the question is asked within a context, right? Given its an online webapp, it would be valid to interpret the question as asking about number patterns modulo all the artifacts of floating point arithmetic. By this interpretation, it's not asking for your answer to incorporate all the rounding and overflow behaviors of computer represented numbers. Because that's a different level of abstraction.
Reading communicative intent and context is also a political behavior. For better or worse.
posted by polymodus at 1:15 PM on July 2, 2015
But the subtle thing is, the question is asked within a context, right? Given its an online webapp, it would be valid to interpret the question as asking about number patterns modulo all the artifacts of floating point arithmetic. By this interpretation, it's not asking for your answer to incorporate all the rounding and overflow behaviors of computer represented numbers. Because that's a different level of abstraction.
Reading communicative intent and context is also a political behavior. For better or worse.
posted by polymodus at 1:15 PM on July 2, 2015
Ugh I just got linked to this and it only served to make me mad. I wish I could favorite Cyclopsis Raptor's comment a bajillion more times.
I can't get over thinking that it's stupid to simplify confirmation bias like this and then apply it to something with much more complexity.
"THIS PUZZLE IS WHY WE CAN'T HAVE NICE THINGS" --> No, actually. But nice try.
posted by erratic meatsack at 2:51 PM on July 2, 2015
I can't get over thinking that it's stupid to simplify confirmation bias like this and then apply it to something with much more complexity.
"THIS PUZZLE IS WHY WE CAN'T HAVE NICE THINGS" --> No, actually. But nice try.
posted by erratic meatsack at 2:51 PM on July 2, 2015
Got it in 10 guesses with 6 noes.
Then I spent a little time trying to figure out an answer that I was sure would match whatever pattern matcher they had and also be correct. I got that one, but of course, you only get one try there.
Not uninteresting.
> So.. it's "increases by more than x"
Not really. We all know there are edge cases here. Work with the problem poser!
posted by lupus_yonderboy at 3:21 PM on July 2, 2015
Then I spent a little time trying to figure out an answer that I was sure would match whatever pattern matcher they had and also be correct. I got that one, but of course, you only get one try there.
Not uninteresting.
> So.. it's "increases by more than x"
Not really. We all know there are edge cases here. Work with the problem poser!
posted by lupus_yonderboy at 3:21 PM on July 2, 2015
In the thread I linked earlier I did theorize that the choice of starting example can prime people to expect a more restrictive rule.
They start with the example of 2 4 8, which would also fit the more restrictive rule that you double the numbers. If you presented the same problem, but without the starting example, would you get fewer people going astray?
posted by RobotHero at 3:30 PM on July 2, 2015 [1 favorite]
They start with the example of 2 4 8, which would also fit the more restrictive rule that you double the numbers. If you presented the same problem, but without the starting example, would you get fewer people going astray?
posted by RobotHero at 3:30 PM on July 2, 2015 [1 favorite]
If you presented the same problem, but without the starting example, would you get fewer people going astray?
Yes, which is why they should probably give at least two starting examples, such as 2 4 8 and 1024 2048 4096. Fewer computer programmers and scientists would feel left out of the satisfying experience of being wrong like everyone else.
posted by sfenders at 4:20 PM on July 2, 2015 [1 favorite]
Yes, which is why they should probably give at least two starting examples, such as 2 4 8 and 1024 2048 4096. Fewer computer programmers and scientists would feel left out of the satisfying experience of being wrong like everyone else.
posted by sfenders at 4:20 PM on July 2, 2015 [1 favorite]
I got it right, and I gave up on the article because I can't be arsed with someone going "OK, that is correct, but people often give the wrong answer and that's a problem!"
Fine. Go lecture them. Except, they apparently didn't read your article either. Hey, I have a simple rule that explains why some articles are read and some aren't. Your job is to guess what the rule is. Ready?
HINT: YOUR ARTICLE GETS A "NO".
posted by Joe in Australia at 5:15 PM on July 2, 2015
Fine. Go lecture them. Except, they apparently didn't read your article either. Hey, I have a simple rule that explains why some articles are read and some aren't. Your job is to guess what the rule is. Ready?
HINT: YOUR ARTICLE GETS A "NO".
posted by Joe in Australia at 5:15 PM on July 2, 2015
I hypothesized this was just an ascending test after one guess, got three "no"s in testing this hypothesis (1,1,1; 1,1,2; 3,2,1) plus "yes"s for (10,20,30) and (1,10,100) and then typed in "strictly monotonically increasing sequence."
As your nation's future policy wonk / czar, I guarantee all things will be pretty sensible.
posted by zippy at 5:49 PM on July 2, 2015
As your nation's future policy wonk / czar, I guarantee all things will be pretty sensible.
posted by zippy at 5:49 PM on July 2, 2015
"But many of the examples the article then cites when talking about confirmation bias are not ones where the available evidence was leading to a wrong conclusion, but where the available evidence was/is ignored."
You did ignore the available evidence. It told you a bunch of times that you could test as many combinations as you wanted — that evidence was available to you. You chose not to test them.
"This demonstrated to me something I already know about myself, which is that I'm really interested in failure states, as several people noted above. I'm totally fascinated by the marginal cases on the edges of rules where EVERYTHING GOES BATSHIT WONKY BIZARRE. People always get annoyed with me in management/governance positions because I always want to talk about the 1% of cases that our rule doesn't properly cover and what will happen then ... but I always turn out to look psychic when exactly that thing happens. "
This is a tendency that I really, really, really have to work to keep bridled on MeFi, and that has earned me no end to sour looks in activist meetings. I'm super glad that I lucked into a consulting gig for a fellow MeFite where part of my role is to ask those questions implicitly, usually contained in the phrase, "And how would we measure that?" I'm now kinda wondering if there's a career in helping people measure and predict failure, though it seems like it'd be a hell of a downer at cocktail parties.
(My favorite pictures from my recent vacation were when I figured out how graffiti murals will break iPhone autofocus to chunk up into abstract weirdness.)
posted by klangklangston at 6:10 PM on July 2, 2015 [2 favorites]
You did ignore the available evidence. It told you a bunch of times that you could test as many combinations as you wanted — that evidence was available to you. You chose not to test them.
"This demonstrated to me something I already know about myself, which is that I'm really interested in failure states, as several people noted above. I'm totally fascinated by the marginal cases on the edges of rules where EVERYTHING GOES BATSHIT WONKY BIZARRE. People always get annoyed with me in management/governance positions because I always want to talk about the 1% of cases that our rule doesn't properly cover and what will happen then ... but I always turn out to look psychic when exactly that thing happens. "
This is a tendency that I really, really, really have to work to keep bridled on MeFi, and that has earned me no end to sour looks in activist meetings. I'm super glad that I lucked into a consulting gig for a fellow MeFite where part of my role is to ask those questions implicitly, usually contained in the phrase, "And how would we measure that?" I'm now kinda wondering if there's a career in helping people measure and predict failure, though it seems like it'd be a hell of a downer at cocktail parties.
(My favorite pictures from my recent vacation were when I figured out how graffiti murals will break iPhone autofocus to chunk up into abstract weirdness.)
posted by klangklangston at 6:10 PM on July 2, 2015 [2 favorites]
That's interesting. How do the murals do that?
posted by Joe in Australia at 7:15 PM on July 2, 2015
posted by Joe in Australia at 7:15 PM on July 2, 2015
klangklangston: " I'm now kinda wondering if there's a career in helping people measure and predict failure"
Contract law.
posted by Eyebrows McGee at 7:40 PM on July 2, 2015 [1 favorite]
Contract law.
posted by Eyebrows McGee at 7:40 PM on July 2, 2015 [1 favorite]
Yeah, though I think the first failure I'd predict is a late-30s law student earning enough to pay off his degree in the current market.
posted by klangklangston at 12:40 AM on July 3, 2015 [3 favorites]
posted by klangklangston at 12:40 AM on July 3, 2015 [3 favorites]
I got it right, it never even occurred to me not to try a few sequences I thought would get a no and see if they really did, and although I was aware of confirmation bias, didn't see the link until I read the article.
And then the "majority of people guess without seeing a single no answer because they avoid trying things that they think will result in a 'no,' because people don't like to be told 'no.'" part confused and puzzled me.
I mean... There's 'no' and there's 'no.' There's 'No, you can't have candy before dinner," there's 'No, you didn't guess the rule right,' and then there is 'No, that sequence doesn't fit the rule.'
I totally get why people don't like the first 'no,' can understand why they don't like the second 'no' (fear of failure stopping me from attempting stuff is A Thing with me), but can't for the life of me link the third 'no' to the first two, emotionally. It's. Just. The thing isn't even judging a rule you designed. It's just telling you that the peg doesn't fit in the hole, there's no penalty, try another peg now if you want.
And like amtho I like to hear that 'no,' because of what overeducated_alligator said (eponysterical?); I went seeking for it because it'll increase my information.
I just don't see how that seeking is lumped in with negative emotions against rejection in so obviously mechanical a context.
(Maybe I'm a soulless monster? Or a robot? How can I tell?)
(And like amtho, I'd totally use this in a job interview because I would be interviewing for an engineering position and I really don't want coworkers who don't instinctively test to failure. I liked this test a bunch.)
posted by seyirci at 12:14 PM on July 3, 2015 [1 favorite]
And then the "majority of people guess without seeing a single no answer because they avoid trying things that they think will result in a 'no,' because people don't like to be told 'no.'" part confused and puzzled me.
I mean... There's 'no' and there's 'no.' There's 'No, you can't have candy before dinner," there's 'No, you didn't guess the rule right,' and then there is 'No, that sequence doesn't fit the rule.'
I totally get why people don't like the first 'no,' can understand why they don't like the second 'no' (fear of failure stopping me from attempting stuff is A Thing with me), but can't for the life of me link the third 'no' to the first two, emotionally. It's. Just. The thing isn't even judging a rule you designed. It's just telling you that the peg doesn't fit in the hole, there's no penalty, try another peg now if you want.
And like amtho I like to hear that 'no,' because of what overeducated_alligator said (eponysterical?); I went seeking for it because it'll increase my information.
I just don't see how that seeking is lumped in with negative emotions against rejection in so obviously mechanical a context.
(Maybe I'm a soulless monster? Or a robot? How can I tell?)
(And like amtho, I'd totally use this in a job interview because I would be interviewing for an engineering position and I really don't want coworkers who don't instinctively test to failure. I liked this test a bunch.)
posted by seyirci at 12:14 PM on July 3, 2015 [1 favorite]
Programmer here, also feeling smug....
Because of your lack of confirmation bias? :)
posted by storybored at 7:15 PM on July 3, 2015
Because of your lack of confirmation bias? :)
posted by storybored at 7:15 PM on July 3, 2015
« Older Proposing certain things in terms of dystopia that... | You went above the line Newer »
This thread has been archived and is closed to new comments
posted by dry white toast at 8:08 AM on July 2, 2015 [2 favorites]