Google’s Product Development Philosophy

August 7, 2010 11:44 AM

“If you're a politician, admitting you're wrong is a weakness, but if you're an engineer, you essentially want to be wrong half the time. If you do experiments and you're always right, then you aren't getting enough information out of those experiments. You want your experiment to be like the flip of a coin: You have no idea if it is going to come up heads or tails. You want to not know what the results are going to be. ” A Slate interview with Google Research Director Peter Norvig on Google's product development process. [via]
posted by Jasper Friendly Bear (20 comments total)

But is he sure about this?
posted by weapons-grade pandemonium at 11:47 AM on August 7, 2010 [1 favorite]

You've linked to the front page of the blog which happens to have that entry at the top, but it's going to scroll off eventually as new posts are made. The link should be to the permalink of the entry not to the blog itself.
posted by Rhomboid at 12:00 PM on August 7, 2010 [2 favorites]

*Throws this idea against the wall.*
posted by carsonb at 12:02 PM on August 7, 2010

Rhomboid, thanks for pointing that out. Hopefully one of the mods can fix that for me.
posted by Jasper Friendly Bear at 12:05 PM on August 7, 2010

Fixed!
posted by jessamyn at 12:14 PM on August 7, 2010

The whole beta model is completely at odds with conventional production and manufacturing; you never see General Motors release a beta version of a car, for instance.

Obviously, this person has never purchased a GM product.
posted by three blind mice at 1:24 PM on August 7, 2010 [8 favorites]

Since long before he worked at Google, Peter Norvig has been bloody brilliant. I'm looking forward to reading this.
posted by koeselitz at 2:30 PM on August 7, 2010 [3 favorites]

I enjoyed the interview, but I liked even more the interview with Ira Glass where he talks about storytelling and narrative and the creative process. My favorite quote:

To do any kind of creative work well, you have to run at stuff knowing that it's usually going to fail. You have to take that into account and you have to make peace with it. We spend a lot of money and time on stuff that goes nowhere. It's not unusual for us to go through 25 or 30 ideas and then go into production on eight or 10 and then kill everything but three or four. In my experience, most stuff that you start is mediocre for a really long time before it actually gets good. And you can't tell if it's going to be good until you're really late in the process. So the only thing you can do is have faith that if you do enough stuff, something will turn out great and really surprise you.
posted by incessant at 3:33 PM on August 7, 2010 [6 favorites]

This was an excellent interview. I know some engineers who would be pulling their hair out at some of the thing Norvig said, though. Engineering to a lot of professional engineers is about making something that does not ever fail.

I liked the story about 9 in 10 stories getting spiked on This American Life and 39 headlines in 40 getting spiked at the Onion. There is a reason you see so much goofy stuff posted on Metafilter. In Engineering this is referred to as Quality Control.
posted by bukvich at 3:57 PM on August 7, 2010

This is a beta comment, so don't be surprised if it doesn't catch on and I ask the mods to delete it.
posted by dhartung at 4:06 PM on August 7, 2010

Also that interviewer has a blog with interviews of an interesting collection of folks.

Link.
posted by bukvich at 4:10 PM on August 7, 2010 [1 favorite]

Norvig is pretty great but Slate really sucks.

So I guess it's 50/50 here...
posted by delmoi at 4:28 PM on August 7, 2010

if you're an engineer, you essentially want to be wrong half the time.

So this is why my EE and ME friends always make fun of us software guys.
posted by RobotVoodooPower at 5:00 PM on August 7, 2010 [1 favorite]

If you do experiments and you're always right, then you aren't getting enough information out of those experiments.

How much sense this makes: 0.
posted by DU at 5:04 PM on August 7, 2010 [2 favorites]

He might make some nice software but I'm not ever going near a bridge this guy makes.
posted by WalterMitty at 5:25 PM on August 7, 2010

If high-performance hardware never breaks, you haven't yet optimized it. The point is to try to push things to the limit of failure, then back off just a little bit.

I am an engineer. I learned a lot from a quirky Israeli engineer who worked at another company that did work for the company I worked for several years ago. The prototype hardware his company built would, at times, act completely opposite of what was expected and intended. At the time, it would freak me out and I would panic because I would be worried about schedule slips, cost overruns, etc. He would just look at the problems gleefully and shout, "We are learning something!!!"

If you never break anything, you don't learn nearly as quickly. Many engineers will say the most intellectually stimulating bit of the job is when you are troubleshooting especially difficult problems. The first few times through, there is a panic response. At least for me, I finally adopted the Israeli engineer's mantra: We are learning something!
posted by Doohickie at 7:04 PM on August 7, 2010 [9 favorites]

This rebranding of (software) engineering as embracing failure has only happened in the last 3 or 4 years, and I think it's a bit silly. Bridges and airplanes that fail aren't normally considered inevitable, so this idea of failure is really the thinking of a venture capitalist, not an engineer. And the reality is they aren't even really talking about engineering failures in the sense of embracing experiments that cause performance issues, bugs, unreliable services and downtime. Google Wave was a failure, but not when judged by the standard of good engineering practices -- it was a huge success by that standard. It failed because no-one used it, so the failure they embrace is the kind where the software fails the users - it has a poor user experience, doesn't solve their problem, is awkward and badly thought out, etc. What's funny is that this is a direct outcome of Google's supposedly successful engineering-centric culture that gives engineers a lot of influence over product design decisions, but no accountability. Failures are inevitable, so why bother thinking through the user experience? No-one's getting fired for screwing that up.
posted by AlsoMike at 7:25 PM on August 7, 2010 [4 favorites]

But is he sure about this?

He's about 50% sure.
posted by hal_c_on at 2:43 AM on August 8, 2010

you could use these kind of processes in government but

a) people think government experiments are unethical/unfair/etc
b) government doesn't seem to be good at failing. failing is about recognizing when things have gone wrong and trying to do that as fast as possible. a failing program will often give concentrated benefits to a few people and do a very small amount of harm to a lot of people. trying to kill off failing programs is a very difficult collective action problem.
posted by drscroogemcduck at 3:39 AM on August 8, 2010

Actually, drscroogemcduck, I think the US constitution has the idea of small laboratories of failture built right in. The idea of state's rights, the 9th and 10th amendment, etc. have the possibility of allowing each state to experiment in different ways of doing things. Other states could then replicate successful programs or policy as it makes sense to their area.

There are a few problems in our contemporary context, however:
1) There is *good* mechanism for a partnership between the federal government and state governments to try to new things while simultaneously helping people out if it doesn't work.

2) states rights have become synonymous with the most vile policies of our nation. I would call myself a "state's rights democrat," except that name was already taken by the jerks who gave us such things as the Civil War, Jim Crow, segregation, and every horrible evangelical law on the books.

3) People see the control of the centralized government as an easier way to establish their political power, to the detriment of decentralization of power.
posted by imneuromancer at 6:09 AM on August 8, 2010