Comments on: Foldit - Crystal structure of a monomeric retroviral protease solved by protein folding game players

Foldit - Crystal structure of a monomeric retroviral protease solved by protein folding game players

East Manitoba Regional Junior Kabaddi Champion '94 — Sun, 18 Sep 2011 12:21:28 -0800

Gamers solve molecular puzzle that baffled scientists. The structure of a protein causing AIDS in rhesus monkeys had not been discovered in 15 years of attempts. Players of a videogame did it in ten days. Foldit, the game in question. Abstract. Previously, previously.

By: hippybear

hippybear — Sun, 18 Sep 2011 12:28:40 -0800

That's really cool. I downloaded and played with FoldIt back when it was first introduced, and saw how it could be an interesting crowdsourcing tool to harness the power of people who like to fiddle with things to solve protein folding problems. It held my interest pretty much not at all, but then I'm not really much into videogames in general and puzzle games in particular. But how cool that it was actually used like this! I love that the solution wasn't the product of a single person, and that the collective which came up with the solution requested that credit go to the group and not any one person. Thanks so much for posting!

By: reenum

reenum — Sun, 18 Sep 2011 12:34:11 -0800

Take that, mom! Games do help in real life!

By: pracowity

pracowity — Sun, 18 Sep 2011 12:36:10 -0800

Just not the ones you play.

By: Philosopher Dirtbike

Philosopher Dirtbike — Sun, 18 Sep 2011 12:39:53 -0800

Just not the ones you play. Says you. I play "Global Thermonuclear War".

By: slater

slater — Sun, 18 Sep 2011 12:40:51 -0800

Gary Larson predicted this

By: hanoixan

hanoixan — Sun, 18 Sep 2011 12:49:15 -0800

Achievement unlocked

By: neuromodulator

neuromodulator — Sun, 18 Sep 2011 12:51:33 -0800

That's amazing. I often wondered about the possibility of stuff like this when looking at the complex system games like Magic: The Gathering, Starcraft, and World of Warcraft and the tremendous amount of information that the gaming community generates around these games.

By: zipadee

zipadee — Sun, 18 Sep 2011 13:06:18 -0800

What a feelgood story! Unfortunately, the next step is that the military starts investigating small-unit tactics using multiplayer shooters. Or the CIA does counterintelligence using spying games.

By: thanotopsis

thanotopsis — Sun, 18 Sep 2011 13:08:30 -0800

I tried Foldit long ago, and I similarly wasn't engaged. Now, if someone talented in the gaming industry could develop a version of the game that includes variable rewards, an achievement ladder, and perhaps some way to slice up your competitors with a large flaming sword or semi-automatic rifle, then we could have a really productive environment, here.

By: Navelgazer

Navelgazer — Sun, 18 Sep 2011 13:12:19 -0800

Unfortunately, the next step is that the military starts investigating small-unit tactics using multiplayer shooters. Or the CIA does counterintelligence using spying games. "Pentagon report details dire need for strategically-placed instant health paks and body armor sprinkled throughout enemy territory."

By: zippy

zippy — Sun, 18 Sep 2011 13:24:19 -0800

Players of a videogame did it in ten days One day to solve, plus nine days cumulative of taunting the molecule and teabagging it.

By: kersplunk

kersplunk — Sun, 18 Sep 2011 13:41:50 -0800

Unfortunately, the next step is that the military starts investigating small-unit tactics using multiplayer shooters. Or the CIA does counterintelligence using spying games. "No need for the grappling hook, I can just rocket jump up"

By: Godspeed.You!Black.Emperor.Penguin

Godspeed.You!Black.Emperor.Penguin — Sun, 18 Sep 2011 13:48:45 -0800

Next up: Unlocking the 9th Chevron.

By: subject_verb_remainder

subject_verb_remainder — Sun, 18 Sep 2011 13:59:39 -0800

Foldit is still the first and only serious game that I've found fun and engaging. I'm sure there's been changes since it first came out, but you did have scores and leaderboards and weekly challenges, and there were general rules on how to unfold the proteins best (certain things parallel to others, for example). I'd put it up and mess with it while waiting for dinner to cook or listening to podcasts or NPR. At the time, the highest ranking teams were MIT and Blizzard Entertainment. I loved that alliance between the science nerds and gamers (which often intersect). I'd loved it more if they had nurtured the competitiveness a bit more with some more common game qualities (like the ability to 'steal' intel (copy solutions) from rival teams to work off of). The only thing I didn't like was the lingering suspicion that making a difference was really a lie, and that unfolding these 3D structures would ultimately be totally useless to researchers. It's really great to see this kind of positive feedback to dispel that notion.

By: Kid Charlemagne

Kid Charlemagne — Sun, 18 Sep 2011 14:08:05 -0800

This is my surprised fa...no, sorry, that this sort of thing was coming was so painfully obvious to me that I can't even fake it. There's a common attitude in biochemistry that everything is so complex that bringing math to the table for solving real world problems is an exercise in futility and our only hope is in empirical measurements and lots of 'em. As a result, large swaths of biochemistry are using the very best tools of the nineteenth century and dismiss things like Michaelis Menten equation and the Einstein Stokes equation as being "just theoretical". When I left my job (hint - rhymes with wiser) everyone had the equivalent of a Cray II supercomputer (or better) sitting on their desk, but our bio-assay development took a couple weeks to come up with the right concentrations of product and target via the very best groping and hoping techniques known to man and we dutifully plotting our data on a semi-log plot, because that's what you did when you only had a slide rule and had to interpret your data graphically, never mind that it totally wrecked your ability to think intuitively about what was going on. I wish I had a link to some of the stuff that Turtles all the way down had written in his blog about the crappy software that scientists saddle themselves with, because this is probably a direct consequence of that. And I need to get back to learning R.

By: radiobishop

radiobishop — Sun, 18 Sep 2011 14:15:33 -0800

Unfortunately, the next step is that the military starts investigating small-unit tactics using multiplayer shooters. Next? VBS1: "The system enables the practice of small unit military tactics in an interactive multiplayer 3D environment" VBS1 customers

By: Wyatt

Wyatt — Sun, 18 Sep 2011 14:21:45 -0800

So crowd source everything... but provide a small stipend to every member of society so they can tinker with molecules and microchip designs all they want! Right? Right??

By: lazaruslong

lazaruslong — Sun, 18 Sep 2011 14:26:32 -0800

Gaming to solve problems has been in the buzz a lot lately, mostly thanks to Jane McGonigal. Check out this TED Talk for a primer. Also, the government already has been using FPS games as a recruiting tool, via America's Army. Great link! Thanks for posting.

By: Aquaman

Aquaman — Sun, 18 Sep 2011 14:39:11 -0800

Finally, my skillz have purpose! Take that, going outside!

By: Grimgrin

Grimgrin — Sun, 18 Sep 2011 14:49:03 -0800

It seems like what they're doing with games generally is building a system that automates a lot of the tasks you'd otherwise have to know how to do, to be able to perform a task, and using this to allow a greater pool of individuals to solve problems. The question I have is "are assumptions made by the game in automating the underlying tasks, limiting or distorting the possible solutions the game comes up with"? There was an example from back in the 1980's with a computer simulated naval war game, that a team entered with a genetic algorithm, it had basically used a flaw in the points system that described the 'cost' of each ship to produce a huge armada of otherwise useless single purpose ships it could use to achieve victory. The simulation became useless because the model created solutions that only worked in the model, that were better at achieving victory than solutions that would work in the real world. Now in the case of something like FoldIt, there's little danger of this happening, because there's such a strong model underlying the game, and there's such a strong mechanism for checking the results. I worry though about applying this to things like economics military strategy or social problems, because not just the models are fuzzier, but also because the solutions themselves are subject to external pressures and manipulation. Look at criticism on both sides of the "Millennium Challenge". Van Riper's proponents say he showed the fatal weakness of US Naval doctrine. His opponents say he was doing things like launching 2 ton silkworm missiles off of Zodiacs, because he wanted to make the brass look bad.

By: delmoi

delmoi — Sun, 18 Sep 2011 14:57:30 -0800

What I'm wondering here is who got paid. The research was being done for Pfizer, did the people doing the actual work get paid here?

By: East Manitoba Regional Junior Kabaddi Champion '94

East Manitoba Regional Junior Kabaddi Champion '94 — Sun, 18 Sep 2011 15:06:48 -0800

The research was being done for Pfizer, did the people doing the actual work get paid here? This research was done "for" the University of Washington, and the results are publicly available. Perhaps you are mistaken due to the name of the "Mason-Pfizer" virus?

By: floam

floam — Sun, 18 Sep 2011 15:07:45 -0800

The scientists were really baffled until these gamers came along?

By: East Manitoba Regional Junior Kabaddi Champion '94

East Manitoba Regional Junior Kabaddi Champion '94 — Sun, 18 Sep 2011 15:08:50 -0800

Baffled.

By: motty

motty — Sun, 18 Sep 2011 15:19:52 -0800

Baffled. They were wandering around in their white coats, nervously cleaning their glasses, muttering dark imprecations, bumping into things and one another, as baffled people do, and literally had no idea how they could possibly solve this problem, until...

By: Wyatt

Wyatt — Sun, 18 Sep 2011 15:21:12 -0800

Grimgrin, flaws in a game's representative quality would just require that the developers follow the way the game is playing out and patch as needed. If Blizzard never patched their games, exploitative and imbalanced strategies would prevail. However, going into the game-making process recognizing that often the players can be more innovative than the designers really allows the harnessing of crowdsourcing not just for completing the tasks in the game, but for designing a better version of the game itself.

By: Sublimity

Sublimity — Sun, 18 Sep 2011 15:58:22 -0800

Grimgrin asks: The question I have is "are assumptions made by the game in automating the underlying tasks, limiting or distorting the possible solutions the game comes up with"? David Baker spoke about Foldit at my workplace recently. (A few of my teammates and I did our graduate and/or postgrad work in his dept at UW, or in his lab.) Regarding the above question, the problem-solving that the players do is pretty much the exact opposite of what you are wondering about. The game does things like assess the suitability of the structure the players make up (i.e., are the atoms too close or far apart, etc etc) but it doesn't limit in terms of where the players can put the atoms, or how they can alter the sequence they're working with. This is where the interaction of real live people makes the biggest difference in the folding problem--it's computationally useless to try to write an algorithm to try every single alteration (insertions/ deletions of all kinds of amino acids), but a thoughtful and eagle-eyed person can make reasonably intelligent choices about what kinds of alterations might work in *this* particular place and *this* particular structure. I totally love that the Baker lab is tapping the brain cloud for protein folding problems. Science for the win!

By: obiwanwasabi

obiwanwasabi — Sun, 18 Sep 2011 16:22:52 -0800

Unfortunately, the next step is that the military starts investigating small-unit tactics using multiplayer shooters. "Pull the trigger, Mr Wiggin! It's only a game!"

By: Grimgrin

Grimgrin — Sun, 18 Sep 2011 16:23:26 -0800

Sublimity: That still assumes the algorithm is correct in it's assessments though. Again, as a hypothetical, assume the algorithm does not model a certain set of interactions correctly, and so disallows valid solutions that depend on that interaction. I'm not saying there is a mistake, and in the case of protein folding if there is a discrepancy between the assessment made by the algorithm and the real world I'd expect it to be caught pretty quickly. Don't get me wrong, this is a great idea, and I'm sure it is going to be very useful, I'm just saying this is where I see problems, particularly if you try to generalize it into areas where it's much harder to test your results, and the underlying models are much more approximate. Wyatt: The quality of these games is determined by how accurately they model the real world process or thing that the game represents. You have to have some method of checking your results, and I'm not convinced you can crowd source that.

By: hippybear

hippybear — Sun, 18 Sep 2011 16:33:21 -0800

a brain cloud? What? Has Lily Tomlin invaded Steve Martin's body again?

By: nzero

nzero — Sun, 18 Sep 2011 16:54:20 -0800

By: Evilspork

Evilspork — Sun, 18 Sep 2011 16:59:06 -0800

You think that's impressive? I've done all these things without even being at my computer!

By: twoleftfeet

twoleftfeet — Sun, 18 Sep 2011 17:09:39 -0800

It's cool that they've solved this problem, but I didn't shout "hooray" until I read this phrase: University of Washington's Center for Game Science. There's a Center for Game Science? There's a Center for Game Science! Hooray! They've got some other games besides Foldit. Science really has come a long way. The fusion of science and gaming means more interest in science, and that's a Good Thing. Now, to get even more interest, I'm off to start the Center for Porn Science.

By: ZenMasterThis

ZenMasterThis — Sun, 18 Sep 2011 17:37:47 -0800

... going into the game-making process recognizing that often the players can be more innovative than the designers ... *Imagines God as a game designer*

By: emjaybee

emjaybee — Sun, 18 Sep 2011 17:55:15 -0800

Of course Star Trek warned us years ago that this idea could be used for evil.

By: maxwelton

maxwelton — Sun, 18 Sep 2011 18:20:12 -0800

(Regarding ideas being used for evil, everything can be used for evil. It's definitely not a reason to avoid an idea. It's when people don't think of anything but the good uses that we end up crap like 99% of email being spam and browsers being built with architecture that allows distant programs to execute root-level programs on your local computer. Things would be so much better if people just asked "what would a direct marketing douche do with this tool?" and build in safeguards against using it that way.)

By: Twang

Twang — Sun, 18 Sep 2011 18:23:40 -0800

And so begins the Age of Cybermen! The Human Use of Human Beings is a book by Norbert Wiener. It was first published in 1950 and revised in 1954.... It analyzes the meaning of productive communication and discusses ways for humans and machines to cooperate, with the potential to amplify human power and release people from the repetitive drudgery of manual labor, in favor of more creative pursuits in knowledge work and the arts.

By: goethean

goethean — Sun, 18 Sep 2011 18:31:31 -0800

When I left my job (hint - rhymes with wiser You were the German Kaiser?!

By: Panjandrum

Panjandrum — Sun, 18 Sep 2011 18:38:36 -0800

When I left my job (hint - rhymes with wiser You were the German Kaiser?! Duh! He's Kid Charlemange, the Holy Roman Emperor!

By: twoleftfeet

twoleftfeet — Sun, 18 Sep 2011 19:05:43 -0800

hint - rhymes with wiser You were selling vaporizers?

By: obiwanwasabi

obiwanwasabi — Sun, 18 Sep 2011 19:17:49 -0800

Ladies and gentlemen, I give you Paul Reiser!

By: zippy

zippy — Sun, 18 Sep 2011 19:20:56 -0800

Ladies and gentlemen, I give you Paul Reiser He would have sent a bunch of colonists out to investigate the molecule, then arrived later with a detachment of Space Marines and Ellen Ripley.

By: hippybear

hippybear — Sun, 18 Sep 2011 19:25:39 -0800

Either that or he would have had a way-too-expensive (yet miraculously affordable) apartment in NYC with his wife, and then ruined all the fun and hilarity by finally having a child and tried to eek out another year or two of enjoyment even after it was obvious that nobody cared any more.

By: Kid Charlemagne

Kid Charlemagne — Sun, 18 Sep 2011 19:45:20 -0800

You have to have some method of checking your results, and I'm not convinced you can crowd source that. You go through the journals, you find proteins with known structures, then you throw them at the gamers and let them grind on them and see if the highest scores are the ones that match the known structures. And, to be honest, given what you have to do to a protein for x-ray crystalography, there is a good chance that if there is a discrepancy it's because they denatured the protein when they were preparing the crystal.

By: twoleftfeet

twoleftfeet — Sun, 18 Sep 2011 19:56:21 -0800

hint - rhymes with wiser He would have sent a bunch of colonists out to investigate the molecule, then arrived later with a detachment of Space Marines and Ellen Ripley. You worked on the Starship Enterprise, Sir?

By: en forme de poire

en forme de poire — Sun, 18 Sep 2011 20:49:50 -0800

>> You have to have some method of checking your results, and I'm not convinced you can crowd source that. > You go through the journals, you find proteins with known structures, then you throw them at the gamers and let them grind on them and see if the highest scores are the ones that match the known structures. I think in this specific case, they had crystals, but couldn't solve the structure. (For non-biologists, the diffraction pattern you get from looking at a crystallized protein isn't enough to give you the structure, but you can check if a given structure is consistent with that pattern pretty easily.) So checking the results is sort of "built in." And actually, KC, I think they did one better than what you're talking about, because FoldIt competed at CASP. (CASP is a big academic structure prediction contest, where people try to predict structures that haven't been published. The entries are scored once the real structures come out.)

By: hattifattener

hattifattener — Sun, 18 Sep 2011 21:46:32 -0800

are assumptions made by the game in automating the underlying tasks, limiting or distorting the possible solutions the game comes up with? Protein folding and diffraction crystallography, AIUI, are some of those problems that we know very well how to evaluate solutions to, or to make incremental improvements to a given solution, but the space of solutions is large and we don't have terribly good ways of finding possibilities to evaluate. Many techniques like GA and simulated annealing basically consist of randomly generating trials, scoring them, and using the results to restrict the space of the next randomly generated set of solutions (repeat millions of times). So in the case of Foldit, the humans can't generate distorted results any more than the original random-number generator could. You'll notice that before the researchers celebrated, they verified Foldit's solution using what I assume were proven tools. The game design (or, more likely, the limitations of human cognition) might restrict the space of solutions that Foldit can produce, but this success suggests that's not a problem for protein folding. I'd guess it helps a lot that we've evolved and learned to understand various physical phenomena that protein folding feels similar to. Crowdsourcing something that has no intuitive analogue might not be as successful.

By: Grimgrin

Grimgrin — Sun, 18 Sep 2011 22:39:45 -0800

My question was not about Foldit, it was about the idea of generalizing the technique. hattifattener: These are the questions that interest me. How good to your verification tools have to be, and how good does your model have to be and how 'intuitive' does the behavior of the system have to be for this to work? Basically how far can this technique be pushed and still give useful results?

By: Kid Charlemagne

Kid Charlemagne — Sun, 18 Sep 2011 22:51:11 -0800

The other thing about fold.it is that you have a human working on global minima (what happens if I drag this helix over here?) paired with a computer working on local minima (side chain shake and the like). I'm not sure that you couldn't crowdsource something without an intuitive analog - you'd just have to put a bunch of simple exercises that exposed the players a new set of heuristics and let them internalize that until they could intuit logical solutions. I think this technique can be pushed a long long way, but in order to do that you have to figure out a way to make it fun enough that people are willing to push through the initial learning stages. And you can't push it very fast because the moment you adjust your model so much that your good players are suddenly no longer able to play the game, you're done.

By: memebake

memebake — Mon, 19 Sep 2011 01:02:41 -0800

Kid Charlemagne There's a common attitude in biochemistry that everything is so complex that bringing math to the table for solving real world problems is an exercise in futility and our only hope is in empirical measurements and lots of 'em. I've suspected this since I saw a talk by Craig Venter years ago - if I understand it correctly he accelerated the whole genome project rapidly by rounding up the right sort of mathematicians and computer science people and concentrating on the computation parts of the task rather than the biological ones.

By: blue_beetle

blue_beetle — Mon, 19 Sep 2011 05:45:35 -0800

The human mind is an awesome computer, just not in an an immediately obvious way.

By: en forme de poire

en forme de poire — Mon, 19 Sep 2011 08:59:59 -0800

My question was not about Foldit, it was about the idea of generalizing the technique. I think some of the lessons about why this worked so well with Foldit do generalize. For NP-complete problems like protein folding, it's easy to score a given solution (and for Foldit there were some extra reasons why this was true) but intractable to find the best solution. There's also the visual part of it, which people are still better at than computers. So I think Foldit is a particularly good example, but maybe also representative of a broader class of problems: I can see people being really good at things like clique finding, too.

By: Llama-Lime

Llama-Lime — Mon, 19 Sep 2011 13:10:14 -0800

memebake: "accelerated" is an interesting way to put it, but yes, the public effort had not planned nearly enough resources to assembling (or even distributing) the data they were generating. Jim Kent was the grad student (and retired game programmer) who performed the heroic effort of assembling the public effort's data at the last minute. Venter "accelerated" the public consortiums efforts because there was a fear that Venter would swoop in at the last minute and patent the genome, nullifying the largest and greatest collaboration that the field of biology has ever seen. This did not make Venter many friends, but he's wealthy enough that he doesn't need them, and the ill will is mostly forgotten today.

''It's easy for Venter to say of course you should have had this all planned out,'' Dr. Haussler said. ''But he had the ability to organize an industrial-scale effort. Given the culture of the public project, it would have been very difficult to graft on an autocratic new software project, and if you had assigned a team to build this whole thing over several months it would have been a very difficult proposition. So what Jim has done is miraculous in many ways. No one expected anyone could come in and put this together in four weeks.''

More trivia: Venter's lead programmer on the assembler went to grad school with Haussler, in CS at Boulder, where they both shared an interest in biology. And the lead author of the HIV protein FoldIt paper that started this whole thread, Firas Khatib, got his Ph.D. in Haussler's and Kent's academic department at UC Santa Cruz just a couple years ago. Bioinformatics is a small small world, and a lot of it goes through the small small town of Santa Cruz, where I live and know many of these people, so take anything I say with the appropriate amount of salt.

By: -harlequin-

-harlequin- — Tue, 20 Sep 2011 05:38:07 -0800

Maybe foldit has been improved since I tried it - I hope so, because my problem was that I just now learned more about what I was supposed to be doing in the "game" from reading the summary about it in this thin glossy MSNBC science article, than from when I was trying to get into the game itself. I tried to find help, an introduction, controls, but foldit was colossally unhelpful in telling me what I was doing or how to use it or controls or anything. I hope it was some beta issue or something. It seemed like quite an oversight, so trivially fixed compared to all the work that had obviously already gone into making the simulator actually work.