May the bots have mercy on us all
December 18, 2017 9:07 AM Subscribe
How machines learn. (Main video plus Footnote video.) Explained for laypersons by C. G. P. Grey.
Cute video. Love the adorable "3's, bees" sorting bot GIF. I'm checking out Grey's "Cortex" YouTube channel. Thanks!
"Right now, the cutting edge is very much 'I hope you like linear algebra'." That really cracked me up. And so true. I say something to that effect almost every semester to students who have taken a poorly taught and poorly motivated course in linear algebra without realizing just how fucking important that subject is to all of applied mathematics, let alone machine learning.
I appreciate that the video is aimed at lay people, and it does it's job engagingly well. But I have a small beef with part of its story, namely, its use of a currently fashionable trope when talking about machine learning: something to the effect that "scientists don't know how their machine learning algorithms work." Especially for something aimed at lay people, I think this is a very bad approach, because it's misleading. I get what is intended (we don't know exactly how the algorithm, once trained, is doing what it's doing, or why the weights are exactly what they are, and so on), but it clearly is an exaggeration. Experts certainly understand, at an abstract level, what the algorithms are doing.
Indeed, if we don't understand how machine learning algorithms work, then we don't understand statistical physics in exactly the same way. And I think that would be an unfortunate idea to convey to lay people. The idea that complex systems, which are unknowable in the small, can be understandable in the large (the "thermodynamic limit") is an incredibly beautiful and deep discovery.
posted by mondo dentro at 9:47 AM on December 18, 2017 [6 favorites]
"Right now, the cutting edge is very much 'I hope you like linear algebra'." That really cracked me up. And so true. I say something to that effect almost every semester to students who have taken a poorly taught and poorly motivated course in linear algebra without realizing just how fucking important that subject is to all of applied mathematics, let alone machine learning.
I appreciate that the video is aimed at lay people, and it does it's job engagingly well. But I have a small beef with part of its story, namely, its use of a currently fashionable trope when talking about machine learning: something to the effect that "scientists don't know how their machine learning algorithms work." Especially for something aimed at lay people, I think this is a very bad approach, because it's misleading. I get what is intended (we don't know exactly how the algorithm, once trained, is doing what it's doing, or why the weights are exactly what they are, and so on), but it clearly is an exaggeration. Experts certainly understand, at an abstract level, what the algorithms are doing.
Indeed, if we don't understand how machine learning algorithms work, then we don't understand statistical physics in exactly the same way. And I think that would be an unfortunate idea to convey to lay people. The idea that complex systems, which are unknowable in the small, can be understandable in the large (the "thermodynamic limit") is an incredibly beautiful and deep discovery.
posted by mondo dentro at 9:47 AM on December 18, 2017 [6 favorites]
The other dimension that isn't really covered in the video is what features are. This is quite important to understanding how machine learning is done in practice and setting policy around what a model is or is not "allowed" to do.
In the example of image recognition the training example is an entire image (a bunch of pixels), and the label is "bee" or "three". This is pretty much all-or-nothing, there's no way to understand what the model is doing or give it anything other than more images.
For something like "is the user going to watch this video", the situation is a bit different. The training example is a collection of features that are things known about the user, the video, and the state of the world in general.
Although the impact of each feature may be hidden in the neural net and unknowable, the choice of what features to use is very significant (called feature selection in the research literature) and is controlled more or less entirely by humans. There are automated feature selection algorithms, but humans still have to supply the set of features that are available to choose from.
posted by allegedly at 10:28 AM on December 18, 2017 [1 favorite]
In the example of image recognition the training example is an entire image (a bunch of pixels), and the label is "bee" or "three". This is pretty much all-or-nothing, there's no way to understand what the model is doing or give it anything other than more images.
For something like "is the user going to watch this video", the situation is a bit different. The training example is a collection of features that are things known about the user, the video, and the state of the world in general.
Although the impact of each feature may be hidden in the neural net and unknowable, the choice of what features to use is very significant (called feature selection in the research literature) and is controlled more or less entirely by humans. There are automated feature selection algorithms, but humans still have to supply the set of features that are available to choose from.
posted by allegedly at 10:28 AM on December 18, 2017 [1 favorite]
If you want a little more nitty gritty on how machine learning works, these videos from 3Blue1Brown are pretty great: parts one, two, and three. While the focus is on building a visible representation of a neural network to provide an intuitive understanding of how they work, the videos do use some basic linear algebra notation and concepts from calculus to explain the underlying functions, so some mathematical literacy might be needed to get the most out of it.
posted by peeedro at 12:16 PM on December 18, 2017 [8 favorites]
posted by peeedro at 12:16 PM on December 18, 2017 [8 favorites]
Although the impact of each feature may be hidden in the neural net and unknowable,
? why? nothing stopping anyone from using a partial dependence plot
posted by MisantropicPainforest at 12:26 PM on December 18, 2017
? why? nothing stopping anyone from using a partial dependence plot
posted by MisantropicPainforest at 12:26 PM on December 18, 2017
? why? nothing stopping anyone from using a partial dependence plot
That may be reasonable for a small feature set, but for large ones (e.g. image recognition nets with tens of thousands of inputs), it's not really feasible to systematically vary the inputs or feature weights to see what happens. The feature space is just too large.
However, other techniques have been developed that start to give us some sense of how these more complex networks do what they do. For example, Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson's Understanding neural networks through deep visualization.
posted by jedicus at 1:00 PM on December 18, 2017 [1 favorite]
That may be reasonable for a small feature set, but for large ones (e.g. image recognition nets with tens of thousands of inputs), it's not really feasible to systematically vary the inputs or feature weights to see what happens. The feature space is just too large.
However, other techniques have been developed that start to give us some sense of how these more complex networks do what they do. For example, Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson's Understanding neural networks through deep visualization.
posted by jedicus at 1:00 PM on December 18, 2017 [1 favorite]
I loved this and now feel like I can explain machine-learning (at it's most basic and theoretical) as a layperson to people who are even more layperson-esque than myself (my mom, basically).
Are there more computer science-y videos like this? Is there an algorithm that can deliver them to me?
posted by windbox at 2:55 PM on December 18, 2017
Are there more computer science-y videos like this? Is there an algorithm that can deliver them to me?
posted by windbox at 2:55 PM on December 18, 2017
Yay new CGP Grey video!! If you liked this, and have already checked out his youtube channel but haven't yet checked out his podcasts Hello Internet or Cortex, then you should do that!
And man I wish I had this 3 days ago. I was *literally* presenting on machine learning and how it worked. Damm you Grey!
posted by cgg at 2:57 PM on December 18, 2017
And man I wish I had this 3 days ago. I was *literally* presenting on machine learning and how it worked. Damm you Grey!
posted by cgg at 2:57 PM on December 18, 2017
Is it weird that I feel sorry for the sad stupid bots?
posted by AFABulous at 7:00 PM on December 18, 2017
posted by AFABulous at 7:00 PM on December 18, 2017
See also, 3blue1brown: But what *is* a Neural Network? | Chapter 1, deep learning
posted by pharm at 5:59 AM on December 19, 2017
posted by pharm at 5:59 AM on December 19, 2017
Reminds me of Jonathan Coulton's song "(It's Gonna Be) the Future Soon."
The singer of the song builds an army of robots, "building them one laser gun at a time."
I will do my best to teach them
About life and what it's worth.
I just hope that I can keep them from destroying the Earth....
posted by destinyland at 10:43 AM on December 19, 2017 [1 favorite]
The singer of the song builds an army of robots, "building them one laser gun at a time."
I will do my best to teach them
About life and what it's worth.
I just hope that I can keep them from destroying the Earth....
posted by destinyland at 10:43 AM on December 19, 2017 [1 favorite]
Is it weird that I feel sorry for the sad stupid bots?
I don't think you're the only one; /r/cgpgrey wants bot plushies.
posted by a snickering nuthatch at 12:24 PM on December 21, 2017
I don't think you're the only one; /r/cgpgrey wants bot plushies.
posted by a snickering nuthatch at 12:24 PM on December 21, 2017
« Older PRESS ANY KEY WHEN READY | Corporations are robots, my friend Newer »
This thread has been archived and is closed to new comments
posted by a snickering nuthatch at 9:33 AM on December 18, 2017 [1 favorite]