Graphical views of artificial neural networks
August 1, 2014 9:06 AM   Subscribe

Two approachable visual presentations of simple neural networks: one showing how a soft activation function allows the successive layers of a neural network to distort the input until the different classes are separable, and the other showing how a hard step activation function can be represented as carving out polygons in the space of inputs. Don't be intimidated by the rather condensed summaries above- the actual articles are very readable.
posted by Jpfed (4 comments total) 36 users marked this as a favorite
This is triggering some kind Voight-Kampff response in me. "It's trying to come up with a classifier, but you haven't given it enough layers. Why is that, Leon?"
posted by benito.strauss at 9:23 AM on August 1, 2014 [5 favorites]

This aside (for some reason the link to it in the article doesn't work) is also worth checking out, since I find that knowing what such a network can't do is usually very helpful for understanding it.
posted by invitapriore at 10:20 AM on August 1, 2014

The cool and frustrating thing about neural networks is how much they can be extended and modified to suit the demands of different problem spaces. In the lab I worked at, we were using them to classify the activity of drug candidates based on assay data. If you're a Bayesian like my PI, you can put a prior distribution on the weights in a layer, use the logistic function for activation (tanh will learn better in general since it tends not to cause as much saturation of layers down the line, but we only ever used one hidden layer and since the logistic function gives output in the range [0, 1] it's useful when you want to interpret your output as a probability) and the cross-entropy as your error function, and on the other end you get pretty reliable probabilistic output for each test sample. This is mostly useful when you're not so much interested in characterizing specific instances as whole datasets, since if you sum those predictions across the dataset you get -- bam! -- the expected number of active compounds. It's all very cool, but it makes you realize just how much these things need to be tuned to suit the application.
posted by invitapriore at 11:23 AM on August 1, 2014

Thank you! I often heard during undergrad that understanding what goes on inside a neural network is "hard." Naturally, I've been curious about it ever since; this goes some way toward better explaining how they work.
posted by Verg at 11:23 AM on August 1, 2014

« Older NASA claims to have tested a "reactionless" space...   |   I Will Be There Newer »

This thread has been archived and is closed to new comments