Neural network playground: train one in your browser

Every press of Train runs real gradient descent with backpropagation on this page, with nothing sent to a server. The background shows what the network currently believes at every spot: the stronger the color, the surer it is. The points drawn are the half of the data it trains on. The test numbers come from the other half, which it never trains on.

What to try

1. See why hidden layers matter. On Circle, set Hidden layers to 0 and press Train. With no hidden layer the network can only draw a straight line, so it stays near chance on test data. Now set one hidden layer of 4 neurons and train again: the line bends into a closed shape, and test accuracy passes 90%.
2. Try XOR the same way. Opposite corners share a label, so no single straight line can work. No hidden layer fails, and one small hidden layer solves it.
3. Take on the spiral. One layer of 4 neurons gets stuck well short. Three layers of 8 neurons get every training point right, and take longer.
4. Watch it overfit. On XOR, set Noise to 0.4, then use 4 hidden layers of 8 neurons and let it train for a few thousand steps. Training loss keeps falling while test loss climbs: it's memorizing noise. The two lines on the chart separate.
5. Change the learning rate. At 0.01 it barely moves for a long time. Compare that with 0.5.

What each step does

Each training step has four parts. The network predicts a probability for every training point. A loss adds up how wrong those predictions are. Backpropagation works out how much each weight contributed to that loss. Then every weight moves a little in the direction that reduces it, by an amount set by the learning rate.

The idea of a playground like this comes from Google's TensorFlow Playground, which is worth trying too. For a network that learns to write rather than sort points, teach one on our home page.