What is a neural network?

Beginner · 19 min

Layers, weights, biases and activations — the whole forward pass, drawn on one handwritten digit.

Before thisMatrix multiplication

This lesson is part of a preview roadmap.

The engine around it is real — progress, notes and bookmarks all work — but the video has not been through the sourcing and review protocol a shipped lesson goes through. See the roadmap for what that means.

3Blue1Brown · 19 min

What is a neural network?

The player loads from youtube-nocookie.com when you press play. Nothing reaches YouTube before that.

Taught by 3Blue1BrownReport a problem with this video
Not started
Set this lesson's status

What you'll learn

  • A layer is a matrix multiplication, a bias, and a nonlinearity, applied in that order.
  • Weights are what the network learns; the architecture is what you choose before it learns anything.
  • Without the nonlinearity, any stack of layers collapses into a single layer.

The architecture, before any learning

This node is deliberately about the forward pass only. A network that has learned nothing still computes something, and understanding what it computes — a chain of matrix multiplications with a squashing function between them — makes the training story in the next nodes much shorter.

Where this sits

Everything so far has been fixed. The next node introduces the cost function and the idea of nudging weights downhill, which is where a network stops being a formula and starts being a thing that learns.

Check yourself

  1. 01What would happen to a ten-layer network if you removed every activation function?

    It would become mathematically identical to a one-layer network, because a composition of linear maps is itself linear.

  2. 02What is the bias term for?

    It shifts the threshold at which a unit starts responding, so a unit is not forced to pass through the origin.

Go deeper

The maths underneath

Notes

Private to you

Esc
↑↓ navigate↵ open? shortcuts

Keyboard shortcuts

Press Esc to close

Anywhere

⌘K
Search and commands
/
Search
?
This sheet
Esc
Close

Reading a lesson

J
Next lesson
K
Previous lesson
B
Bookmark