Picture this: you show a trained network a photo of a face.
Layer 1 sees edges. Diagonal lines, curves, horizontal stripes. Layer 2 sees eye corners, nose tips, ear lobes. Layer 5 sees this person. Their mood. Whether they’re wearing glasses.
Same pixels going in. Totally different lens at every level. Each layer is watching the one below it and writing down what it noticed. It’s like a game of telephone, except the message gets smarter at every hop.
That’s what depth is. That’s literally why “deep” learning is deep.
What a layer actually is
A layer is a group of neurons that all receive the same inputs and all produce outputs that feed forward together.
Every neuron in a layer:
- Takes all the activations from the previous layer as input
- Applies its own weights to them
- Produces its own single activation value
- Passes that to every neuron in the next layer
Net result: each layer transforms its inputs into a new representation. Same information, viewed from a different angle, with different things highlighted.
Slide through the layers of a small vision model. At the bottom you see pixels. At the top you see categories. In between, the network turns the first into the second.
How this demo works
A real pretrained model is too heavy to ship inline, so this is an illustrative pipeline: Layer 0 draws the image on a canvas; Layer 1–2 apply Sobel edge detection; Layer 3–5 show animated feature-map tiles; Layer 6–8 draw a class-activation-style heatmap overlay; Layer 9–11 blur toward "concept space"; Layer 12 shows the final class distribution. The progression mirrors what happens in a real ConvNet (see Zeiler & Fergus 2013, Olah et al. 2017). Swap the image, the predictions change accordingly.
Input and output layers
Input layer. No math. Just receives the raw data and passes it forward. Images: pixel values. Text: token embedding vectors. Tabular data: feature values. The input layer is pure format conversion. “Here’s the world, in number form.”
Output layer. The final decision. Classification: a probability over each class (softmax). Language modelling: a probability over every word in the vocabulary (often 50,000+ options). Regression: a single continuous number.
Everything interesting happens between these two.
Hidden layers, the middle of the machine
Don’t let the word fool you. “Hidden” just means “not the input, not the output.” These layers are where all the action happens, and where MI spends basically all its time.
In early vision models, researchers noticed the hidden layers had a striking, almost biological structure. Here’s the rough gradient:
Layer 1: Gabor filters
Neurons respond to oriented edges. Horizontal, vertical, 45-degree. Nobody programmed this. Every model trained on natural images independently discovers it. Emerges from the statistics of images themselves.
Layer 2: Textures and simple shapes
Combinations of edges form textures. Checkerboards. Crosshatches. Dots. Neurons looking for local patches that match a pattern.
Layer 3–4: Object parts
Eyes. Wheels. Leaves. Neurons now looking for parts of real objects. Things that have names in human language.
Layer 5–7: Objects and scenes
Full objects, faces, specific categories. High-level human concepts represented in the network’s internal language.
This progression is called hierarchical feature extraction, and it appears in every deep network trained on natural data. Images, text, audio. The depth lets the model compose simple features into complex ones, repeatedly.
Click an object in the bottom row, then trace your way up. Every face is a composition of eye-detectors and mouth-detectors. Every eye-detector is a composition of curve-detectors and edge-detectors. The whole thing is recursive: one small library of features, reused at every level of detail.
What layers do in language models
In text transformers, the same hierarchy exists, but it’s harder to see because language doesn’t have the obvious spatial structure of images.
Research into transformer layers has found patterns like:
- Early (1–4). Local, syntactic. Nearby word relationships. Part of speech. Simple co-occurrence.
- Middle (5–16). Syntactic structure. Subject-verb relationships. Clause boundaries. Entity tracking.
- Late (17–final). Semantic, pragmatic. What the text means. Who’s saying what to whom. What should come next.
Not perfectly clean (features mix across layers), but the gradient from syntax to semantics is consistent and has been verified by probing experiments across many models.
The logit lens
A cool MI technique that reads out the model’s best guess at each layer, before the final output. Early layers, the guess is mostly garbage. Middle layers, it starts approaching the right semantic category. Late layers, it converges on the final answer. Shows you how the model builds its answer progressively.
Depth vs width
Width = more neurons per layer. More “workers” doing parallel analysis at each step.
Depth = more layers. More steps of abstraction before the final answer.
Both help, but differently.
Width gives the model more capacity to represent complex things at each level of abstraction. Depth gives the model more steps to compose simple patterns into complex ones.
Modern networks are both wide and deep. GPT-4 is estimated to have around 120 layers. Many of those layers have ~25,000 neurons each. Which, you know, is a lot.
For interpretability: more layers means more places for information to transform. Also means there’s more “room” for information to be stored in intermediate representations. Which is one reason large models are more capable but also harder to interpret.
Try the Spiral dataset with 0 hidden layers — a single linear boundary can’t separate it, no matter how long you train. Now bump depth up to 3 and watch the boundary curl. Every additional layer is one extra fold the network can put into space. Width adds patience; depth adds expressiveness.
Skip connections, the highway system
In modern networks (including all transformers), there’s a trick that changed everything: residual connections. Also called skip connections.
Instead of each layer replacing the previous layer’s output entirely, it adds to it. The output of layer N+1 = what layer N produced + what layer N+1 computed.
Sounds small. It’s enormous.
Means information can flow directly from early layers to late layers without passing through every layer in between. Early features don’t get “forgotten” or overwritten.
Also means each layer can focus on adding a small correction, rather than computing everything from scratch. Makes training much more stable and allows much deeper networks.
The MI connection
Understanding what each layer does is one of the central projects of mechanistic interpretability. Not “layer 7 does something useful”. Exactly what. Which features live in which layers. Which operations happen where. When we know that, we can start to decompose a model’s behaviour the same way you’d decompose a program into functions.
Okay, one more thing worth naming. Every weight in every layer we’ve talked about was set by a single procedure: gradient descent. I’ll write about how that actually works in the next blog.