The residual stream is the central data structure of a transformer. It is a tensor of shape $[T, d_\text{model}]$ where $T$ is the sequence length and $d_\text{model}$ is the model dimension. Each block reads it, computes an additive update, and writes it back. The stream is never overwritten.
This post defines the stream formally, explains why the additive structure is load-bearing, and introduces the two interpretability primitives derived from it: the logit lens and direct logit attribution.
Demo: logit lens trajectory
The residual stream is the conveyor belt every block reads from and writes to. Each cell below shows the model's best guess at the next token if we stopped the belt at that layer and position. This is the logit lens. Pick a prompt, watch the predictions sharpen as you move down the belt.
What the logit lens is (and why it works at all)
The logit lens: take the residual stream at any intermediate layer, project it through the final unembedding matrix, see what token would win if the model stopped right there. Early layers produce noise. Middle layers narrow down to the right semantic neighbourhood. Late layers commit to a single answer. The fact this works at all is the deep observation: the residual stream stays "in the same language" as the final output throughout the network. Technique due to Nostalgebraist, 2020. Activation patterns shown here are realistic distilGPT2 traces, precomputed and visualized so the page loads instantly.
Each cell shows the model’s top-1 prediction at layer $\ell$, position $t$. Saturation = confidence. Click a cell for the full top-5 distribution and the residual norm.
Formal definition
For a transformer with $L$ blocks operating on $T$ tokens, the residual stream is the sequence of states ${X_0, X_1, \ldots, X_L}$, each $X_\ell \in \mathbb{R}^{T \times d_\text{model}}$.
Initial state: $X_0 = E + P$ where $E$ is the token embedding and $P$ is the positional encoding (or $X_0 = E$ for models with rotary/RoPE applied inside attention).
Block update:
\[X_{\ell+1} = X_\ell + \text{Attn}_\ell(\text{LN}(X_\ell)) + \text{MLP}_\ell(\text{LN}(X_\ell + \text{Attn}_\ell(\text{LN}(X_\ell))))\]The layer norms ($\text{LN}$) appear inside each sublayer in the pre-norm configuration used by GPT-2, Llama, and most modern models. Schematically:
\[X_{\ell+1} = X_\ell + \Delta_\ell^\text{attn} + \Delta_\ell^\text{mlp}\]Final read: logits = $\text{LN}(X_L) \cdot W_U$, where $W_U \in \mathbb{R}^{d_\text{model} \times V}$ is the unembedding matrix.
The architectural choice that everything depends on is the +: each $\Delta$ is added, never substituted.
Why additive matters
Compare the residual update with a non-residual update $X_{\ell+1} = f_\ell(X_\ell)$. Two structural problems with the latter:
Vanishing information. Any signal computed at layer $\ell$ must be re-encoded by $f_{\ell+1}, f_{\ell+2}, \ldots$ to survive. After 30 layers of arbitrary nonlinear transformations, layer-1 signals are effectively destroyed.
Vanishing gradients. Backprop multiplies gradients through every $f_\ell$. With layer-norm and sigmoid/tanh nonlinearities the gradient norm shrinks geometrically. Pre-2015 networks rarely trained stably past 20 layers.
Residual connections (He et al., 2015) solve both: $X_{\ell+1} = X_\ell + f_\ell(X_\ell)$ has an identity path from layer $\ell$ to $\ell+1$. Information and gradients flow through the + without distortion. This is what made GPT-2’s 48-layer and GPT-3’s 96-layer networks trainable.
For interpretability, the additive structure has a stronger consequence: the final state is literally a sum.
\[X_L = X_0 + \sum_{\ell=0}^{L-1} \Delta_\ell^\text{attn} + \sum_{\ell=0}^{L-1} \Delta_\ell^\text{mlp}\]Every component’s contribution is a linear term. This makes the residual stream linearly decomposable.
The logit lens
Final-layer logits are computed as $\text{LN}(X_L) \cdot W_U$. Because every $X_\ell$ lives in $\mathbb{R}^{T \times d_\text{model}}$, the same projection is well-defined at every layer:
\[\text{logits}_\ell := \text{LN}(X_\ell) \cdot W_U\]Nostalgebraist (2020) called this the logit lens. It returns the model’s “current best guess” at each intermediate layer, treating the partial residual stream as if it were the final state.
Empirically (visible in the demo above for “Paris is the capital of”):
- Layers 0–2: predictions are close to a unigram distribution. The model has not yet aggregated context.
- Layers 3–6: top-k starts ranking semantically related tokens (countries, cities).
- Layers 7–11: the correct answer (
France) reaches top-1 with high probability.
The lens is not exact, intermediate $X_\ell$ has different statistics than $X_L$, but it is informative and free. Refinements include the tuned lens (Belrose et al., 2023), which learns a per-layer affine correction.
Direct logit attribution (DLA)
Because $X_L$ is a sum, the logit for any vocabulary token $w$ is also a sum:
\[\text{logit}(w) = (X_0 \cdot W_U[:, w]) + \sum_{\ell=0}^{L-1} (\Delta_\ell^\text{attn} \cdot W_U[:, w]) + \sum_{\ell=0}^{L-1} (\Delta_\ell^\text{mlp} \cdot W_U[:, w])\]Each term is a scalar: how much that component pushed the prediction toward $w$. This is direct logit attribution.
Practical use:
# in TransformerLens
import transformer_lens as tl
model = tl.HookedTransformer.from_pretrained("gpt2")
tokens = model.to_tokens("Paris is the capital of")
logits, cache = model.run_with_cache(tokens)
# decompose final residual stream into per-component contributions
per_layer = cache.decompose_resid(layer=-1, return_labels=True)
# project each onto W_U for the answer token
answer_id = model.to_single_token(" France")
W_U = model.W_U[:, answer_id]
contributions = per_layer[0] @ W_U # one scalar per component
The largest entries in contributions identify the layers/heads/MLPs that drove the answer. DLA is the starting point for circuit analysis: keep zooming in (head → query/key/value → input neurons) until you have a mechanism.
- Pick a prompt — each one has 2–4 candidate tokens fighting to win.
- Watch the per-layer bars: how much each layer pushes each candidate up or down.
- The cumulative line on the right shows the running logit total — that's the actual logit that becomes the prediction.
- Hover or click a layer to see which circuit component (name-mover head, induction head, MLP) lives there.
- Hit play to accumulate one layer at a time and watch the answer emerge.
What to notice: the cumulative line is what the model's prediction actually depends on. Tiny early-layer bumps don't matter; the big middle/late layers (where name-movers, induction heads, and MLP fact-recall live) are doing the real work. Negative bars in late layers are negative name-movers — components that suppress the answer to keep calibration honest. This is the entire game of circuit attribution.
Subspaces and superposition
The stream has $d_\text{model}$ dimensions but generally encodes far more features than that. Components write to and read from subspaces of the stream, generally not axis-aligned.
Elhage et al. (2022, “Superposition”) characterize this: when features are sparse (most are off most of the time), a $d$-dim space can represent ~$d / \log d$ features by overlapping them. The cost is interference: reading one feature picks up small projections from others.
Consequences:
- Single neurons are typically polysemantic (active for multiple unrelated concepts).
- Single residual coordinates are not interpretable; directions are.
- Sparse autoencoders (SAEs) (Bricken et al., 2023; Templeton et al., 2024) recover interpretable directions by training an overcomplete dictionary on cached residual streams.
Provisional model: think of the stream as a high-dimensional space where many features overlap, recoverable by linear probes or SAEs but not by reading individual coordinates.
Reading the heatmap
In the demo, watch:
- Vertical evolution. The same column (token position) refines its top prediction across layers.
- Horizontal differences. Earlier positions are not trying to predict the next token, they are accumulating information that attention will later pull into the final position. Their logit-lens predictions are largely incidental.
- Final column saturation. This is where the actual next-token prediction happens. Saturation increases monotonically (with rare exceptions in degenerate prompts).
BOS and attention sinks
The first token’s residual stream typically accumulates “housekeeping” state. Attention heads with no relevant key in a given query often place mass on the BOS token as a default, the attention sink (Xiao et al., 2023). Templeton et al. (2024) found that BOS-position SAE features are systematically distinct from content-position features.
Treat the BOS column as anomalous when interpreting diagnostics.
The unifying claim
Every transformer mechanism can be expressed as “component $C$ reads from subspace $R$ of the residual stream and writes to subspace $W$ of the residual stream.”
Examples:
- Copy heads (attention): read content from position $i$, write the same content to position $j$.
- Induction heads: read a match-detection signal at the previous position, write a “copy this token” signal at the current.
- Factual-recall MLPs (Meng et al., 2022, ROME): read subject embedding from subject tokens, write attribute information back.
- IOI circuit (Wang et al., 2022): a chain of read/write heads juggling name and position information.
- Pick an example pair below — clean prompt vs. corrupt prompt that flip the model's answer.
- Click any cell in the layer × position heatmap to patch the residual stream at that (layer, token) from the corrupt run into the clean run.
- Watch the prediction probabilities shift. Hot cells = answer-relevant computation lives there.
- Or hit auto-scan to sweep every cell and let the heatmap paint itself.
What the heatmap reveals: hot cells form a small region — usually mid-to-late layers at the last subject token for factual recall, or at name positions for IOI. Click those cells: the answer flips. Click anywhere else: nothing happens. That's how Meng et al. found ROME and how the IOI circuit was traced.
The next two posts cover attention and MLPs as readers/writers in detail.
Resources
Foundational papers
- Deep Residual Learning for Image Recognition
- A Mathematical Framework for Transformer Circuits
- Interpreting GPT: the logit lens
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Toy Models of Superposition
- Scaling Monosemanticity