The Transformer, Demystified: A Factory Floor That Runs on Language

A decoder-only transformer is a stack of $N$ identical blocks operating on a sequence of $T$ token vectors. Input: a token sequence. Output: a probability distribution over the vocabulary for the next token. The internals consist of six stations on a conveyor belt called the residual stream.

This post defines each station precisely, explains why the architecture replaced RNNs, and sets up the vocabulary used in the rest of the series.

Tour the factory

Click a station, press play, watch one token’s vector traverse the pipeline. The structure is identical for every modern LLM: GPT-2, GPT-3, GPT-4, Claude, Llama, Gemini.

The six stations

1. Tokenizer. Maps a string to a sequence of integer IDs using a fixed vocabulary $V$ (typical sizes: 50,257 for GPT-2, 100,277 for GPT-4, 128,000 for Llama 3). Modern tokenizers use byte-pair encoding (BPE) or its variants. Example: "unhappily" → ["un", "happ", "ily"] → [403, 7829, 6148].

2. Embedding. A learned lookup table $W_E \in \mathbb{R}^{V \times d_\text{model}}$ maps each token ID to a $d_\text{model}$-dimensional vector. Common sizes: $d_\text{model} = 768$ (GPT-2 small), 4,096 (Llama 3 8B), 12,288 (GPT-3 175B). The $i$-th row of $W_E$ is the embedding for token $i$.

3. Self-attention. At each position $t$, the model computes weighted sums over all positions $\le t$ (causal mask). Weights are derived from query/key dot products. Output: a new vector at each position that has read from earlier positions. Multiple heads run in parallel ($n_\text{heads} = 12, 32, 96, \ldots$), each in a lower-dimensional subspace.

4. MLP. A position-wise feed-forward network: two linear layers with a non-linearity (GeLU, SwiGLU). Hidden dimension is typically $4 d_\text{model}$. Same shape in, same shape out. Operates independently on each position.

5. Repeat. One block = attention + MLP + residual connections + layer norms. Stack $N$ of them. GPT-2 small: $N=12$. Llama 3 8B: $N=32$. GPT-3 175B: $N=96$. GPT-4 (rumoured): $N \approx 120$.

6. Unembedding. Multiply the final residual stream vector by $W_U \in \mathbb{R}^{d_\text{model} \times V}$ to produce logits. Apply softmax to get a probability distribution over the vocabulary. Sample → next token.

Total parameters scale roughly as $N \cdot d_\text{model}^2 \cdot 12$ for the standard recipe.

The residual stream

In the literature this conveyor belt is the residual stream. Each block’s output is added to its input, not substituted:

Three consequences follow directly from this additive structure:

1. Gradient flow. Information from layer 0 reaches layer $N$ along the identity path. This is why deep transformers train in the first place.
2. Per-block delta. Each block computes a correction, not a full representation. Easier optimization target.
3. Linear decomposability. The output is a sum of token embeddings + each block’s contribution. Mechanistic interpretability uses this directly: ablate one head, see the difference in the logits.

Elhage et al. (2021, “Mathematical Framework”) formalize this view. The residual stream is the central object in the rest of this series.

Why transformers replaced RNNs

Recurrent neural networks (RNNs) process tokens sequentially, maintaining a hidden state $h_t = f(h_{t-1}, x_t)$. Two structural problems:

1. No parallelism. Token $t$ depends on $h_{t-1}$. Cannot batch positions on a GPU.
2. Vanishing dependencies. Gradients through 1,000 timesteps decay or explode. LSTMs and GRUs help but do not solve it at scale.

Transformers (Vaswani et al., 2017) replace recurrence with self-attention: every position reads from every other position in parallel via a single matrix multiplication. Sequence length $T$ → $O(T^2)$ time, but fully parallel within a sequence and across batch dimensions. On modern GPUs the constant factor wins.

Decoder-only

The original 2017 paper had two halves: an encoder (reads input) and a decoder (writes output), wired for machine translation. For autoregressive text generation, only the decoder is needed.

A decoder-only transformer uses causal self-attention: position $t$ may only attend to positions $0, 1, \ldots, t$. This enforces left-to-right generation and matches the next-token-prediction training objective.

This series is exclusively about decoder-only transformers. (Encoder-only models like BERT are used for classification and embedding tasks, not generation.)

Autoregressive generation

The transformer produces one probability distribution per forward pass. To generate text:

input:  "The cat sat on the"          → logits → sample → "mat"
input:  "The cat sat on the mat"      → logits → sample → "."
input:  "The cat sat on the mat."     → logits → sample → "<eos>"

This is autoregressive decoding. Each generated token is appended to the input and the model runs again. Sampling strategies (greedy, top-k, top-p, temperature) determine which token gets picked from the distribution. KV-caching avoids recomputing attention over previous tokens, making the per-step cost roughly $O(T)$ rather than $O(T^2)$.

Key dimensions, by model

Memorize one row (GPT-2 small is most common in MI papers) and use it as the reference point.

What’s ahead

Subsequent posts cover each station in mechanistic detail:

• Tokens. BPE, vocabulary quirks, the bugs they cause.
• Residual stream. The most important MI primitive; logit lens; linear decomposition.
• Attention. QK/OV decomposition; induction heads; IOI circuit.
• MLPs. Key-value memory framing; superposition; sparse autoencoders.
• Full forward pass. End-to-end with real GPT-2 numbers.

The next post is on tokenization.

Resources

Foundational papers

• Attention Is All You Need
  Vaswani et al., 2017 · the original transformer paper
• A Mathematical Framework for Transformer Circuits
  Elhage et al., Anthropic 2021 · residual-stream view used throughout this series
• Language Models are Unsupervised Multitask Learners
  Radford et al., OpenAI 2019 · GPT-2 architecture and training
• Language Models are Few-Shot Learners
  Brown et al., 2020 · GPT-3, scaling laws in practice

Tutorials and code

• The Illustrated Transformer
  Jay Alammar · diagram-led walkthrough of the original architecture
• nanoGPT
  Andrej Karpathy · ~300-line PyTorch implementation of GPT-2
• Let's build GPT: from scratch, in code, spelled out
  Karpathy · 2-hour lecture; pairs with nanoGPT
• TransformerLens
  Neel Nanda et al. · the standard MI library; load any HF model and inspect activations
• A Comprehensive Mechanistic Interpretability Explainer
  Neel Nanda · glossary and recommended reading order