Who's Doing This, and Why Now?

12 min read

Here’s a fun piece of history.

In the 1950s a small group of biologists set out to do something that sounded, at the time, a little mad: describe exactly how a single neuron in a living animal works. Not the nervous system. Not neurons in general. One neuron. What signal, in response to what input, with what timing.

That one obsession grew into all of modern neuroscience.

We’re at something like that moment with AI. Early 1950s, give or take. Handful of people. Wild energy. Real progress.

The players

MI is small. Delightfully small. Handful of central teams, a growing cloud of academics, and a surprising number of solo researchers doing real work. The field is tight-knit enough that you’ll see the same names on most of the landmark papers. Which is a huge opportunity, by the way. Here’s the rough map.

Interactive · Who to follow

The people and teams whose work you'll actually read if you go deep. Hover (or tap) for their core contribution and what to read first.

Not an exhaustive list, mechanistic interpretability is small enough that most active researchers publish on the Alignment Forum or transformer-circuits.pub. Read 10 posts there and you'll know half the field.

How this demo works

Hard-coded profile data. Each tile is a CSS card; the hover reveals the "read first" link via a transition. If you have a person to add, edit _includes/demos/researcher-grid.html.

Anthropic Interpretability Team

The team most associated with the current wave. Founded by Chris Olah, one of the original Distill.pub researchers. They produced:

  • The circuits framework (2020)
  • A Mathematical Framework for Transformer Circuits (2021)
  • In-context Learning and Induction Heads (2022)
  • Toy Models of Superposition (2022)
  • Towards Monosemanticity (2023), the SAE breakthrough
  • Scaling Monosemanticity (2024), applied to Claude 3 Sonnet

Explicitly motivated by AI safety: if we can’t understand what the model has learned, we can’t verify it’s safe. Most of their work lives on transformer-circuits.pub. Beautifully written, often interactive. Worth reading linearly if you want the field’s intellectual history. They publish rarely and thoroughly; each release is a small event.

Neel Nanda · DeepMind + independent

Probably the most accessible voice in MI. Produces extremely well-explained research, runs workshops, wrote 200 Concrete Open Problems. His work on grokking and circuits is foundational. His TransformerLens library is what most researchers actually use.

If Anthropic’s style is “careful flagship releases”, Nanda’s style is “constant, generous, hands-on output”. YouTube walkthroughs. Twitter threads explaining new papers. The ARENA curriculum. Paper-replication exercises. A huge fraction of the current generation of MI researchers got started by working through his material.

Academic research groups

Short, non-exhaustive list:

  • MIT (Jacob Andreas, David Bau). Language-model representations, model editing (ROME, MEMIT).
  • Berkeley (Jacob Steinhardt). Training dynamics, emergent capabilities, benchmarks.
  • Harvard / Kempner. Broader ML-theory work, feature learning.
  • Northeastern (Bau Lab). Causal tracing, factual knowledge localisation.
  • NYU, Princeton, CMU. Various smaller efforts on probing, circuits, evaluation.

Academia contributes mostly methods and theory rather than scale, and that’s genuinely load-bearing. Labs don’t always have Claude to poke at, but they have the time to develop the cleaner formal tools everyone ends up using.

Independent researchers

One of the beautiful things about MI: you can contribute without institutional affiliation. The Alignment Forum, LessWrong, and EleutherAI have produced real research from independent contributors. The field is young and open. Novel findings from unknown researchers show up there regularly and get taken seriously.

A few that matter:

Not at a lab and want to get published? This is the culture you’re entering. More forum than conference.

The tools you’ll actually use

Interactive · What tool should I pick?

A non-exhaustive tour of the software you'll actually use. Filter by what you want to do.

How this demo works

Each tool has a tag set. Clicking a filter shows only tools with that tag. "All" resets. Tools and descriptions are hand-curated; update them in _includes/demos/mi-tools.html.

Three rough tracks for picking one:

  1. Learning MI from zero. Start with ARENA (structured exercises), then pick up TransformerLens and follow Neel Nanda’s tutorial to replicate IOI or induction heads on GPT-2. You’ll hit every major concept.
  2. SAE research. SAELens plus Neuronpedia. Train a small autoencoder, find interesting features, label them, compare against published lists.
  3. Frontier model work. nnsight is the only realistic option outside the frontier labs themselves, because it gives you intervention access to hosted large models.

The timeline of discoveries

Click a milestone.

Interactive · Timeline of discoveries
How this demo works

Nine events are stored as a JS array. Clicking a node renders the detail panel and updates the active state. No framework, just document.createElement and a template literal. On mobile the panel stacks under the timeline.

Headline: the field went from “curves detectable in vision models” (2019) to “millions of features decomposed in production Claude” (2024) in five years. That compression is almost unprecedented in ML research.

Why now, the urgency

Not purely academic. There’s a race.

Figure out how to interpret AI systems before they’re making high-stakes autonomous decisions, we can verify they’re safe. Miss that window, interpretability keeps lagging capability, we’re in a much harder position.

This is why Anthropic, DeepMind, and others put serious resources into it. Not just interesting science. One of the most important technical bets for making AI go well. Concrete things MI would enable if it were solved:

  • Detecting deceptive behaviour before deployment, not during incidents.
  • Auditing a model for dangerous capabilities (bioweapons knowledge, cyber tools, persuasion) mechanistically, not just behaviourally.
  • Explaining individual decisions in high-stakes applications (medicine, law, hiring).
  • Verifying that fine-tuning didn’t introduce hidden backdoors.

None of this requires fully solving MI. Partial progress helps. That’s why the field is so alive right now: every small win is immediately useful.

You can do this too

Field is young enough that smart outsiders contribute. Methods are learnable. Papers are readable. Code is open source.

That’s the why. The how is a longer story: what a feature actually is, what a circuit actually is, how we find them. I’ll write about the raw materials (neurons, weights, layers, and training) in the next blog.

Research referenced in this post