Your brain: 86 billion neurons. GPT-4: about 1.8 trillion “neurons”.
Same word. Not remotely the same thing, and this bothers me a little.
The AI version is embarrassingly simple. I could sketch one on a napkin and you’d get it in thirty seconds. Stack trillions of napkin sketches together, though, and you get something that writes poetry, debugs code, and beats grandmasters at chess. That mismatch is basically the whole story.
What a neuron actually does
An artificial neuron does one thing: takes a list of numbers, does some arithmetic, spits out a single number. That’s the entire spec.
Say the neuron’s job is to decide is this review positive? It gets three inputs:
0.9. How positive the words are.0.2. How many exclamation marks.0.7. How long the review is.
Multiply each by its own weight, which is just a number saying how much the neuron cares about that input:
- positivity ×
+0.8(a lot) - exclamations ×
+0.1(a little) - length ×
−0.3(negative: long reviews tend to be more critical)
Add them up. That sum is the neuron’s raw opinion.
Then (this is the part nobody explains) push that sum through an activation function before passing it on.
A neuron takes a list of numbers in, multiplies each by a weight, adds a bias, and squashes the result through an activation function. That's the whole thing. Play with the controls, watch the math update.
f(x) = max(0, x), silent when negative, linear when positive.How this demo works
Three inputs, three weights, one bias, all plain JS. Weighted sum → activation → result. ReLU: max(0, x). Sigmoid: 1 / (1 + e⁻ˣ). Tanh: tanh(x). The neuron's fill intensity scales with the post-activation output. The small curve plot to the right is the activation function with your current pre-activation marked on it.
Why the activation function matters
Okay, bear with me for a sec. Without an activation function, stacking layers is completely pointless. Every layer does the same kind of math, so they all collapse into one. You literally just get a line. No matter how many layers you pile on. Twelve layers, twelve hundred layers, same line.
The activation function is what lets a deep network represent things that can’t be drawn with a straight line. Bends the space. Curves become possible. Non-linearity, if you want the jargon.
Three you’ll meet:
- Sigmoid. Squishes any input into 0 to 1. An S-curve. Used to mean “probability”. Classic, elegant, still gets used in output layers. It was the default for a long time and earned its keep.
- Tanh. Same S, but outputs −1 to 1. A nice upgrade on sigmoid. Still pops up in places.
- ReLU.
max(0, x). Negative in, zero out. Positive in, pass it through unchanged. That’s the whole function.
ReLU’s simplicity is the superpower. Doesn’t saturate. Doesn’t vanish. And when a ReLU neuron outputs zero, it’s not half-committed. It’s silent. A clean, readable state. Which turns out to matter a lot when you’re trying to reverse-engineer what the thing is doing.
The dashed line above is the derivative of the activation, which is the number backprop multiplies by to push gradient through the neuron. Where the curve is flat, the derivative is zero, and learning stops in its tracks. That single observation is why ReLU beat sigmoid and why every training failure of the 2000s now has a name.
What “firing” actually means
Pop-sci loves “a neuron fires”. Great image, very evocative, gets people interested. Just not quite what’s happening here.
Artificial neurons are dimmer switches, not lightbulbs.
With ReLU you’re either silent (zero) or linearly active (some positive number proportional to how excited the neuron is). With softmax, used at the output layer, every neuron outputs a number and the set sums to 1.0. A probability distribution.
The useful thing isn’t on vs off. It’s the degree. Slightly, strongly, barely. That continuous answer is what lets networks handle fuzziness. Which is most of what real-world data is.
Layers, a panel of critics
One neuron doesn’t do much. A layer of them (512, 1024, 4096 neurons all looking at the same input, each with different weights) can represent rich structure.
Think of a layer as a panel of critics watching the same movie. One obsesses over pacing. One about dialogue. One only cares about cinematography. Each outputs a score. Together their scores paint a richer picture than any single critic could.
The next layer reads the scores and forms opinions about opinions. Each layer summarises the one below it into something more compact and more meaningful. That’s how depth becomes abstraction.
Move the inputs around. Each unique combination produces a different fingerprint across the 36 neurons. That fingerprint is what later layers actually see — not the raw inputs, not even the individual neurons, but the shape of the firing pattern across the whole layer. Concept = pattern. Pattern = vector. That’s the whole game.
Bias, the neuron’s default mood
One last thing, almost always skipped: biases.
Every neuron has a bias, a number added to its weighted sum before the activation. The neuron’s default opinion before it sees any input. Positive bias? The neuron is eager; it takes real negative input to shut it up. Negative bias? It’s reluctant; only strong signals wake it.
Biases let each neuron have its own firing threshold. Without them, every neuron would need its inputs to sum to exactly zero to stay silent, which is far too rigid.
The MI connection
This whole setup (weights times inputs, plus bias, through an activation) is what mechanistic interpretability is trying to read at scale.
Finding features = figuring out which neurons fire, on which inputs, why. Finding circuits = figuring out which neurons talk to which other neurons, and what that conversation computes.
Everything else in mechanistic interpretability is built on this one tiny unit. Wire a million of them together, every neuron talking to every other neuron, and things get weird.