Explainer14 min

Neural coding

Rate codes, temporal codes, population codes, tuning curves, and low-dimensional manifolds. The theory that says what a decoder can and cannot read from a set of neurons.

AssumesProject E: Large-scale neural data Bayesian decoding and the Kalman filter SpineComputational neuroscienceDecoding / signal processing / ML

On this page14 min

Tuning curves
Rate versus timing
Population codes and the manifold
Sparse codes and grandmother cells
What the code says about ethics

When you brushed the cockroach leg and heard the firing rate jump, you heard a rate code: intensity encoded as spikes per second. When you built the Kalman decoder, you read a population code: direction encoded across many neurons at once. The question of how neurons represent the world, “the neural code,” is the theoretical heart of the field and the thing every decoder is implicitly a theory of. Let’s lay out the candidates and what the data say.

Tuning curves

Record one neuron while varying a stimulus and plot its rate against the stimulus value. A retinal ganglion cell’s rate against light intensity; a visual cortex cell’s rate against edge orientation; a motor cortex cell’s rate against movement direction. The resulting curve is the neuron’s tuning. Most cortical tuning curves are broad (a motor neuron responds to a wide range of directions, most strongly to one), noisy (the same stimulus gives a different count each trial), and overlapping (neighbouring neurons prefer neighbouring values). Broad, noisy, overlapping tuning is not a flaw; it is the design that makes population codes robust.

A typical tuning curve is a baseline plus a bump at the preferred value:

f(\theta) = b + m,\exp\left(-\frac{(\theta-\theta_0)^2}{2\sigma^2}\right)

Here $f$ is the firing rate, $\theta$ the stimulus value, $\theta_0$ the preferred value, $b$ the baseline rate, $m$ the modulation depth, and $\sigma$ the tuning width.

Rate versus timing

The claim

The evidence for it

What it implies for a decoder

Rate code

Information is in spikes per unit time

Nearly every tuning curve ever measured; counts look roughly Poisson

Count spikes in a window. This is what works today

Temporal code

Information is in precise spike times

Real in the auditory system, in the retina, and in first-spike latency

You would need millisecond resolution and a phase reference

Population code

Information is spread across many neurons at once

Population vectors, manifolds, and every working cursor BCI

More electrodes beat better electrodes, up to the manifold dimension

Sparse code

A few highly selective cells carry it

Concept cells in human medial temporal lobe

Would make small arrays viable, if you found the right cells

These are not exclusive and the cortex almost certainly uses several at once. The practical fact is that low-dimensional population structure is why a hundred electrodes can control a cursor at all.

Figure 1. Four candidate codes and where the evidence actually stands. Every decoder you have built is a bet on one of these rows, usually the first.

The rate code says information is in how many spikes per unit time. Its evidence: nearly every tuning curve ever measured is a rate curve; rates decode movement and stimuli well; and the noise in spike counts looks roughly Poisson, as if only the rate were controlled.

Spike counts in a fixed bin follow roughly a Poisson distribution around the rate:

P(k) = \frac{\lambda^k e^{-\lambda}}{k!}

Here $k$ is the spike count and $\lambda$ the mean count. Variance equals the mean ( $F = \sigma^2/\mu \approx 1$ ), so single-trial counts are noisy by nature.

The temporal code says the precise timing of spikes matters too. Its evidence: auditory neurons phase-lock to sound waves at sub-millisecond precision; the timing of the first spike after a stimulus carries information before any rate could be estimated; in the hippocampus, when a cell fires relative to the theta rhythm encodes position within its place field (phase precession).

The honest summary: rates carry most of the information a decoder can currently use, timing carries more in specific systems (hearing, spatial navigation), and the brain probably uses both to different degrees in different places. Every practical BCI decoder is a rate decoder with 20 to 50 millisecond bins, and it works.

Predict before you look

A neuron's firing is so noisy that its count for a repeated stimulus varies by a factor of two. How can a population of such neurons encode the stimulus precisely?

By pooling across neurons in a single trial. If a hundred neurons each report the stimulus noisily and their noise is independent, the population average has a tenth the noise of any one. Independent noise averages as the square root of the count:

\sigma_{pop} = \sigma/\sqrt{N}

so $1/\sqrt{100} = 1/10$ . That is why decoding accuracy rises with neuron count in the Allen data, and why the noise correlations between neurons (which limit how much pooling helps) are a central topic in coding theory.

Population codes and the manifold

Take N neurons and plot each trial’s activity as a point in N-dimensional space. For a hundred neurons, a hundred dimensions. The remarkable empirical finding of the last fifteen years is that the points do not fill the space; they lie on or near a low-dimensional surface, a manifold, of perhaps ten dimensions, and movement of the population along that surface is what encodes the variable of interest and what evolves over time. PCA on your Allen data found it. Motor cortex during reaching traces smooth, rotational trajectories on its manifold that look more like a dynamical system than like a lookup table of muscle commands.

Each trial is then a mean plus a few weighted patterns:

x \approx \mu + \sum_{j=1}^{d} c_j v_j

Here $x$ is the trial’s activity vector, $\mu$ the mean, $v_j$ the principal patterns, $c_j$ the trial’s weights, and $d$ the manifold dimension, near ten for a hundred neurons.

For decoders this is good news. A hundred noisy neurons carry only ten dimensions of signal, so the decoder’s job is to find the ten, and a linear method often can. It is also why decoders survive losing neurons: the manifold is redundantly represented. And it is why the newer decoders learn the manifold’s dynamics rather than mapping neurons to kinematics directly.

Sparse codes and grandmother cells

At the other extreme, some neurons in the human medial temporal lobe respond to one specific person or concept, across pictures, names, and voices. Sparse, selective coding exists. It is efficient for memory and hopeless for movement, and the brain appears to use dense population codes in sensorimotor areas and sparser ones as you move toward memory. A decoder for speech in ventral premotor cortex reads a dense code; a hypothetical decoder of “what are you thinking about” would face a sparse one, which is one reason it is not close.

What the code says about ethics

The codes we can read are motor intent and sensory content, from dense population activity in the relevant cortex, with electrodes in that cortex. Reading thoughts, memories, or intentions beyond movement is not a matter of better electrodes; it would require decoding sparse, distributed, poorly understood codes from areas nobody records from in humans outside of epilepsy surgery. Neuroethics discussions that assume mind-reading is near are not describing this field. Discussions about who owns your motor cortex’s recordings are.

Stop here if you like. Tuning, rate versus timing, population manifolds, sparse codes. That is what a decoder is a theory of.

Deep dive Fisher information and why broad tuning is good 3 min

How precisely a population can encode a variable is measured by Fisher information: roughly, how steeply firing changes with the variable, divided by the noise, summed over neurons. Sharp tuning curves give steep slopes but only near their peaks; broad curves give shallower slopes over a wide range. For a fixed number of neurons and realistic noise, moderately broad tuning maximizes information over the whole range, which is what cortex has. Noise correlations between neurons can raise or lower the total depending on their structure; that is an active area.

For one variable encoded by independent neurons with Gaussian noise, precision is slope squared over noise, summed:

I(\theta) = \sum_i \frac{(f_i'(\theta))^2}{\sigma_i^2}

Here $f_i'$ is how steeply neuron $i$ ’s rate changes with the variable and $\sigma_i^2$ is its noise variance. Broad curves keep moderate slopes over a wide range, which is why they win for a fixed neuron count.

Deep dive Neural dynamics as the code 3 min

An influential view of motor cortex (Churchland, Shenoy, and others) holds that its population activity is best understood not as representing movement parameters but as a dynamical system whose evolution generates the muscle commands. Preparation sets an initial state; the dynamics unfold. Decoders built on this view (latent factor models, recurrent networks trained on population activity) fit held-out neurons and predict behaviour better than tuning-curve models. The Neural Latents BenchmarkThe Neural Latents BenchmarkPei, Deng, O'Shea, et al. · NeurIPS · 2021Standard monkey datasets with held-out evaluation, the right place to compare a decoder against the field. measures exactly this.

Recall

What is the evidence for rate coding and for temporal coding, in one sentence each?

Rate: nearly every tuning curve is a rate curve and rates decode behaviour well. Temporal: auditory phase-locking, first-spike latency information, and hippocampal phase precession show timing carries information in specific systems.

Recall

What is a neural manifold and why is it good news for decoders?

A low-dimensional surface in the space of population activity on which the data lie; the decoder need only find those few dimensions, linear methods often can, and redundancy makes decoding robust to losing neurons.

Recall

Why is 'reading thoughts' not a matter of better electrodes?

Decodable codes are dense population codes for motor intent and sensory content in the relevant cortex. Thoughts and memories involve sparse, distributed, poorly understood codes in areas not recorded from in humans outside epilepsy surgery.

Explain it to your roommate

Explain what this page was about to your roommate in three sentences. No jargon they would not know.