Explainer16 min

Bayesian decoding and the Kalman filter

A decoder as an estimator that blends what it expected with what it measured, weighted by how much it trusts each. The population vector, the Kalman filter, and why intracortical cursor control works.

AssumesClassification done honestly SpineDecoding / signal processing / MLComputational neuroscience

On this page16 min

First, the population vector
Why snapshots are not enough
The Kalman filter in two lines
Fitting it
Why it works for intracortical and less for EEG

A classifier picks one of a few labels. A cursor decoder has to produce a continuous position, sixty times a second, from neural activity that is noisy and only loosely related to movement. The trick that makes this work is to stop asking “what does the neural data say right now” and start asking “given what I believed a moment ago and what I just measured, what should I believe now.” That is Bayesian estimation, the Kalman filter is its simplest form, and it has been the decoder behind most intracortical cursor control since the early 2000s.

Kalman cursor decoderSimulated signal

This interactive needs JavaScript. If you are reading a printout, the caption describes what it shows.

Figure 1. A cursor decoder as a Kalman filter. Blue is the intended path; yellow integrates the noisy velocity observations directly; green is the Kalman estimate. Change how much the filter trusts its observations.

Try this

Raise the observation noise. The naive path wanders off; the Kalman path smooths but lags.
Lower the process noise Q. The filter trusts its own prediction more: smoother, slower to turn.
Raise Q. It trusts observations more: faster, noisier. There is a sweet spot, and it depends on the noise.

First, the population vector

Motor cortex neurons fire more for movements in a preferred direction, following roughly a cosine of the angle between the movement and that direction. Georgopoulos in the 1980s showed that if you take each neuron’s preferred direction, weight it by how fast the neuron is firing now, and add the vectors up, the sum points where the arm is about to move. That population vector was the first real neural decoder, and it is a snapshot method: each moment’s estimate uses only that moment’s firing.

Each neuron’s rate follows a cosine of the movement direction:

r_i = b_i + m_i \cos(\theta - \theta_i)

Here $r_i$ is the firing rate of neuron $i$ , $b_i$ its baseline, $m_i$ its modulation depth, $\theta$ the movement direction, and $\theta_i$ the neuron’s preferred direction. The population vector weights each preferred direction by its rate:

\hat{\mathbf{v}} = \sum_i (r_i - b_i), \mathbf{p}_i

Here $\mathbf{p}_i$ is the unit vector for neuron $i$ ’s preferred direction and $\hat{\mathbf{v}}$ estimates the movement vector.

Why snapshots are not enough

Spike counts in a 50 ms bin are noisy: a neuron firing at 20 per second gives one spike per bin on average, so the count is zero or one or two by chance. A snapshot decoder inherits all of that noise, and a cursor driven by it jitters. You could average over longer bins, but then the cursor lags. What you want is to use the past without being stuck in it.

The Kalman filter in two lines

Keep a belief about the state (position and velocity) and an uncertainty about it. Each time step:

In probability terms, each step multiplies the prior belief by the likelihood of the new measurement:

p(x \mid z) \propto p(z \mid x), p(x)

Here $x$ is the cursor state, $z$ is the neural observation, $p(x)$ is what was believed before, and $p(x \mid z)$ is the updated belief.

Predict. Move the belief forward with a model of how cursors move: position advances by velocity, velocity mostly persists. The uncertainty grows, by an amount Q that says how unpredictable movement is.

Update. Take the new neural observation, which the model says should relate linearly to velocity with noise R. Compare it to what the prediction said the observation should be. Move the belief toward the observation by a fraction called the Kalman gain, which is large when the prediction is uncertain and the observation is reliable, and small when the reverse. The uncertainty shrinks.

That is the whole algorithm. The gain is computed from Q and R, so you never hand-tune a smoothing constant; you state how noisy the world and the sensors are and the optimal blend follows. In Figure 1 the sliders are Q and R.

Predict before you look

A participant's neurons become noisier over the course of a session. If R is fixed, what happens to the cursor?

Jittery. The filter’s trust in observations is set by R. If the real noise rises but R does not, the filter keeps weighting noisy observations as if they were clean. Real systems re-estimate R periodically, or adapt it online, which is one form of the “closed-loop decoder adaptation” that made intracortical BCIs practical.

Fitting it

From a calibration block where the participant watches or attempts movements with known velocity: fit the observation model (a linear regression from velocity to each neuron’s firing rate) and estimate R from its residuals; estimate the movement model and Q from the kinematics. Twenty minutes of data suffices. Then run it. The BrainGate group’s refinement, ReFIT, refits the model using the participant’s intended direction (toward the target) rather than the cursor’s actual path, and roughly doubles performance; it is the single most important practical trick in the field.

Why it works for intracortical and less for EEG

Intracortical arrays give a hundred channels of spikes, each a reasonably direct report of a few neurons’ intent about movement direction. The linear model between velocity and firing holds well enough. EEG gives a few tens of channels of blurred oscillatory power whose relation to continuous movement is weak, so continuous EEG decoding gives correlations of 0.3 to 0.5 rather than 0.8 to 0.9. The Kalman machinery is the same; the signal is not.

Stop here if you like. Believe, predict, measure, blend. That is the Kalman filter and the reason implanted people move cursors. The deep dives are the equations and the frontier.

Deep dive The equations 5 min

State x (position, velocity), transition A, process noise Q. Observation z (firing rates), observation matrix H, observation noise R.

Predict moves the belief through the movement model and grows the uncertainty:

\hat{x} \gets A\hat{x}, \quad P \gets APA^\top + Q

The gain weights the prediction against the observation:

K = PH^\top(HPH^\top + R)^{-1}

Update blends the observation in and shrinks the uncertainty:

\hat{x} \gets \hat{x} + K(z - H\hat{x}), \quad P \gets (I - KH)P

P is the belief’s covariance. Everything in Figure 1 is these five lines in JavaScript. For a hundred neurons, H is 100 × 4 and R is 100 × 100, and the matrix inverse is the whole computational cost, which a laptop does thousands of times a second.

Deep dive Beyond linear: what the current decoders do 3 min

Firing rates are not Gaussian and not linear in velocity. Extensions: a Poisson observation model (the point-process filter), an unscented or particle filter for nonlinearities, and, since about 2020, recurrent neural networks trained end to end from neural data to kinematics, which win on offline benchmarks like the Neural Latents Benchmark by learning the neural population’s own low-dimensional dynamics. In closed loop, with a person adapting to the decoder, simple Kalman variants remain surprisingly competitive, and the interaction between a learning user and a learning decoder is a research field of its own.

Recall

What does the Kalman gain represent and what determines it?

How far the belief moves toward the new observation. It is large when the prediction is uncertain (P large) and observations are reliable (R small), and it is computed from Q and R rather than hand-tuned.

Recall

Why does a population vector decoder produce a jittery cursor and how does the Kalman filter fix it?

It uses only the current bin's noisy spike counts. The Kalman filter carries a belief forward with a movement model and blends each new observation in proportion to its reliability, smoothing without simply lagging.

Recall

What is ReFIT and why did it matter?

Refitting the decoder using the participant's intended direction (toward the target) rather than the actual cursor path during calibration. It roughly doubled closed-loop performance and made intracortical cursor control practical.

Explain it to your roommate

Explain what this page was about to your roommate in three sentences. No jargon they would not know.