Project A: The SSVEP speller

You are skimming: the title, the first figure, and the short version. Switch to Read in the header for the full page, or Deep to open every deep dive.

The signal

The response sits at the flicker frequency and its harmonics, over occipital electrodes (Oz strongest, O1 and O2 next). Amplitude is a few microvolts, which is small, but it is a pure tone, and pure tones are what spectra find best. Two to four seconds of data is enough to decide among four to eight frequencies for most people.

The decoder

Canonical correlation analysisCanonical correlation analysis (CCA)Finding the combination of channels that correlates best with reference sinusoids at each candidate frequency; the standard SSVEP decoder. Glossary entry is the standard. For each candidate frequency, build reference signals: sine and cosine at f, at 2f, and at 3f. Find the combination of EEG channels that correlates most with the combination of those references. The frequency with the highest correlation wins. With one channel it reduces to fitting the references by least squares and comparing the fit, which is what the simulator does. With several occipital channels the multichannel version is meaningfully better.

import numpy as np
from sklearn.cross_decomposition import CCA

def refs(f, fs, n, harmonics=3):
    t = np.arange(n) / fs
    return np.column_stack([g(2*np.pi*h*f*t) for h in range(1, harmonics+1) for g in (np.sin, np.cos)])

def decode(X, freqs, fs):           # X: samples × channels, a 2–4 s window
    scores = []
    for f in freqs:
        cca = CCA(n_components=1).fit(X, refs(f, fs, len(X)))
        u, v = cca.transform(X, refs(f, fs, len(X)))
        scores.append(np.corrcoef(u[:, 0], v[:, 0])[0, 1])
    return freqs[int(np.argmax(scores))], scores

The stimulus

PsychoPy, full screen, black background, four to eight white squares each toggling on a fixed frame schedule, with a letter or word under each. Push an LSL marker whenever the layout changes. Frame-locked toggling is the whole trick; do it inside the flip loop, never with timers.

The speller

A four-target selector spells slowly. The classic layout is a tree: four squares each holding a group of letters; look at a group to select it, then the letters in that group spread across the four squares, then select the letter. Two selections per character, about eight seconds each with a comfortable window: a character every fifteen to twenty seconds. Slow, and it works on the first try for most people, which no other paradigm can say.

Measure and report two numbers: accuracy (fraction of selections correct) and information transfer rateBrain-computer interface (BCI)A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. Glossary entry in bits per minute, which combines accuracy, number of targets, and speed into one figure that lets you compare against the literature. The classification explainer has the formula.