Project B: The P300 speller
A grid of letters flashes row by row and column by column. The one you are counting produces a bump 300 milliseconds later. Average, classify, and the computer types it. The 1988 design that started the field, built in a weekend.
On this pageA weekend to a month
Look at a six-by-six grid of letters and silently count the flashes of the one you want. Rows and columns flash in random order. Whenever your letter’s row or column flashes, your brain produces a P300P300 A positive bump about 300 milliseconds after a rare or meaningful stimulus, the basis of the classic BCI speller. Related: Event-related potential (ERP)Event-related potential (ERP) The small, stereotyped voltage waveform the brain produces in response to an event, invisible in a single trial and revealed by averaging many., Oddball paradigmOddball paradigm An experiment in which a rare target is mixed in among frequent standards; the rare one produces a P300.. Glossary entry: a positive bump over the top and back of the head about three hundred milliseconds later, because that flash was the rare, meaningful event you were waiting for. Average the responses to each row and each column, find the row and the column with the bump, and their intersection is your letter. Farwell and Donchin built this in 1988Talking off the top of your head, toward a mental prosthesis utilizing event-related brain potentialsThe original P300 speller, six by six and still the design every speller project starts from.. It still works, and building it teaches averaging, classification, and honest evaluation in one project.
A single flash of the target row produces a 5 µV P300 in 20 µV of noise. How many flashes must you average before the P300 is twice the noise?
About 64. Noise shrinks as the square root of the number of trials: after N flashes the noise is 20/√N. For the 5 µV signal to be twice the noise, 20/√N = 2.5, so √N = 8 and N = 64. Real spellers get away with fewer, around 10 to 15 per row, because a classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry can use the whole waveform rather than one peak and because it only needs to rank the six rows, not measure any one cleanly. But the arithmetic is why P300P300 A positive bump about 300 milliseconds after a rare or meaningful stimulus, the basis of the classic BCI speller. Related: Event-related potential (ERP)Event-related potential (ERP) The small, stereotyped voltage waveform the brain produces in response to an event, invisible in a single trial and revealed by averaging many. , Oddball paradigmOddball paradigm An experiment in which a rare target is mixed in among frequent standards; the rare one produces a P300. . Glossary entry spellers are slower than SSVEPSteady-state visual evoked potential (SSVEP) When you stare at something flickering at a fixed rate, visual cortex oscillates at that rate, and the frequency can be read straight out of the EEG spectrum. Related: Canonical correlation analysis (CCA)Canonical correlation analysis (CCA) Finding the combination of channels that correlates best with reference sinusoids at each candidate frequency; the standard SSVEP decoder. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry ones.
The stimulus
A 6×6 grid (26 letters plus digits and a few commands). Each “sequence” flashes all six rows and six columns once each, in random order: 12 flashes, each 100 ms on, 75 ms off, so a sequence takes about two seconds. Repeat 10 to 15 sequences per character. Push an LSLLab Streaming Layer (LSL) A protocol that timestamps and synchronizes streams from many devices and stimulus programs on a network; every BCI lab uses it. Related: PsychoPyPsychoPy Free Python software for presenting stimuli with precise timing and logging responses. , BrainFlowBrainFlow A library that talks to OpenBCI and most consumer EEG headsets with one API, in Python and other languages. . Glossary entry marker at each flash saying which row or column it was. The user counts flashes of their letter; the counting is what makes the P300P300 A positive bump about 300 milliseconds after a rare or meaningful stimulus, the basis of the classic BCI speller. Related: Event-related potential (ERP)Event-related potential (ERP) The small, stereotyped voltage waveform the brain produces in response to an event, invisible in a single trial and revealed by averaging many. , Oddball paradigmOddball paradigm An experiment in which a rare target is mixed in among frequent standards; the rare one produces a P300. . Glossary entry large.
The classifier
EpochEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry from 0 to 800 ms after each flash, at Pz, Cz, PO7, PO8, Oz and a few others. Downsample to about 20 Hz, so each epochEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry is 16 samples per channel, and concatenate channels into one feature vector. Train a linear discriminant analysisLinear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples.. Glossary entry classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry (or the stepwise version, SWLDA, which the original literature uses) on labelled epochsEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry: target flashes versus non-target flashes, from a calibrationCalibration The session at the start of BCI use in which the decoder is trained on this person, today, because brains and electrodes drift. Related: Transfer learningTransfer learning Using data from other people or other days to shorten calibration for a new user. . Glossary entry run in which you spell a known word.
To select a character, apply the classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry to each flash’s epochEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry to get a score, sum the scores over all sequences for each row and each column, pick the row and column with the largest sums. Summing scores is the averaging, done after classification instead of before.
xDAWN spatial filtering before LDALinear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. . Glossary entry, or a Riemannian classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry on the epochEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry covariances, improves things noticeably and is what current systems use. Start with LDALinear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. . Glossary entry so you can see what the improvement is.
Evaluation
Accuracy per character as a function of the number of sequences; the curve rises and saturates. Information transfer rate at each point, so you can pick the number of sequences that maximizes bits per minute rather than accuracy. And the honest split: train on the calibrationCalibration The session at the start of BCI use in which the decoder is trained on this person, today, because brains and electrodes drift. Related: Transfer learningTransfer learning Using data from other people or other days to shorten calibration for a new user. . Glossary entry word, test on a different word spelled later in the session, then on a different day.
- Write the PsychoPyPsychoPy Free Python software for presenting stimuli with precise timing and logging responses. Related: Lab Streaming Layer (LSL)Lab Streaming Layer (LSL) A protocol that timestamps and synchronizes streams from many devices and stimulus programs on a network; every BCI lab uses it. . Glossary entry grid with row/column flashing and LSLLab Streaming Layer (LSL) A protocol that timestamps and synchronizes streams from many devices and stimulus programs on a network; every BCI lab uses it. Related: PsychoPyPsychoPy Free Python software for presenting stimuli with precise timing and logging responses. , BrainFlowBrainFlow A library that talks to OpenBCI and most consumer EEG headsets with one API, in Python and other languages. . Glossary entry markers.
- Record a calibrationCalibration The session at the start of BCI use in which the decoder is trained on this person, today, because brains and electrodes drift. Related: Transfer learningTransfer learning Using data from other people or other days to shorten calibration for a new user. . Glossary entry run: spell a ten-character word with 15 sequences per character. EpochEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry and average target versus non-target epochsEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry at Pz. Confirm the P300P300 A positive bump about 300 milliseconds after a rare or meaningful stimulus, the basis of the classic BCI speller. Related: Event-related potential (ERP)Event-related potential (ERP) The small, stereotyped voltage waveform the brain produces in response to an event, invisible in a single trial and revealed by averaging many. , Oddball paradigmOddball paradigm An experiment in which a rare target is mixed in among frequent standards; the rare one produces a P300. . Glossary entry is visible in the average; if not, check timing with the photodiode and check that you are actually counting.
- Train LDALinear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. . Glossary entry on the calibrationCalibration The session at the start of BCI use in which the decoder is trained on this person, today, because brains and electrodes drift. Related: Transfer learningTransfer learning Using data from other people or other days to shorten calibration for a new user. . Glossary entry epochsEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry. Cross-validate by character (train on 8 characters, test on 2), not by epochEpoch A fixed-length slice of the recording locked to an event, such as one second starting 200 milliseconds before each stimulus. Related: Trial averagingTrial averaging Adding many time-locked epochs together so the consistent response grows and the random noise shrinks by the square root of the number of trials. , Baseline correctionBaseline correction Subtracting the mean of the pre-stimulus interval from each epoch so trials line up. . Glossary entry.
- Spell a test word. Plot accuracy and ITR against number of sequences.
- Add xDAWN or a Riemannian classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry and repeat. Note the improvement.
- Test on another person, on a different day. Report all numbers.