Project E: Intracortical data
Human motor cortex recordings from published BCI studies, downloaded free. Spike sorting, local field potentials, and a decoder on the real thing. The end of the road that started with a cockroach leg.
On this pageA month
The DANDI archiveDANDI archive A public repository of neurophysiology data in NWB format, including human intracortical BCI recordings from published papers. Related: Neurodata Without Borders (NWB)Neurodata Without Borders (NWB) The standard file format for neurophysiology data, so recordings from different labs can be read the same way.. Glossary entry and the data supplements of recent Nature papers hold recordings from Utah ArraysUtah Array The 10-by-10 grid of 1.5 mm silicon needles invented at the University of Utah, the electrode in nearly every human intracortical BCI study to date. Related: Intracortical recordingIntracortical recording Recording from tiny electrodes inserted into the cortex itself, close enough to hear individual neurons., Utah Slanted Electrode Array (USEA)Utah Slanted Electrode Array (USEA) A version of the Utah Array with needles of graded lengths, designed to reach different depths of a peripheral nerve.. Glossary entry in the motor cortex of people with paralysis: the actual data behind the cursor and speech-decoding results. In NWBNeurodata Without Borders (NWB) The standard file format for neurophysiology data, so recordings from different labs can be read the same way. Related: DANDI archiveDANDI archive A public repository of neurophysiology data in NWB format, including human intracortical BCI recordings from published papers.. Glossary entry format, with the behaviour aligned. You have built every tool needed to work with them: spikeSpike Shorthand for an action potential as seen by an electrode, a brief blip about a millisecond wide. Related: Action potentialAction potential The brief, all-or-nothing voltage spike a neuron fires, about a millisecond long and about 100 millivolts tall across its membrane. , Spike sortingSpike sorting Deciding which spikes on an electrode came from which neuron, by clustering their waveform shapes. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry detection from Phase 1’s thinking, the Kalman decoderDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. , Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry from Phase 3, the population analysis from Phase 4. Now run them on a human brain.
A Utah Array has 96 recording electrodes. In a participant implanted three years ago, roughly how many electrodes still show clearly sortable single neurons?
Often a dozen or two clean single units, with many more channels carrying threshold crossings from several unsortable neurons and some carrying only field potentials. Recordings degrade over years as the tissue responds and electrodesElectrode The conductor that turns ion currents in tissue into electron currents in a wire; its chemistry and contact quality set the noise you live with. Related: Silver/silver-chloride (Ag/AgCl)Silver/silver-chloride (Ag/AgCl) The standard electrode material for biopotentials, because its chemistry produces a stable, low-noise, low-drift interface with salty gel. , Electrode impedanceElectrode impedance How hard it is to push current through the skin-electrode contact, typically 5 to 50 kilohms with gel and megohms dry, higher at low frequencies. , Half-cell potentialHalf-cell potential The DC voltage that appears wherever a metal meets an electrolyte; two electrodes with different half-cell potentials produce an offset that can swamp the signal. . Glossary entry fail, which is why most human BCIBrain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. Related: DecoderDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. , Closed loopClosed loop A system in which the output feeds back to influence the input, such as a user adjusting their brain activity while watching the cursor it controls. . Glossary entry decodersDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. , Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry now use threshold crossings and spikeSpike Shorthand for an action potential as seen by an electrode, a brief blip about a millisecond wide. Related: Action potentialAction potential The brief, all-or-nothing voltage spike a neuron fires, about a millisecond long and about 100 millivolts tall across its membrane. , Spike sortingSpike sorting Deciding which spikes on an electrode came from which neuron, by clustering their waveform shapes. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry-band power rather than sorted units, and why the foreign body response is the field’s central engineering problem.
The data
Search DANDIDANDI, Distributed Archives for Neurophysiology Data IntegrationNeurophysiology in NWB format, including the human intracortical data behind the speech BCI papers. for human intracortical datasets: the BrainGate consortium has released cursor-control and handwriting data; the Willett et al. 2023A high-performance speech neuroprosthesisSpeech decoded from motor cortex at conversational rates, with data public for your own reimplementation. speech dataset and the Metzger et al. 2023A high-performance neuroprosthesis for speech decoding and avatar controlThe second 2023 speech BCI, decoding speech and facial movement together, with public data. dataset are public through Dryad and the papers’ supplements. The Neural Latents BenchmarkThe Neural Latents BenchmarkStandard monkey datasets with held-out evaluation, the right place to compare a decoder against the field. packages several (mostly monkey) datasets in a standard format with held-out evaluation, which is the right place to compare a decoderDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. , Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry against the field. Load with pynwb or the NLB tools.
Three analyses
Signal quality across channels. For each electrodeElectrode The conductor that turns ion currents in tissue into electron currents in a wire; its chemistry and contact quality set the noise you live with. Related: Silver/silver-chloride (Ag/AgCl)Silver/silver-chloride (Ag/AgCl) The standard electrode material for biopotentials, because its chemistry produces a stable, low-noise, low-drift interface with salty gel. , Electrode impedanceElectrode impedance How hard it is to push current through the skin-electrode contact, typically 5 to 50 kilohms with gel and megohms dry, higher at low frequencies. , Half-cell potentialHalf-cell potential The DC voltage that appears wherever a metal meets an electrolyte; two electrodes with different half-cell potentials produce an offset that can swamp the signal. . Glossary entry: the spikeSpike Shorthand for an action potential as seen by an electrode, a brief blip about a millisecond wide. Related: Action potentialAction potential The brief, all-or-nothing voltage spike a neuron fires, about a millisecond long and about 100 millivolts tall across its membrane. , Spike sortingSpike sorting Deciding which spikes on an electrode came from which neuron, by clustering their waveform shapes. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry-band (300 to 3000 Hz) RMS, the threshold-crossing rate, and, where waveforms are provided, a sorting. Plot the array as a 10-by-10 grid coloured by signal quality. Count the clean single units. Compare to the paper’s methods section. This grid is the honest picture of what a chronic array delivers.
Decode kinematics. On a cursor or handwriting dataset: bin threshold crossings at 20 ms, fit the Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. Related: DecoderDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. , Bayesian estimationBayesian estimation Updating a belief about a hidden quantity by combining what you expected with what you measured, in proportion to their reliabilities. . Glossary entry from Phase 3 on the calibrationCalibration The session at the start of BCI use in which the decoder is trained on this person, today, because brains and electrodes drift. Related: Transfer learningTransfer learning Using data from other people or other days to shorten calibration for a new user. . Glossary entry block, test on the held-out block, report correlation between decoded and true velocity. Then fit the same on sorted units only and compare; then on spikeSpike Shorthand for an action potential as seen by an electrode, a brief blip about a millisecond wide. Related: Action potentialAction potential The brief, all-or-nothing voltage spike a neuron fires, about a millisecond long and about 100 millivolts tall across its membrane. , Spike sortingSpike sorting Deciding which spikes on an electrode came from which neuron, by clustering their waveform shapes. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry-band power; then on LFPLocal field potential (LFP) The slow, summed electrical activity of many neurons near an electrode, below a few hundred hertz, as opposed to individual spikes. Related: Electroencephalography (EEG)Electroencephalography (EEG) Recording the brain's electrical activity from electrodes on the scalp, a few tens of microvolts of blurred, summed cortical activity. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry features. The comparison says which signal carries the information, and it usually surprises people how well the unsorted signals do.
Population dynamics. PCA on the binned population activity during reaches. Plot the trajectories per reach direction. See the rotational structure that the neural dynamics literature describes, in a human motor cortex. Then try a latent-variable model (the NLB baselines include several) and report held-out likelihood.
Speech
If you take on the speech dataset: neural features per 20 ms, attempted-speech phoneme labels, a recurrent network mapping one to the other, a language model on top. Weeks, not days, and the result is a reimplementation of a paper that made headlines. Report phoneme error rate and word error rate alongside the paper’s.
What you will understand at the end
What a human intracortical recordingIntracortical recording Recording from tiny electrodes inserted into the cortex itself, close enough to hear individual neurons. Related: Utah ArrayUtah Array The 10-by-10 grid of 1.5 mm silicon needles invented at the University of Utah, the electrode in nearly every human intracortical BCI study to date. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry actually looks like after years, and why decodersDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. , Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry were redesigned to cope. Why threshold crossings won. How much of the population’s information lives in a few dimensions. And that the arc from a cockroach leg on a speaker to a paralyzed person’s motor cortex on your laptop is one arc, made of the same physics, the same amplifiers, and the same statistics.
- Choose a dataset and load it. Plot raw traces from a good channel and a bad one.
- Build the array quality grid. Count units.
- Kalman decoderDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. , Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry on threshold crossings, held-out block; then on sorted units, spikeSpike Shorthand for an action potential as seen by an electrode, a brief blip about a millisecond wide. Related: Action potentialAction potential The brief, all-or-nothing voltage spike a neuron fires, about a millisecond long and about 100 millivolts tall across its membrane. , Spike sortingSpike sorting Deciding which spikes on an electrode came from which neuron, by clustering their waveform shapes. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry-band power, and LFPLocal field potential (LFP) The slow, summed electrical activity of many neurons near an electrode, below a few hundred hertz, as opposed to individual spikes. Related: Electroencephalography (EEG)Electroencephalography (EEG) Recording the brain's electrical activity from electrodes on the scalp, a few tens of microvolts of blurred, summed cortical activity. , Extracellular recordingExtracellular recording Recording with an electrode near neurons rather than inside one; you see small, fast blips from nearby cells rather than the full membrane swing. . Glossary entry. Table.
- PCA trajectories by reach direction.
- Optionally, the speech decoderDecoder The algorithm that maps neural signals to intended actions, such as a classifier or a Kalman filter. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. , Kalman filterKalman filter An estimator that blends a prediction from a motion model with a noisy measurement, weighting each by how much it trusts it; the decoder behind most intracortical cursor control. , Brain-computer interface (BCI)Brain-computer interface (BCI) A system that turns measured brain activity into control of something outside the body, from a cursor to a wheelchair to a speech synthesizer. . Glossary entry.
- Write it up as a reproduction, in the Phase 3 format.