Project C: Silent speech

Speech is muscle. Even when you mouth a word without making a sound, the tongue, lips, jaw, and larynx move, and surface electrodesElectrode The conductor that turns ion currents in tissue into electron currents in a wire; its chemistry and contact quality set the noise you live with. Related: Silver/silver-chloride (Ag/AgCl)Silver/silver-chloride (Ag/AgCl) The standard electrode material for biopotentials, because its chemistry produces a stable, low-noise, low-drift interface with salty gel. , Electrode impedanceElectrode impedance How hard it is to push current through the skin-electrode contact, typically 5 to 50 kilohms with gel and megohms dry, higher at low frequencies. , Half-cell potentialHalf-cell potential The DC voltage that appears wherever a metal meets an electrolyte; two electrodes with different half-cell potentials produce an offset that can swamp the signal. . Glossary entry over them pick up the EMGElectromyography (EMG) Recording the electrical activity of muscles, hundreds of microvolts to millivolts, much larger than EEG. Related: Myoelectric prosthesisMyoelectric prosthesis An artificial limb controlled by EMG from the remaining muscles., Electroencephalography (EEG)Electroencephalography (EEG) Recording the brain's electrical activity from electrodes on the scalp, a few tens of microvolts of blurred, summed cortical activity.. Glossary entry. A classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry can learn to tell “yes” from “no” from “help” from “water” from those patterns. This is a silent speech interfaceSilent speech interface Decoding intended words from muscle activity of the face and throat while the person mouths or subvocalizes without sound. Related: Electromyography (EMG)Electromyography (EMG) Recording the electrical activity of muscles, hundreds of microvolts to millivolts, much larger than EEG.. Glossary entry: what a laryngectomy patient could use, what a person in a loud or quiet place could use, and what several companies are now building. With eight channels and the armband’s pipeline, a ten-word vocabulary is a month’s project.

Beyond the amplifier

Electrode placement

Bipolar pairs at: the masseter (jaw, in front of the ear), the orbicularis oris (around the lips, above and below), the mentalis and digastric (under the chin, for the tongue), the zygomaticus (cheek), and the throat over the larynx. Eight channels, reference on the mastoid, bias on the collarbone. Photograph the placement and mark it; cross-session shift is worse on the face than on the arm because the skin moves.

Data

Ten words that matter to a user: yes, no, help, water, pain, stop, more, hot, cold, thank you. Twenty repetitions each per session, mouthed silently, with a marker at the start of each. Three sessions on different days. Add a “rest” class. Do not skip the rest class; a system that fires on chewing is unusable.

Features and classifier

Windows of 100 ms with 50 ms hop over the one-second utterance, the same features as the armband (RMS, waveform length, zero crossings, plus a few spectral bins), concatenated across the utterance into one vector, or fed to a small recurrent network if you have the data. LDALinear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. Related: ClassifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. . Glossary entry or a Riemannian classifierClassifier An algorithm that assigns each trial a label, such as left hand or right hand, after being trained on labelled examples. Related: Linear discriminant analysis (LDA)Linear discriminant analysis (LDA) A simple, robust classifier that draws a straight boundary between classes; the workhorse of EEG decoding for two decades. , Support vector machine (SVM)Support vector machine (SVM) A classifier that finds the boundary with the widest margin between classes; strong on small datasets. , Cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. . Glossary entry first. Grouped cross-validationCross-validation Testing a model on data it never saw during training, repeatedly, so the reported accuracy means something. Related: Data leakageData leakage Information from the test set sneaking into training, for example by shuffling trials from one session across both; it inflates accuracy and is the most common flaw in student BCI papers. , OverfittingOverfitting A model that has memorized its training data, including its noise, and fails on anything new. . Glossary entry by session, always.

Evaluation

Accuracy on the eleven classes by held-out session. Confusion matrix: which words confuse (hot and cold will). Then the real test: a live session where the person mouths random words from the list and the system speaks its guess aloud, and you count. Then a session where they mouth words not in the list and you count false positives.

Where this goes

More electrodesElectrode The conductor that turns ion currents in tissue into electron currents in a wire; its chemistry and contact quality set the noise you live with. Related: Silver/silver-chloride (Ag/AgCl)Silver/silver-chloride (Ag/AgCl) The standard electrode material for biopotentials, because its chemistry produces a stable, low-noise, low-drift interface with salty gel. , Electrode impedanceElectrode impedance How hard it is to push current through the skin-electrode contact, typically 5 to 50 kilohms with gel and megohms dry, higher at low frequencies. , Half-cell potentialHalf-cell potential The DC voltage that appears wherever a metal meets an electrolyte; two electrodes with different half-cell potentials produce an offset that can swamp the signal. . Glossary entry and finer placement move toward phonemes. Combining with a language model turns phoneme guesses into words. Companies (and a well-known 2023 paper) reach conversational rates with surface EMGElectromyography (EMG) Recording the electrical activity of muscles, hundreds of microvolts to millivolts, much larger than EEG. Related: Myoelectric prosthesisMyoelectric prosthesis An artificial limb controlled by EMG from the remaining muscles. , Electroencephalography (EEG)Electroencephalography (EEG) Recording the brain's electrical activity from electrodes on the scalp, a few tens of microvolts of blurred, summed cortical activity. . Glossary entry on the face using dozens of channels and deep models trained on hours per person. Your ten words are the first rung.