Project C: Prosthesis simulators

You are skimming: the title, the first figure, and the short version. Switch to Read in the header for the full page, or Deep to open every deep dive.

The vocoder in Python

Split the sound into N bands between 200 and 7000 Hz, spaced logarithmically like the cochlea. For each band, extract the envelope (rectify and low-pass at a few hundred hertz, or take the Hilbert magnitude). Generate noise, filter it into the same band, multiply by the envelope. Sum the bands.

import numpy as np
from scipy import signal
def vocode(x, fs, n_ch=8, lo=200, hi=7000, env_fc=160):
    edges = lo * (hi/lo) ** (np.arange(n_ch+1)/n_ch)
    out = np.zeros_like(x); rng = np.random.default_rng(0)
    for f1, f2 in zip(edges[:-1], edges[1:]):
        sos = signal.butter(4, [f1, f2], btype='band', fs=fs, output='sos')
        band = signal.sosfiltfilt(sos, x)
        env = signal.sosfiltfilt(signal.butter(2, env_fc, fs=fs, output='sos'), np.abs(band))
        carrier = signal.sosfiltfilt(sos, rng.standard_normal(len(x)))
        out += carrier * env
    return out / np.max(np.abs(out))

Record yourself reading a paragraph. Vocode it at 1, 2, 4, 8, 16, and 22 channels. Play them to a friend who has not heard the original, in increasing order, and ask them to write down what they hear. Plot words correct against channels. You have just reproduced a landmark experiment. Then try music.

What a real implant adds and lacks

Real implants also compress the envelope (the electrical dynamic range is tiny), pick the loudest bands each frame rather than stimulating all (an “n-of-m” strategy), and pulse at a fixed rate rather than delivering noise. What they lack: the fine timing of the sound within each band, which carries pitch, and the sharp frequency resolution of a healthy cochlea, because current from each electrode spreads to neighbouring nerve fibres. Twenty-two electrodes yield perhaps eight independent channels. The simulator’s “channels” slider is optimistic for that reason.

The phosphene simulator

Each electrode produces a phosphenePhospheneA spot of light perceived when the visual system is stimulated electrically rather than by light. Glossary entry, a blob of light, at roughly a fixed location. Brightness follows the stimulation strength, with only a few distinguishable levels. Some electrodes do nothing. The grid is irregular. Build it in Python with OpenCV: capture a frame, downsample to N×N, quantize to L levels, render each cell as a Gaussian blob with jitter and dropout. Then wear it: a laptop screen held in front of you, or a phone strapped to a headband, and walk a corridor. Sixty electrodes, the count of the first commercial retinal implant, is a humbling amount of vision. A thousand, the ambition of current cortical projects, starts to be useful for navigation.

What both simulations teach

That a prosthesis does not restore a sense; it delivers a new, sparse signal that the brain must learn to use. That the brain is startlingly good at this (four channels of noise becoming speech). That the engineering limit is often not the electronics but current spread in tissue, which caps the number of independent channels. And that the correct question for any prosthesis is not “how much information does it carry” but “what can a person do with it after a year of learning.”