Project G: Neurofeedback, and how to test whether it does anything
Build an alpha trainer that shows your own rhythm in real time. Then read the skeptical literature, design a sham-controlled test, and find out whether your device does anything. The critical thinking is the product.
You are skimming: the title, the first figure, and the short version. Switch to Read in the header for the full page, or Deep to open every deep dive.
NeurofeedbackNeurofeedbackShowing someone a measure of their own brain activity in real time so they can try to change it; its clinical claims range from plausible to unsupported. Glossary entry shows people a measure of their own brain activity and asks them to change it: raise alpha to relax, raise the beta-to-theta ratio to focus. There is a fifty-year literature, a billion-dollar industry of clinics and headbands, and a body of well-run trials that mostly find the effects are the same whether the feedback is real or sham. Building a trainer is easy. The project is designing a test that could tell you whether yours does anything, running it on yourself, and living with the answer.
A clinic reports that 80 percent of clients felt calmer after ten neurofeedback sessions. What does that tell you about whether the feedback worked?
Weak evidence. Ten sessions of sitting quietly with attentive supervision produce calmer people whatever the screen shows. The only test that separates the feedback from everything around it is a sham: identical sessions where the display is driven by someone else’s recording, or by the participant’s own but with the target reversed, with neither participant nor experimenter knowing which. The best-run sham-controlled trials of neurofeedback for ADHD found no difference between real and sham. That is the finding your test has to reckon with.
Build the trainer
One occipital channel. Compute alpha power (8 to 12 Hz) every quarter second over a two-second window. Show it as a bar or a tone whose pitch follows it. Give a running score for time spent above a threshold set from a two-minute baseline. Twenty minutes a session. That is the whole device, and it is roughly what the headbands do.
Design the test
You are the only participant, so this is an n-of-1 experiment, and it can still be rigorous. Twenty sessions. For each, a coin flip (recorded by a friend, sealed) decides real or sham. Real: the bar follows your alpha. Sham: the bar follows a recording of your alpha from a previous session, played back, so it looks exactly as plausible and you cannot tell. Before and after each session, a two-minute eyes-closed baseline with no feedback, and a short questionnaire (calm, focused, on a scale). After twenty sessions, unseal the coin flips and compare: did resting alpha rise more across real sessions than sham? Did the questionnaire differ? A permutation test on the twenty sessions gives the p-value.
Predict the result in your notebook before you start. Then run it.
Read while you run it
The Cochrane-style reviews and the large sham-controlled trials (the ADHD trials are the best-known). The methodological critiques: unblinded experimenters, no sham, outcome measures chosen after the fact. And the defenders’ replies. Form a view about what would count as evidence and whether the field has produced it.
- Build the trainer. Verify that closing your eyes moves the bar.
- Record five baseline sessions to have sham material and a stable resting-alpha measurement.
- Have a friend generate and seal twenty coin flips.
- Run twenty sessions over four weeks. Same time of day. Baselines before and after each.
- Unseal. Compute the change in resting alpha per session type. Permutation test. Same for the questionnaire.
- Write it up as a two-page report with the pre-registered prediction next to the result.