Digital Illustration of a Neuron cell
Could mind-reading become a reality? Scientists have discovered a method of tracking the brain’s response to sounds and then translating the activity back into the words people heard. In a new study reported in PLoS Biology this week, the researchers say they identified words with 89% accuracy.
Further research is still needed, but the advance could someday help doctors communicate with patients who are unable to speak because of stroke, paralysis or other disorders. The hope is that if researchers can figure out how to identify speech that people hear, they may also learn to decode imagined speech as well.
The study involved 15 epilepsy and brain tumor patients who were already undergoing neurosurgery to treat seizures. The patients had electrodes placed on the cortex of their brains — including atop the temporal lobe where speech is processed — to track the location of seizures. So the researchers, led by Brian Pasley, a postdoctoral researcher at University of California, Berkeley, used the opportunity to record brain activity for their auditory study.
The patients listened to 5 to 10 minutes of audio of different words and sentences, which triggered electrical activity in the brain. Using a computational model, the researchers mapped the sounds the patient was hearing to the patterns of brain activity picked up by the electrodes. The model could then learn how to match certain sounds to specific electrical signals.
The researchers then tested the accuracy of their model by conducting another experiment, but in the opposite direction. They looked at listeners’ brain activity then translated it back into sounds. The model could reconstruct sounds — but not exact words — so the researchers matched each sound to the closest word it resembled.
The study looked at sounds people actually heard, and it’s not clear whether the same brain regions are involved in sounds or words people imagine. One goal of the research is to figure that out. “From a prosthetic view, people who have speech disorders … could possibly have a prosthetic device when they can’t speak but they can imagine what they want to say,” Robert Knight of U.C. Berkeley, a senior author of the study, told the BBC.