Flaws emerge in a new analysis of the original study data.
In November 2011, researchers from the University of Western Ontario in Canada reported in the journal Lancet that they successfully used an electroencephalogram (EEG), which measures electrical activity in the brain, to record brain signals that suggested awareness in patients in a vegetative state. The study involved 16 patients in a persistent vegetative state and 12 healthy control participants. After wiring each of the participants’ scalps, the researchers asked all of them to imagine—at the sound of a tone—making a fist with their right hand or wiggling their toes while the scientists recorded their brain activity using the EEG. Nine of the 12 healthy control participants showed distinct brain activity in the part of their brain associated with movement, and three of the patients in the vegetative state showed the same patterns.
Their findings created excitement among doctors and patients, since they hinted that patients in vegetative states may have more awareness and ability to interact with their environment than previously thought. In addition, the relatively non-invasive EEG was heralded as a potential bed-side tool for detecting which patients showed promising signs of cognitive activity, raising the possibility of not only redefining the vegetative state but creating new criteria for diagnosing and treating the condition.
Now, however, led by scientists at Weill Cornell Medical College, a separate group of researchers says the original data may not have been interpreted correctly. Because the original researchers and the new team shared a grant, the original data was made available for re-analysis.
Initially, the new group of scientists set out to validate the original authors’ findings, given their potential to improve diagnosis and treatment of patients, and not disprove them. But upon review, Weill Cornell-led team reports in the journal Lancet that the original study failed to account for EEG signal noise that likely caused false-positive results. They say the original study’s researchers did not adjust for factors such as muscle activity and random EEG blips that appeared in the readings of the vegetative patients, making it appear as if they registered brain activity in response to the commands.
The omission may have been due to the power of the algorithm that the original researchers used to detect and record changes in brain activity. “It’s a very powerful tool. It will find a pattern for you no problem, but is it a meaningful pattern? It is going to err on the side of over-fitting; finding something there when there really isn’t something there,” says Dr. Nicholas Schiff, professor of public health at Weill Cornell and one of the authors of the new study. “What you’re really finding is actually due to chance, but you haven’t controlled for chance because it’s not so easy to do when you’re working with very complex data that have lots and lots of dimensions.”
When Schiff and his colleagues used their own approach in analyzing the data, taking into account EEG interference, they were unable to find meaningful fluctuations of brain activity.
In a written statement published with the new study, the original study authors recognize that if the new study’s analysis is applied, two of the three vegetative state patients no longer showed awareness, but they note that two of the three patients showed signs of responding to commands while having their brains scanned by functional MRI, which records brain activity during tasks. “These data confirm that these patients were aware during the same week in which the EEG data in question was acquired,” they write.
More research will clearly be needed to fully explore whether EEG can indeed be a useful tool for detecting signs of cognition in unconscious patients. And the re-analysis does not necessarily mean there is a problem with using EEGs at the bedside. “It is a problem with the specific method published in this study. [The study] got a lot of attention as a great way of doing this, and the problem is it doesn’t work as specified,” says Schiff. Using EEGs to detect awareness may still be possible, if researchers can refine their understanding of how to eliminate signal noise and assure themselves that they are recording meaningful brain activity.
In fact, the original study authors have already published an updated method for using EEG in vegetative state patients, and note that “it is only through the continuing improvement of our complementary approaches that we will converge on the optimum methods for accurately identifying covert awareness, where it exists, in every severely brain-injured patient.”