PhD candidate Nir Even-Chen and his advisor, professor Krishna Shenoy, et al., share recent strides in brain-machine interface (BMI) innovation. BMIs are devices that record neural activity from the user's brain and translate it into movement of prosthetic devices. BMIs enable people with motor impairment, e.g. a spinal cord injury, to control and move prosthetic devices with their minds. They can control robotic arms for improving their independence or a computer cursor for typing and browsing the web. Even-Chen, et al's, recently published paper, "Augmenting intracortical brain-machine interface with neurally driven error detectors," describes a new system that reads users minds, detects when the user perceives a mistake, and intervenes with a corrective action. The new system allows users to control BMIs more easily, smoothly, and efficiently.
While most BMI studies focus on designing better techniques to infer the user's movement intention, Even-Chen, et al, improved the BMI performance by taking a very different approach, detecting and undoing mistakes. Their work presents both novel fundamental science and implementation of their idea. They showed for the first time that it is possible to detect key-selection errors from the motor cortex — a brain area mainly involved in movement control. Then, they used the data in real-time to undo—or even prevent—mistakes.
The need for real-time error correction
In our daily life, we all make mistakes, from typos during texting, clicking the wrong link on a web page, or knocking our cup of coffee over while reaching for the cake. Correcting these mistakes might be time-consuming, and annoying—especially when they occur frequently during challenging tasks. Imagine a system that could detect – or predict – your mistakes (e.g., typos) and automatically undo, or even prevent them from happening. This can save the time of manually correcting the mistake, especially when the errors are frequent and the actions to correct them slow you down. Error detection is not always trivial, in some cases only the person who made the mistake knows what she intended. Thus, such an error detection system needs to infer one's intention, i.e., read her mind. An automatic error detection system is most effective when the task is challenging or when our skill is limited, and errors are common. A good candidate for testing such an error detection approach is a BMI system. First, BMIs enable a readout of the user's mind. And second, it can be highly beneficial for BMI users, since BMI control is challenging and prone to errors.
Intracortical BMIs, which records neural activity directly from the brain, showed a promising result in pilot clinical trials and are the highest-performing BMI systems to date. This makes them prime candidates for serving as an assistive technology for people with paralysis. Although the performance of intracortical BMI systems has markedly improved in the last two decades, errors — such as selecting the wrong key during typing — still occur and their performance is far from able-bodied performance.
Previously it was unknown if errors can be detected from the same brain region traditionally used for decoding BMI user's movement intention—the motor cortex. In their work, Even-Chen and colleagues found that when errors occur a characteristic brain activity can be observed. That brain activity pattern enables them to detect mistakes with high accuracy shortly after and even before they occurred.
This finding encouraged them to develop and implement first-of-its-kind error "detect-and-act" system. This system reads the user's mind, detects when the user thinks an error occurred, and can automatically "undo" or "prevent" them. The detect-and-act system works independently and in parallel to a traditional movement BMI that estimate user's movement intention (see figure). In a challenging BMI task that resulted in substantial errors, this approach improved the performance of a BMI. Using the detect-and-act system, hard tasks will have fewer errors and become easier, the use of a BMI will become smoother, and be less frustrating.
A detect-and-act system can potentially be used to improve how fast people with paralysis can type or control a robotic arm using a BMI. For example, automatically correcting a mistake when they type, or stopping the movement of a robotic arm when they are about to knock over their coffee. While this work has been done in pre-clinical trial with monkeys, Even-Chen and colleagues also presented encouraging preliminary results of a clinical trial (BrainGate2) at a conference, and showed the potential translation to humans.
Read more: Journal of Neural Engineering, "Augmenting intracortical brain-machine interface with neurally driven error detectors."
Additional authors include Sergey Stavisky, Jonathan Kao, Stephen Ryu, and Krishna Shenoy.