Brain-computer interfaces (BCIs) are being developed to help people who, due to neurodegenerative disease or brain injury, are locked-in and are unable to interact directly with the world through normal motor channels such as speech or body movements. Electroencephalography (EEG) provides a way of non-invasively measuring electrical activity from neurons in the brain as well as other electrical activity from the environment, muscles, and eye movements.  One popular form of EEG-based BCI is the motor imagery (MI) BCI where the user controls a cursor movement on a screen or other output device through imagining movement.  By imagining movement of different body parts, cursor movement can be commanded in different directions.  A common motor imagery paradigm is for the user to imagine right hand movements to move a cursor right and left hand movements to move a cursor left.  As the EEG signal is very noisy, the control is usually broken into multiple steps, with the cursor moving one step every second, to increase the overall accuracy of hitting a target on the right or left of the screen.

One problem with EEG is that it is a global measure of neuronal activity and thus reflects activity throughout the brain that may or may not be related to the task the user has been set.  For instance, frustration or excitement with the task reflect brain activity that may also be present in the EEG. The goal of this grant was to create a more robust BCI system by turning a signal that was previously considered "noise" into a useful signal to help infer the user's intent.  The signal of interest is the EEG response to individual cursor movements during the MI paradigm.  These cursor movements serve two purposes;  they track overall progress towards the targets, and each movement serves as a feedback signal to the user informing them of the computer's interpretation of their last motor imagery attempt.  We found that we could train the computer to determine from the user's EEG whether the cursor moved in the desired/"good" direction or the non-desired/"bad" direction with above chance probability.  In half the users, this signal was better classified than whether they were performing right or left motor imagery.  The information about the "goodness" of the cursor movements was contained in some of the same features of the EEG as the information about the motor imagery.  Previous MI BCIs ignored the issue of whether "good" or "bad" feedback was provided and considered any brain signal other than the motor imagery signal to be noise or simply looked for specific "error" signals, but by specifically learning to classify the "goodness" of cursor movement, and combining this information with the classified direction of motor imagery, we developed a method to obtain better accuracy in predicting the direction the user wanted the cursor to go.  Improvements varied across the users from no improvement (but no degradation) to 18% improvement in classification accuracy of individual seconds of motor imagery.

The project also studied and demonstrated the theoretical performance benefits of using an interactive (one that responds to the computer's feedback) control signal such as "good movement" and "bad movement" compared to a standard control signal that does not depend on the computer's feedback such as "move left" and "move right".  We also developed a modification to the most common spatial filtering algorithm for motor-imagery based brain-computer interfaces that performs more robust filtering in the presence of noise.

In the course of the work the performance of a real-time driver for a major EEG processing software (BCILAB) and a popular type of EEG hardware (Biosemi) was improved by the Swartz Center for Computational Neuroscience.  We also created a github repository for a firmware fix for OpenBCI, a low-cost open source brain-computer interface system, https://github.com/alxrsngrtn/OpenBCI_32 and LSL (Swartz Center for Computational Neuroscience's Lab Streaming Layer) driver for OpenBCI, https://github.com/alxrsngrtn/OpenBCI-LSL The firmware fix provides timestamps when EEG samples are recorded.  Without the fix, the EEG is not time stamped until it is read by the computer which can introduce a variable and unknown time delay between actual recording and the "recorded time". 

Additionally, a class was developed for undergraduate and graduate students to learn about EEG-based BCIs and to participate in projects and develop BCI games and applications.  These BCI games, along with EEG software for visualizing brain activity, were presented to elementary, middle, and high school students in the San Diego area, to foster interest in Neuroscience, Mathematics, and Computer Science.  Outreach presentations were also developed and given by students in the lab to other University students pertaining to machine learning and signal processing for EEG data, as well as how to use OpenBCI and Neurosky consumer-grade EEG hardware and software.

Last Modified: 12/01/2016
Modified by: Virginia De Sa