Multimodal Classification with Deep Convolutional-Recurrent Neural Networks for Electroencephalography
This work addresses the problem of enhancing EEG classification for brain-computer interfaces, particularly for stroke rehabilitation, though it appears incremental as it adapts existing video classification techniques to EEG data.
The paper tackles low classification accuracy in EEG-based brain-computer interfaces by proposing a novel method that treats EEG data as a video classification problem, incorporating optical flow to preserve multimodal information, and reports improved robustness and accuracy.
Electroencephalography (EEG) has become the most significant input signal for brain computer interface (BCI) based systems. However, it is very difficult to obtain satisfactory classification accuracy due to traditional methods can not fully exploit multimodal information. Herein, we propose a novel approach to modeling cognitive events from EEG data by reducing it to a video classification problem, which is designed to preserve the multimodal information of EEG. In addition, optical flow is introduced to represent the variant information of EEG. We train a deep neural network (DNN) with convolutional neural network (CNN) and recurrent neural network (RNN) for the EEG classification task by using EEG video and optical flow. The experiments demonstrate that our approach has many advantages, such as more robustness and more accuracy in EEG classification tasks. According to our approach, we designed a mixed BCI-based rehabilitation support system to help stroke patients perform some basic operations.