Jeong-Hyun Cho

Byeong-Hoo Lee, Jeong-Hyun Cho, Byoung-Hee Kwon et al.

Recently, advanced technologies have unlimited potential in solving various problems with a large amount of data. However, these technologies have yet to show competitive performance in brain-computer interfaces (BCIs) which deal with brain signals. Basically, brain signals are difficult to collect in large quantities, in particular, the amount of information would be sparse in spontaneous BCIs. In addition, we conjecture that high spatial and temporal similarities between tasks increase the prediction difficulty. We define this problem as sparse condition. To solve this, a factorization approach is introduced to allow the model to obtain distinct representations from latent space. To this end, we propose two feature extractors: A class-common module is trained through adversarial learning acting as a generator; Class-specific module utilizes loss function generated from classification so that features are extracted with traditional methods. To minimize the latent space shared by the class-common and class-specific features, the model is trained under orthogonal constraint. As a result, EEG signals are factorized into two separate latent spaces. Evaluations were conducted on a single-arm motor imagery dataset. From the results, we demonstrated that factorizing the EEG signal allows the model to extract rich and decisive features under sparse condition.

Kang Yin, Byeong-Hoo Lee, Byoung-Hee Kwon et al.

Data augmentation approaches are widely explored for the enhancement of decoding electroencephalogram signals. In subject-independent brain-computer interface system, domain adaption and generalization are utilized to shift source subjects' data distribution to match the target subject as an augmentation. However, previous works either introduce noises (e.g., by noise addition or generation with random noises) or modify target data, thus, cannot well depict the target data distribution and hinder further analysis. In this paper, we propose a target-centered subject transfer framework as a data augmentation approach. A subset of source data is first constructed to maximize the source-target relevance. Then, the generative model is applied to transfer the data to target domain. The proposed framework enriches the explainability of target domain by adding extra real data, instead of noises. It shows superior performance compared with other data augmentation methods. Extensive experiments are conducted to verify the effectiveness and robustness of our approach as a prosperous tool for further research.

Byeong-Hoo Lee, Jeong-Hyun Cho, Byung-Hee Kwon

Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierarchically and transfer knowledge to student model. To this end, soft labels and distillation loss functions are applied to the student model training. According to experimental results, student model achieved the best performance among the singular architecture-based methods. It is confirmed that using hierarchical models and knowledge distillation, the performance of a simple architecture can be improved. Since it is uncertain what knowledge is transferred, it is important to clarify this part in future studies.

Jeong-Hyun Cho, Byoung-Hee Kwon, Byeong-Hoo Lee et al.

Brain-computer interface (BCI) is a practical pathway to interpret users' intentions by decoding motor execution (ME) or motor imagery (MI) from electroencephalogram (EEG) signals. However, developing a BCI system driven by ME or MI is challenging, particularly in the case of containing continual and compound muscles movements. This study analyzes three grasping actions from EEG under both ME and MI paradigms. We also investigate the classification performance in offline and pseudo-online experiments. We propose a novel approach that uses muscle activity pattern (MAP) images for the convolutional neural network (CNN) to improve classification accuracy. We record the EEG and electromyogram (EMG) signals simultaneously and create the MAP images by decoding both signals to estimate specific hand grasping. As a result, we obtained an average classification accuracy of 63.6($\pm$6.7)% in ME and 45.8($\pm$4.4)% in MI across all fifteen subjects for four classes. Also, we performed pseudo-online experiments and obtained classification accuracies of 60.5($\pm$8.4)% in ME and 42.7($\pm$6.8)% in MI. The proposed method MAP-CNN, shows stable classification performance, even in the pseudo-online experiment. We expect that MAP-CNN could be used in various BCI applications in the future.

Myoung-Ki Kim, Jeong-Hyun Cho, Hye-Bin Shin

Touch is the first sense among human senses. Not only that, but it is also one of the most important senses that are indispensable. However, compared to sight and hearing, it is often neglected. In particular, since humans use the tactile sense of the skin to recognize and manipulate objects, without tactile sensation, it is very difficult to recognize or skillfully manipulate objects. In addition, the importance and interest of haptic technology related to touch are increasing with the development of technologies such as VR and AR in recent years. So far, the focus is only on haptic technology based on mechanical devices. Especially, there are not many studies on tactile sensation in the field of brain-computer interface based on EEG. There have been some studies that measured the surface roughness of artificial structures in relation to EEG-based tactile sensation. However, most studies have used passive contact methods in which the object moves, while the human subject remains still. Additionally, there have been no EEG-based tactile studies of active skin touch. In reality, we directly move our hands to feel the sense of touch. Therefore, as a preliminary study for our future research, we collected EEG signals for tactile sensation upon skin touch based on active touch and compared and analyzed differences in brain changes during touch and movement tasks. Through time-frequency analysis and statistical analysis, significant differences in power changes in alpha, beta, gamma, and high-gamma regions were observed. In addition, major spatial differences were observed in the sensory-motor region of the brain.

Byeong-Hoo Lee, Jeong-Hyun Cho, Byung-Hee Kwon

Brain-computer interface uses brain signals to communicate with external devices without actual control. Many studies have been conducted to classify motor imagery based on machine learning. However, classifying imagery data with sparse spatial characteristics, such as single-arm motor imagery, remains a challenge. In this paper, we proposed a method to factorize EEG signals into two groups to classify motor imagery even if spatial features are sparse. Based on adversarial learning, we focused on extracting common features of EEG signals which are robust to noise and extracting only signal features. In addition, class-specific features were extracted which are specialized for class classification. Finally, the proposed method classifies the classes by representing the features of the two groups as one embedding space. Through experiments, we confirmed the feasibility that extracting features into two groups is advantageous for datasets that contain sparse spatial features.

Byoung-Hee Kwon, Jeong-Hyun Cho, Byeong-Hoo Lee

An electroencephalogram is an effective approach that provides a bidirectional pathway between user and computer in a non-invasive way. In this study, we adopted the visual perception data for training the visual imagery decoding network. We proposed a visual perception-guided network training approach for decoding visual imagery. Visual perception decreases the power of the alpha frequency range of the visual cortex over time when the user performed the task, and visual imagery increases the power of the alpha frequency range of the visual cortex over time as the user performed with the task. Generated brain signals when the user performing visual imagery and visual perception have opposite brain activity tendencies, and we used these characteristics to design the proposed network. When using the proposed method, the average classification performance of visual imagery with the visual perception data was 0.7008. Our results provide the possibility of using the visual perception data as a guide of the visual imagery classification network training.

Jeong-Hyun Cho, Ji-Hoon Jeong, Myoung-Ki Kim et al.

Noninvasive brain-computer interface (BCI) is widely used to recognize users' intentions. Especially, BCI related to tactile and sensation decoding could provide various effects on many industrial fields such as manufacturing advanced touch displays, controlling robotic devices, and more immersive virtual reality or augmented reality. In this paper, we introduce haptic and sensory perception-based BCI systems called neurohaptics. It is a preliminary study for a variety of scenarios using actual touch and touch imagery paradigms. We designed a novel experimental environment and a device that could acquire brain signals under touching designated materials to generate natural touch and texture sensations. Through the experiment, we collected the electroencephalogram (EEG) signals with respect to four different texture objects. Seven subjects were recruited for the experiment and evaluated classification performances using machine learning and deep learning approaches. Hence, we could confirm the feasibility of decoding actual touch and touch imagery on EEG signals to develop practical neurohaptics.

Myoung-Ki Kim, Jeong-Hyun Cho, Ji-Hoon Jeong

Brain-computer interface allows people who have lost their motor skills to control robot limbs based on electroencephalography. Most BCIs are guided only by visual feedback and do not have somatosensory feedback, which is an important component of normal motor behavior. The sense of touch is a very crucial sensory modality, especially in object recognition and manipulation. When manipulating an object, the brain uses empirical information about the tactile properties of the object. In addition, the primary somatosensory cortex is not only involved in processing the sense of touch in our body but also responds to visible contact with other people or inanimate objects. Based on these findings, we conducted a preliminary experiment to confirm the possibility of a novel paradigm called touch imagery. A haptic imagery experiment was conducted on four objects, and through neurophysiological analysis, a comparison analysis was performed with the brain waves of the actual tactile sense. Also, high classification performance was confirmed through the basic machine learning algorithm.

Byoung-Hee Kwon, Ji-Hoon Jeong, Jeong-Hyun Cho et al.

In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we commonly used in daily life, such as picking up a phone, opening a door, eating food, and pouring water. Nine subjects participated in our experiment. We presented statistical evidence that visual motion imagery has a high correlation from the prefrontal and occipital lobes. In addition, we selected the most appropriate electroencephalography channels using a functional connectivity approach for visual motion imagery decoding and proposed a convolutional neural network architecture for classification. As a result, the averaged classification performance of the proposed architecture for 4 classes from 16 channels was 67.50 % across all subjects. This result is encouraging, and it shows the possibility of developing a BCI-based device control system for practical applications such as neuroprosthesis and a robotic arm.

Jeong-Hyun Cho, Ji-Hoon Jeong, Seong-Whan Lee

Electroencephalogram (EEG) based brain-computer interface (BCI) systems are useful tools for clinical purposes like neural prostheses. In this study, we collected EEG signals related to grasp motions. Five healthy subjects participated in this experiment. They executed and imagined five sustained-grasp actions. We proposed a novel data augmentation method that increases the amount of training data using labels obtained from electromyogram (EMG) signals analysis. For implementation, we recorded EEG and EMG simultaneously. The data augmentation over the original EEG data concluded higher classification accuracy than other competitors. As a result, we obtained the average classification accuracy of 52.49% for motor execution (ME) and 40.36% for motor imagery (MI). These are 9.30% and 6.19% higher, respectively than the result of the comparable methods. Moreover, the proposed method could minimize the need for the calibration session, which reduces the practicality of most BCIs. This result is encouraging, and the proposed method could potentially be used in future applications such as a BCI-driven robot control for handling various daily use objects.

Jeong-Hyun Cho, Ji-Hoon Jeong, Dong-Joo Kim et al.

Developing electroencephalogram (EEG) based brain-computer interface (BCI) systems is challenging. In this study, we analyzed natural grasp actions from EEG. Ten healthy subjects participated in this experiment. They executed and imagined three sustained grasp actions. We proposed a novel approach which estimates muscle activity patterns from EEG signals to improve the overall classification accuracy. For implementation, we have recorded EEG and electromyogram (EMG) simultaneously. Using the similarity of the estimated pattern from EEG signals compare to the activity pattern from EMG signals showed higher classification accuracy than competitive methods. As a result, we obtained the average classification accuracy of 63.89($\pm$7.54)% for actual movement and 46.96($\pm$15.30)% for motor imagery. These are 21.59% and 5.66% higher than the result of the competitive model, respectively. This result is encouraging, and the proposed method could potentially be used in future applications, such as a BCI-driven robot control for handling various daily use objects.