HCJul 24, 2023
End-to-End Deep Transfer Learning for Calibration-free Motor Imagery Brain Computer InterfacesMaryam Alimardani, Steven Kocken, Nikki Leeuwis
A major issue in Motor Imagery Brain-Computer Interfaces (MI-BCIs) is their poor classification accuracy and the large amount of data that is required for subject-specific calibration. This makes BCIs less accessible to general users in out-of-the-lab applications. This study employed deep transfer learning for development of calibration-free subject-independent MI-BCI classifiers. Unlike earlier works that applied signal preprocessing and feature engineering steps in transfer learning, this study adopted an end-to-end deep learning approach on raw EEG signals. Three deep learning models (MIN2Net, EEGNet and DeepConvNet) were trained and compared using an openly available dataset. The dataset contained EEG signals from 55 subjects who conducted a left- vs. right-hand motor imagery task. To evaluate the performance of each model, a leave-one-subject-out cross validation was used. The results of the models differed significantly. MIN2Net was not able to differentiate right- vs. left-hand motor imagery of new users, with a median accuracy of 51.7%. The other two models performed better, with median accuracies of 62.5% for EEGNet and 59.2% for DeepConvNet. These accuracies do not reach the required threshold of 70% needed for significant control, however, they are similar to the accuracies of these models when tested on other datasets without transfer learning.
50.0LGMay 17
Beyond Accuracy: Robustness, Interpretability and Expressiveness of EEG Foundation ModelsUrban Širca, Maryam Alimardani, Stefanos Zafeiriou et al.
EEG foundation models (EEG-FMs) have been evaluated predominantly on clean, in-distribution accuracy, leaving their robustness, interpretability and representational quality largely unexamined. This study addresses these gaps by benchmarking six EEG-FMs against a baseline deep learning model across eight datasets. Beyond clean accuracy, we conduct three layers of analysis: (i) Robustness: we apply test-time perturbations including additive noise, random and region-based channel dropout and region-specific noise injection. Our analyses show that no single model dominates all failure modes. The most noise-robust model is among the most fragile under channel dropout and much of the dropout fragility disappears when channels are removed rather than zero-padded. (ii) Interpretability: we present the first application of Attention-Aware Layer-Wise Relevance Propagation (AttnLRP) to EEG-FMs and show that models broadly concentrate relevance on task-appropriate brain regions consistent with known neurophysiology. However, attribution maps remain spatially stable under perturbation while predictions degrade, suggesting that the models attend to the correct brain regions but decode corrupted content. (iii) Expressiveness: With block-wise probing we show that late blocks are repurposed during fine-tuning, while early blocks already hold task-related information. Furthermore, we demonstrate that the poor head-only performance previously attributed to low-quality pre-trained representations is largely explained by pooling and that EEG-FMs possess sufficient representational capacity when their token-level embeddings are preserved. Together, these findings provide the first systematic assessment of robustness, interpretability and expressiveness for EEG-FMs and highlight critical considerations for their development.
18.6HCMay 7
Prototyping and Evaluating a Real-time Neuro-Adaptive Virtual Reality Flight Training SystemEvy van Weelden, Jos M. Prinsen, Caterina Ceccato et al.
Real-time adjustments to task difficulty during flight training are crucial for optimizing performance and managing pilot workload. This study evaluated the functionality of a pre-trained brain-computer interface (BCI) that adapts training difficulty based on real-time estimations of workload from brain signals. Specifically, an EEG-based neuro-adaptive training system was developed and tested in Virtual Reality (VR) flight simulations with military student pilots. The neuro-adaptive system was compared to a fixed sequence that progressively increased in difficulty, in terms of self-reported user engagement, workload, and simulator sickness (subjective measures), as well as flight performance (objective metric). Additionally, we explored the relationships between subjective workload and flight performance in the VR simulator for each condition. The experiments concluded with semi-structured interviews to elicit the pilots' experience with the neuro-adaptive prototype. Results revealed no significant differences between the adaptive and fixed sequence conditions in subjective measures or flight performance. In both conditions, flight performance decreased as subjective workload increased. The semi-structured interviews indicated that, upon briefing, the pilots preferred the neuro-adaptive VR training system over the system with a fixed sequence, although individual differences were observed in the perception of difficulty and the order of changes in difficulty. Even though this study shows performance does not change, BCI-based flight training systems hold the potential to provide a more personalized and varied training experience.
HCMay 8, 2025
Would You Rely on an Eerie Agent? A Systematic Review of the Impact of the Uncanny Valley Effect on Trust in Human-Agent InteractionAhdiyeh Alipour, Tilo Hartmann, Maryam Alimardani
Trust is a fundamental component of human-agent interaction. With the increasing presence of artificial agents in daily life, it is essential to understand how people perceive and trust these agents. One of the key challenges affecting this perception is the Uncanny Valley Effect (UVE), where increasingly human-like artificial beings can be perceived as eerie or repelling. Despite growing interest in trust and the UVE, existing research varies widely in terms of how these concepts are defined and operationalized. This inconsistency raises important questions about how and under what conditions the UVE influences trust in agents. A systematic understanding of their relationship is currently lacking. This review aims to examine the impact of the UVE on human trust in agents and to identify methodological patterns, limitations, and gaps in the existing empirical literature. Following PRISMA guidelines, a systematic search identified 53 empirical studies that investigated both UVE-related constructs and trust or trust-related outcomes. Studies were analyzed based on a structured set of categories, including types of agents and interactions, methodological and measurement approaches, and key findings. The results of our systematic review reveal that most studies rely on static images or hypothetical scenarios with limited real-time interaction, and the majority use subjective trust measures. This review offers a novel framework for classifying trust measurement approaches with regard to the best-practice criteria for empirically investigating the UVE. As the first systematic attempt to map the intersection of UVE and trust, this review contributes to a deeper understanding of their interplay and offers a foundation for future research. Keywords: the uncanny valley effect, trust, human-likeness, affinity response, human-agent interaction
HCAug 23, 2021
EEG-based Classification of Drivers Attention using Convolutional Neural NetworkFred Atilla, Maryam Alimardani
Accurate detection of a drivers attention state can help develop assistive technologies that respond to unexpected hazards in real time and therefore improve road safety. This study compares the performance of several attention classifiers trained on participants brain activity. Participants performed a driving task in an immersive simulator where the car randomly deviated from the cruising lane. They had to correct the deviation and their response time was considered as an indicator of attention level. Participants repeated the task in two sessions; in one session they received kinesthetic feedback and in another session no feedback. Using their EEG signals, we trained three attention classifiers; a support vector machine (SVM) using EEG spectral band powers, and a Convolutional Neural Network (CNN) using either spectral features or the raw EEG data. Our results indicated that the CNN model trained on raw EEG data obtained under kinesthetic feedback achieved the highest accuracy (89%). While using a participants own brain activity to train the model resulted in the best performances, inter-subject transfer learning still performed high (75%), showing promise for calibration-free Brain-Computer Interface (BCI) systems. Our findings show that CNN and raw EEG signals can be employed for effective training of a passive BCI for real-time attention classification.
HCOct 5, 2020
High Aptitude Motor Imagery BCI Users Have Better Visuospatial MemoryNikki Leeuwis, Maryam Alimardani
Brain computer interfaces (BCI) decode the electrophysiological signals from the brain into an action that is carried out by a computer or robotic device. Motor imagery BCIs (MI BCI) rely on the user s imagination of bodily movements, however not all users can generate the brain activity needed to control MI BCI. This difference in MI BCI performance among novice users could be due to their cognitive abilities. In this study, the impact of spatial abilities and visuospatial memory on MI BCI performance is investigated. Fifty four novice users participated in a MI BCI task and two cognitive tests. The impact of spatial abilities and visuospatial memory on BCI task error rate in three feedback sessions was measured. Our results showed that spatial abilities, as assessed by the Mental Rotation Test, were not related to MI BCI performance, however visuospatial memory, assessed by the design organization test, was higher in high aptitude users. Our findings can contribute to optimization of MI BCI training paradigms through participant screening and cognitive skill training.
HCAug 12, 2020
Robot-Assisted Mindfulness Practice: Analysis of Neurophysiological Responses and Affective State ChangeMaryam Alimardani, Linda Kemmeren, Kazuki Okumura et al.
Mindfulness is the state of paying attention to the present moment on purpose and meditation is the technique to obtain this state. This study aims to develop a robot assistant that facilitates mindfulness training by means of a Brain Computer Interface (BCI) system. To achieve this goal, we collected EEG signals from two groups of subjects engaging in a meditative vs. nonmeditative human robot interaction (HRI) and evaluated cerebral hemispheric asymmetry, which is recognized as a well defined indicator of emotional states. Moreover, using self reported affective states, we strived to explain asymmetry changes based on pre and post experiment mood alterations. We found that unlike earlier meditation studies, the frontocentral activations in alpha and theta frequency bands were not influenced by robot guided mindfulness practice, however there was a significantly greater right sided activity in the occipital gamma band of Meditation group, which is attributed to increased sensory awareness and open monitoring. In addition, there was a significant main effect of Time on participants self reported affect, indicating an improved mood after interaction with the robot regardless of the interaction type. Our results suggest that EEG responses during robot-guided meditation hold promise in realtime detection and neurofeedback of mindful state to the user, however the experienced neurophysiological changes may differ based on the meditation practice and recruited tools. This study is the first to report EEG changes during mindfulness practice with a robot. We believe that our findings driven from an ecologically valid setting, can be used in development of future BCI systems that are integrated with social robots for health applications.
HCMay 6, 2020
Prediction of Human Empathy based on EEG Cortical AsymmetryAndrea Kuijt, Maryam Alimardani
Humans constantly interact with digital devices that disregard their feelings. However, the synergy between human and technology can be strengthened if the technology is able to distinguish and react to human emotions. Models that rely on unconscious indications of human emotions, such as (neuro)physiological signals, hold promise in personalization of feedback and adaptation of the interaction. The current study elaborated on adopting a predictive approach in studying human emotional processing based on brain activity. More specifically, we investigated the proposition of predicting self-reported human empathy based on EEG cortical asymmetry in different areas of the brain. Different types of predictive models i.e. multiple linear regression analyses as well as binary and multiclass classifications were evaluated. Results showed that lateralization of brain oscillations at specific frequency bands is an important predictor of self-reported empathy scores. Additionally, prominent classification performance was found during resting-state which suggests that emotional stimulation is not required for accurate prediction of empathy -- as a personality trait -- based on EEG data. Our findings not only contribute to the general understanding of the mechanisms of empathy, but also facilitate a better grasp on the advantages of applying a predictive approach compared to hypothesis-driven studies in neuropsychological research. More importantly, our results could be employed in the development of brain-computer interfaces that assist people with difficulties in expressing or recognizing emotions.
HCMar 24, 2020
Assessment of Empathy in an Affective VR Environment using EEG SignalsMaryam Alimardani, Annabella Hermans, Angelica M. Tinga
With the advancements in social robotics and virtual avatars, it becomes increasingly important that these agents adapt their behavior to the mood, feelings and personality of their users. One such aspect of the user is empathy. Whereas many studies measure empathy through offline measures that are collected after empathic stimulation (e.g. post-hoc questionnaires), the current study aimed to measure empathy online, using brain activity collected during the experience. Participants watched an affective 360 video of a child experiencing domestic violence in a virtual reality headset while their EEG signals were recorded. Results showed a significant attenuation of alpha, theta and delta asymmetry in the frontal and central areas of the brain. Moreover, a significant relationship between participants' empathy scores and their frontal alpha asymmetry at baseline was found. These results demonstrate specific brain activity alterations when participants are exposed to an affective virtual reality environment, with the level of empathy as a personality trait being visible in brain activity during a baseline measurement. These findings suggest the potential of EEG measurements for development of passive brain-computer interfaces that assess the user's affective responses in real-time and consequently adapt the behavior of socially intelligent agents for a personalized interaction.