Multimodal Brain-Computer Interfaces: AI-powered Decoding Methodologies
It provides a comprehensive overview for researchers in neuroscience and AI, but is incremental as it reviews existing methodologies without introducing new breakthroughs.
This review paper tackles the problem of decoding brain data in multimodal brain-computer interfaces (BCIs) by analyzing algorithmic advancements, such as cross-modality mapping and sequential modeling, and explores applications in visual, speech, and affective decoding, though it does not present new experimental results or concrete numbers.
Brain-computer interfaces (BCIs) enable direct communication between the brain and external devices. This review highlights the core decoding algorithms that enable multimodal BCIs, including a dissection of the elements, a unified view of diversified approaches, and a comprehensive analysis of the present state of the field. We emphasize algorithmic advancements in cross-modality mapping, sequential modeling, besides classic multi-modality fusion, illustrating how these novel AI approaches enhance decoding of brain data. The current literature of BCI applications on visual, speech, and affective decoding are comprehensively explored. Looking forward, we draw attention on the impact of emerging architectures like multimodal Transformers, and discuss challenges such as brain data heterogeneity and common errors. This review also serves as a bridge in this interdisciplinary field for experts with neuroscience background and experts that study AI, aiming to provide a comprehensive understanding for AI-powered multimodal BCIs.