MEGnifying Emotion: Sentiment Analysis from Annotated Brain Data
This work addresses the challenge of understanding human emotion from brain data for neuroscience and AI applications, but it is incremental as it builds on existing methods and datasets.
The paper tackled the problem of decoding emotion from brain activity by creating a new dataset that annotates MEG recordings with sentiment labels using pre-trained text-to-sentiment models and force-alignment, resulting in an improvement in balanced accuracy for brain-to-sentiment models compared to baseline.
Decoding emotion from brain activity could unlock a deeper understanding of the human experience. While a number of existing datasets align brain data with speech and with speech transcripts, no datasets have annotated brain data with sentiment. To bridge this gap, we explore the use of pre-trained Text-to-Sentiment models to annotate non invasive brain recordings, acquired using magnetoencephalography (MEG), while participants listened to audiobooks. Having annotated the text, we employ force-alignment of the text and audio to align our sentiment labels with the brain recordings. It is straightforward then to train Brainto-Sentiment models on these data. Experimental results show an improvement in balanced accuracy for Brain-to-Sentiment compared to baseline, supporting the proposed approach as a proof-of-concept for leveraging existing MEG datasets and learning to decode sentiment directly from the brain.