MuSe-Toolbox: The Multimodal Sentiment Analysis Continuous Annotation Fusion and Discrete Class Transformation Toolbox
This is an incremental contribution for researchers in multimodal sentiment analysis, offering a toolkit to standardize and transform emotion annotations.
The paper tackles the problem of creating continuous and discrete emotion gold standards for multimodal sentiment analysis by introducing the MuSe-Toolbox, a Python-based toolkit that unifies fusion methods and proposes the novel Rater Aligned Annotation Weighting (RAAW) method, with experimental results showing it provides promising class formations that can be better predicted than hard-coded boundaries with minimal human intervention.
We introduce the MuSe-Toolbox - a Python-based open-source toolkit for creating a variety of continuous and discrete emotion gold standards. In a single framework, we unify a wide range of fusion methods and propose the novel Rater Aligned Annotation Weighting (RAAW), which aligns the annotations in a translation-invariant way before weighting and fusing them based on the inter-rater agreements between the annotations. Furthermore, discrete categories tend to be easier for humans to interpret than continuous signals. With this in mind, the MuSe-Toolbox provides the functionality to run exhaustive searches for meaningful class clusters in the continuous gold standards. To our knowledge, this is the first toolkit that provides a wide selection of state-of-the-art emotional gold standard methods and their transformation to discrete classes. Experimental results indicate that MuSe-Toolbox can provide promising and novel class formations which can be better predicted than hard-coded classes boundaries with minimal human intervention. The implementation (1) is out-of-the-box available with all dependencies using a Docker container (2).