Expanding the Role of Affective Phenomena in Multimodal Interaction Research
This work identifies gaps in affective computing research for improving AI systems in human-machine interaction, but it is incremental as it reviews existing literature without proposing new methods.
The paper analyzed over 16,000 papers from multimodal interaction and affective computing conferences, identifying 910 affect-related studies, and found that research primarily focuses on machine recognition and expression of affect, with limited exploration of using affect predictions to enhance AI understanding of human social behaviors and cognitive states.
In recent decades, the field of affective computing has made substantial progress in advancing the ability of AI systems to recognize and express affective phenomena, such as affect and emotions, during human-human and human-machine interactions. This paper describes our examination of research at the intersection of multimodal interaction and affective computing, with the objective of observing trends and identifying understudied areas. We examined over 16,000 papers from selected conferences in multimodal interaction, affective computing, and natural language processing: ACM International Conference on Multimodal Interaction, AAAC International Conference on Affective Computing and Intelligent Interaction, Annual Meeting of the Association for Computational Linguistics, and Conference on Empirical Methods in Natural Language Processing. We identified 910 affect-related papers and present our analysis of the role of affective phenomena in these papers. We find that this body of research has primarily focused on enabling machines to recognize and express affect and emotion. However, we find limited research on how affect and emotion predictions might be used by AI systems to enhance machine understanding of human social behaviors and cognitive states. Based on our analysis, we discuss directions to expand the role of affective phenomena in multimodal interaction research.