Defining and Quantifying Conversation Quality in Spontaneous Interactions
This work addresses the challenge of measuring conversation quality for researchers in social computing and psychology, though it is incremental as it builds on existing annotation methods.
The paper tackles the problem of quantifying social interactions by introducing a novel measure called perceived Conversation Quality to evaluate non-task-directed spontaneous interactions, and finds that naive annotators show lower agreement when assessing low-quality conversations, particularly at the group level.
Social interactions in general are multifaceted and there exists a wide set of factors and events that influence them. In this paper, we quantify social interactions with a holistic viewpoint on individual experiences, particularly focusing on non-task-directed spontaneous interactions. To achieve this, we design a novel perceived measure, the perceived Conversation Quality, which intends to quantify spontaneous interactions by accounting for several socio-dimensional aspects of individual experiences. To further quantitatively study spontaneous interactions, we devise a questionnaire which measures the perceived Conversation Quality, at both the individual- and at the group- level. Using the questionnaire, we collected perceived annotations for conversation quality in a publicly available dataset using naive annotators. The results of the analysis performed on the distribution and the inter-annotator agreeability shows that naive annotators tend to agree less in cases of low conversation quality samples, especially while annotating for group-level conversation quality.