Computational Analysis of Conversation Dynamics through Participant Responsivity
This work addresses the need for better tools to analyze conversation quality, particularly for researchers studying discourse dynamics, though it is incremental as it builds on existing notions of responsivity with new computational methods.
The paper tackled the problem of characterizing prosocial and constructive dialogue by introducing a method based on 'responsivity' to quantify whether conversational turns respond to each other, achieving evaluation against human-annotated ground truth and enabling meaningful differentiation across diverse conversations.
Growing literature explores toxicity and polarization in discourse, with comparatively less work on characterizing what makes dialogue prosocial and constructive. We explore conversational discourse and investigate a method for characterizing its quality built upon the notion of ``responsivity'' -- whether one person's conversational turn is responding to a preceding turn. We develop and evaluate methods for quantifying responsivity -- first through semantic similarity of speaker turns, and second by leveraging state-of-the-art large language models (LLMs) to identify the relation between two speaker turns. We evaluate both methods against a ground truth set of human-annotated conversations. Furthermore, selecting the better performing LLM-based approach, we characterize the nature of the response -- whether it responded to that preceding turn in a substantive way or not. We view these responsivity links as a fundamental aspect of dialogue but note that conversations can exhibit significantly different responsivity structures. Accordingly, we then develop conversation-level derived metrics to address various aspects of conversational discourse. We use these derived metrics to explore other conversations and show that they support meaningful characterizations and differentiations across a diverse collection of conversations.