From Independence to Interaction: Speaker-Aware Simulation of Multi-Speaker Conversational Timing
For researchers in conversational AI and dialogue systems, this work addresses the need for more realistic simulation of conversational timing, though it is an incremental improvement over existing approaches.
The paper introduces a speaker-aware simulation method for multi-speaker conversations that models temporal consistency and realistic turn-taking dynamics, outperforming baseline methods on Switchboard across multiple intrinsic metrics.
We present a speaker-aware approach for simulating multi-speaker conversations that captures temporal consistency and realistic turn-taking dynamics. Prior work typically models aggregate conversational statistics under an independence assumption across speakers and turns. In contrast, our method uses speaker-specific deviation distributions enforcing intra-speaker temporal consistency, while a Markov chain governs turn-taking and a fixed room impulse response preserves spatial realism. We also unify pauses and overlaps into a single gap distribution, modeled with kernel density estimation for smooth continuity. Evaluation on Switchboard using intrinsic metrics - global gap statistics, correlations between consecutive gaps, copula-based higher-order dependencies, turn-taking entropy, and gap survival functions - shows that speaker-aware simulation better aligns with real conversational patterns than the baseline method, capturing fine-grained temporal dependencies and realistic speaker alternation, while revealing open challenges in modeling long-range conversational structure.