Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning
For researchers in dialogue systems and computational pragmatics, this work addresses the underexplored relationship between backchannel form and meaning, providing a method to align backchannel and context representations.
The authors propose a two-stage framework that fine-tunes LLMs on dialogue transcripts and learns a joint embedding space for dialogue contexts and backchannel realizations, achieving substantial improvements in context-backchannel retrieval over prior methods and aligning more closely with human judgments than raw WavLM features.
Backchannels (e.g., `yeah', `mhm', and `right') are short, non-interruptive feedback signals whose lexical form and prosody jointly convey pragmatic meaning. While prior computational research has largely focused on predicting backchannel timing, the relationship between lexico-prosodic form and meaning remains underexplored. We propose a two-stage framework: first, fine-tuning large language models on dialogue transcripts to derive rich contextual representations; and second, learning a joint embedding space for dialogue contexts and backchannel realizations. We evaluate alignment with human perception via triadic similarity judgments (prosodic and cross-lexical) and a context-backchannel suitability task. Our results demonstrate that the learned projections substantially improve context-backchannel retrieval compared to previous methods. In addition, they reveal that backchannel form is highly sensitive to extended conversational context and that the learned embeddings align more closely with human judgments than raw WavLM features.