Discourse over Discourse: The Need for an Expanded Pragmatic Focus in Conversational AI
It addresses the problem of AI lacking pragmatic understanding for researchers and developers in conversational AI, but is incremental as it builds on existing theoretical work.
The paper argues that pragmatic considerations are a key limitation in conversational AI, particularly in conversation summarization, and introduces 'Turing Test Triggers' to highlight issues where AI fails to mimic human behavior.
The summarization of conversation, that is, discourse over discourse, elevates pragmatic considerations as a pervasive limitation of both summarization and other applications of contemporary conversational AI. Building on impressive progress in both semantics and syntax, pragmatics concerns meaning in the practical sense. In this paper, we discuss several challenges in both summarization of conversations and other conversational AI applications, drawing on relevant theoretical work. We illustrate the importance of pragmatics with so-called star sentences, syntactically acceptable propositions that are pragmatically inappropriate in conversation or its summary. Because the baseline for quality of AI is indistinguishability from human behavior, we draw heavily on the psycho-linguistics literature, and label our complaints as "Turing Test Triggers" (TTTs). We discuss implications for the design and evaluation of conversation summarization methods and conversational AI applications like voice assistants and chatbots