Expanding the Set of Pragmatic Considerations in Conversational AI
This work addresses the problem of unmet user expectations in conversational AI, though it is incremental as it focuses on taxonomy development without new methods or data.
The paper identifies pragmatic limitations in conversational AI systems that cause them to fall short of human behavior, proposing a taxonomy of pragmatic considerations to guide future design and evaluation.
Despite considerable performance improvements, current conversational AI systems often fail to meet user expectations. We discuss several pragmatic limitations of current conversational AI systems. We illustrate pragmatic limitations with examples that are syntactically appropriate, but have clear pragmatic deficiencies. We label our complaints as "Turing Test Triggers" (TTTs) as they indicate where current conversational AI systems fall short compared to human behavior. We develop a taxonomy of pragmatic considerations intended to identify what pragmatic competencies a conversational AI system requires and discuss implications for the design and evaluation of conversational AI systems.