Towards an Analysis of Discourse and Interactional Pragmatic Reasoning Capabilities of Large Language Models
It provides a structured overview for researchers interested in evaluating LLMs' pragmatic abilities, but it is incremental as it synthesizes prior work without introducing new methods or results.
The paper surveys existing research on the pragmatic reasoning capabilities of large language models, categorizing them into discourse and interactional pragmatics and reviewing tested phenomena and methods.
In this work, we want to give an overview on which pragmatic abilities have been tested in LLMs so far and how these tests have been carried out. To do this, we first discuss the scope of the field of pragmatics and suggest a subdivision into discourse pragmatics and interactional pragmatics. We give a non-exhaustive overview of the phenomena of those two subdomains and the methods traditionally used to analyze them. We subsequently consider the resulting heterogeneous set of phenomena and methods as a starting point for our survey of work on discourse pragmatics and interactional pragmatics in the context of LLMs.