TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages
This addresses the problem of cultural-specific politeness analysis for multilingual NLP researchers, though it is incremental as it extends existing English-focused work to new languages.
The authors tackled the lack of computational politeness studies beyond English by creating TyDiP, a dataset with 4.5K examples across nine languages, and found that multilingual models show robust zero-shot transfer but fall significantly short of human accuracy.
We study politeness phenomena in nine typologically diverse languages. Politeness is an important facet of communication and is sometimes argued to be cultural-specific, yet existing computational linguistic study is limited to English. We create TyDiP, a dataset containing three-way politeness annotations for 500 examples in each language, totaling 4.5K examples. We evaluate how well multilingual models can identify politeness levels -- they show a fairly robust zero-shot transfer ability, yet fall short of estimated human accuracy significantly. We further study mapping the English politeness strategy lexicon into nine languages via automatic translation and lexicon induction, analyzing whether each strategy's impact stays consistent across languages. Lastly, we empirically study the complicated relationship between formality and politeness through transfer experiments. We hope our dataset will support various research questions and applications, from evaluating multilingual models to constructing polite multilingual agents.