EMBRACE: Shaping Inclusive Opinion Representation by Aligning Implicit Conversations with Social Norms
This addresses the issue of fair and diverse representation in NLP models, which is incremental as it builds on existing methods by emphasizing implicit aspects.
The study tackled the problem of shaping inclusive opinion representation in conversation-based models by introducing an alignment evaluation framework that focuses on implicit conversations and social norms, using stance as a proxy for opinion to assess normative alignment with methods like positive-unlabeled learning and instruction-tuned models.
Shaping inclusive representations that embrace diversity and ensure fair participation and reflections of values is at the core of many conversation-based models. However, many existing methods rely on surface inclusion using mention of user demographics or behavioral attributes of social groups. Such methods overlook the nuanced, implicit expression of opinion embedded in conversations. Furthermore, the over-reliance on overt cues can exacerbate misalignment and reinforce harmful or stereotypical representations in model outputs. Thus, we took a step back and recognized that equitable inclusion needs to account for the implicit expression of opinion and use the stance of responses to validate the normative alignment. This study aims to evaluate how opinions are represented in NLP or computational models by introducing an alignment evaluation framework that foregrounds implicit, often overlooked conversations and evaluates the normative social views and discourse. Our approach models the stance of responses as a proxy for the underlying opinion, enabling a considerate and reflective representation of diverse social viewpoints. We evaluate the framework using both (i) positive-unlabeled (PU) online learning with base classifiers, and (ii) instruction-tuned language models to assess post-training alignment. Through this, we provide a principled and structured lens on how implicit opinions are (mis)represented and offer a pathway toward more inclusive model behavior.