A Framework for Personalized Persuasiveness Prediction via Context-Aware User Profiling
This work addresses personalized persuasiveness prediction for applications like recommender systems and LLM safety, but it is incremental as it builds on existing methods with a novel framework.
The paper tackled the problem of predicting message persuasiveness by proposing a context-aware user profiling framework, which improved F1 scores by up to +13.77%p on the ChangeMyView Reddit dataset compared to existing methods.
Estimating the persuasiveness of messages is critical in various applications, from recommender systems to safety assessment of LLMs. While it is imperative to consider the target persuadee's characteristics, such as their values, experiences, and reasoning styles, there is currently no established systematic framework to optimize leveraging a persuadee's past activities (e.g., conversations) to the benefit of a persuasiveness prediction model. To address this problem, we propose a context-aware user profiling framework with two trainable components: a query generator that generates optimal queries to retrieve persuasion-relevant records from a user's history, and a profiler that summarizes these records into a profile to effectively inform the persuasiveness prediction model. Our evaluation on the ChangeMyView Reddit dataset shows consistent improvements over existing methods across multiple predictor models, with gains of up to +13.77%p in F1 score. Further analysis shows that effective user profiles are context-dependent and predictor-specific, rather than relying on static attributes or surface-level similarity. Together, these results highlight the importance of task-oriented, context-dependent user profiling for personalized persuasiveness prediction.