Value-Action Alignment in Large Language Models under Privacy-Prosocial Conflict
This addresses the challenge of evaluating value-action consistency in LLMs for decision-making tasks involving privacy-prosocial conflicts, which is incremental as it builds on existing evaluations by integrating multiple attitudes.
The paper tackled the problem of assessing whether large language models' expressed values (privacy concerns and prosocialness) predict their data-sharing actions under conflict, by introducing a context-based protocol and a metric called Value-Action Alignment Rate (VAAR). The result showed stable but model-specific profiles and substantial heterogeneity in alignment across multiple LLMs.
Large language models (LLMs) are increasingly used to simulate decision-making tasks involving personal data sharing, where privacy concerns and prosocial motivations can push choices in opposite directions. Existing evaluations often measure privacy-related attitudes or sharing intentions in isolation, which makes it difficult to determine whether a model's expressed values jointly predict its downstream data-sharing actions as in real human behaviors. We introduce a context-based assessment protocol that sequentially administers standardized questionnaires for privacy attitudes, prosocialness, and acceptance of data sharing within a bounded, history-carrying session. To evaluate value-action alignments under competing attitudes, we use multi-group structural equation modeling (MGSEM) to identify relations from privacy concerns and prosocialness to data sharing. We propose Value-Action Alignment Rate (VAAR), a human-referenced directional agreement metric that aggregates path-level evidence for expected signs. Across multiple LLMs, we observe stable but model-specific Privacy-PSA-AoDS profiles, and substantial heterogeneity in value-action alignment.