Privacy Reasoning in Ambiguous Contexts
This addresses privacy reasoning for AI agents, but it is incremental as it builds on prior work by focusing on ambiguity.
The paper tackles the problem of language models reasoning about information disclosure in ambiguous contexts, showing that their Camber framework for context disambiguation improves accuracy by up to 13.3% in precision and 22.3% in recall.
We study the ability of language models to reason about appropriate information disclosure - a central aspect of the evolving field of agentic privacy. Whereas previous works have focused on evaluating a model's ability to align with human decisions, we examine the role of ambiguity and missing context on model performance when making information-sharing decisions. We identify context ambiguity as a crucial barrier for high performance in privacy assessments. By designing Camber, a framework for context disambiguation, we show that model-generated decision rationales can reveal ambiguities and that systematically disambiguating context based on these rationales leads to significant accuracy improvements (up to 13.3\% in precision and up to 22.3\% in recall) as well as reductions in prompt sensitivity. Overall, our results indicate that approaches for context disambiguation are a promising way forward to enhance agentic privacy reasoning.