PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliance
This addresses privacy evaluation for LLM developers and regulators by providing a more comprehensive benchmark, though it is incremental as it builds on existing evaluation frameworks.
The paper tackles the problem of evaluating privacy in large language models (LLMs) by introducing PrivaCI-Bench, a benchmark based on Contextual Integrity theory to assess privacy and legal compliance, finding that while LLMs capture key contextual parameters, they need improvements for full compliance.
Recent advancements in generative large language models (LLMs) have enabled wider applicability, accessibility, and flexibility. However, their reliability and trustworthiness are still in doubt, especially for concerns regarding individuals' data privacy. Great efforts have been made on privacy by building various evaluation benchmarks to study LLMs' privacy awareness and robustness from their generated outputs to their hidden representations. Unfortunately, most of these works adopt a narrow formulation of privacy and only investigate personally identifiable information (PII). In this paper, we follow the merit of the Contextual Integrity (CI) theory, which posits that privacy evaluation should not only cover the transmitted attributes but also encompass the whole relevant social context through private information flows. We present PrivaCI-Bench, a comprehensive contextual privacy evaluation benchmark targeted at legal compliance to cover well-annotated privacy and safety regulations, real court cases, privacy policies, and synthetic data built from the official toolkit to study LLMs' privacy and safety compliance. We evaluate the latest LLMs, including the recent reasoner models QwQ-32B and Deepseek R1. Our experimental results suggest that though LLMs can effectively capture key CI parameters inside a given context, they still require further advancements for privacy compliance.