Evaluation of Human Visual Privacy Protection: A Three-Dimensional Framework and Benchmark Dataset
This addresses the need for standardized evaluation in privacy-by-design solutions for AI surveillance, offering a benchmark and framework, though it is incremental in building on existing privacy evaluation efforts.
This paper tackles the problem of objectively evaluating visual privacy-protection methods by proposing a three-dimensional framework (privacy, utility, practicality) and introducing the HR-VISPR dataset with human-centric labels. The result includes evaluating 11 methods to highlight trade-offs and provide a structured tool for diverse contexts.
Recent advances in AI-powered surveillance have intensified concerns over the collection and processing of sensitive personal data. In response, research has increasingly focused on privacy-by-design solutions, raising the need for objective techniques to evaluate privacy protection. This paper presents a comprehensive framework for evaluating visual privacy-protection methods across three dimensions: privacy, utility, and practicality. In addition, it introduces HR-VISPR, a publicly available human-centric dataset with biometric, soft-biometric, and non-biometric labels to train an interpretable privacy metric. We evaluate 11 privacy protection methods, ranging from conventional techniques to advanced deep-learning methods, through the proposed framework. The framework differentiates privacy levels in alignment with human visual perception, while highlighting trade-offs between privacy, utility, and practicality. This study, along with the HR-VISPR dataset, serves as an insightful tool and offers a structured evaluation framework applicable across diverse contexts.