Inducing Uncertainty on Open-Weight Models for Test-Time Privacy in Image Recognition
This addresses a safety concern for users of AI models by providing a method to protect against harmful predictions in image recognition, though it is incremental in applying privacy techniques to a specific threat.
The paper tackles the problem of test-time privacy in open-weight models, where adversaries might misuse predictions on personal data, by inducing maximal uncertainty on protected instances while maintaining accuracy on others, achieving over 3 times stronger uncertainty with marginal accuracy drops on image recognition benchmarks.
A key concern for AI safety remains understudied in the machine learning (ML) literature: how can we ensure users of ML models do not leverage predictions on incorrect personal data to harm others? This is particularly pertinent given the rise of open-weight models, where simply masking model outputs does not suffice to prevent adversaries from recovering harmful predictions. To address this threat, which we call *test-time privacy*, we induce maximal uncertainty on protected instances while preserving accuracy on all other instances. Our proposed algorithm uses a Pareto optimal objective that explicitly balances test-time privacy against utility. We also provide a certifiable approximation algorithm which achieves $(\varepsilon, δ)$ guarantees without convexity assumptions. We then prove a tight bound that characterizes the privacy-utility tradeoff that our algorithms incur. Empirically, our method obtains at least $>3\times$ stronger uncertainty than pretraining with marginal drops in accuracy on various image recognition benchmarks. Altogether, this framework provides a tool to guarantee additional protection to end users.