Variational Leakage: The Role of Information Complexity in Privacy Leakage
It addresses privacy risks in machine learning for scenarios where sensitive attributes are not predefined, but the approach is incremental as it builds on existing variational methods.
The paper investigates how information complexity influences privacy leakage of unknown sensitive attributes in supervised representation learning, finding that factors like regularizer weight and latent dimension affect intrinsic leakage, with experiments on Colored-MNIST and CelebA datasets.
We study the role of information complexity in privacy leakage about an attribute of an adversary's interest, which is not known a priori to the system designer. Considering the supervised representation learning setup and using neural networks to parameterize the variational bounds of information quantities, we study the impact of the following factors on the amount of information leakage: information complexity regularizer weight, latent space dimension, the cardinalities of the known utility and unknown sensitive attribute sets, the correlation between utility and sensitive attributes, and a potential bias in a sensitive attribute of adversary's interest. We conduct extensive experiments on Colored-MNIST and CelebA datasets to evaluate the effect of information complexity on the amount of intrinsic leakage.