Learning Fair Representation via Distributional Contrastive Disentanglement
This work addresses fairness in machine learning by improving representation learning, though it is incremental as it builds on existing disentanglement and contrastive learning approaches.
The authors tackled the problem of learning fair representations by proposing FarconVAE, a method that disentangles latent space into sensitive and nonsensitive parts using distributional contrastive learning, achieving superior performance on fairness, debiasing, and domain generalization tasks across tabular, image, and text modalities.
Learning fair representation is crucial for achieving fairness or debiasing sensitive information. Most existing works rely on adversarial representation learning to inject some invariance into representation. However, adversarial learning methods are known to suffer from relatively unstable training, and this might harm the balance between fairness and predictiveness of representation. We propose a new approach, learning FAir Representation via distributional CONtrastive Variational AutoEncoder (FarconVAE), which induces the latent space to be disentangled into sensitive and nonsensitive parts. We first construct the pair of observations with different sensitive attributes but with the same labels. Then, FarconVAE enforces each non-sensitive latent to be closer, while sensitive latents to be far from each other and also far from the non-sensitive latent by contrasting their distributions. We provide a new type of contrastive loss motivated by Gaussian and Student-t kernels for distributional contrastive learning with theoretical analysis. Besides, we adopt a new swap-reconstruction loss to boost the disentanglement further. FarconVAE shows superior performance on fairness, pretrained model debiasing, and domain generalization tasks from various modalities, including tabular, image, and text.