Breaking the Spurious Causality of Conditional Generation via Fairness Intervention with Corrective Sampling
This addresses fairness issues in generative models for applications like image synthesis, though it is incremental as it builds on existing fairness interventions.
The paper tackles the problem of spurious correlations in conditional generative models, which cause imbalanced label-conditional distributions with respect to latent attributes, by proposing a two-step strategy called FICS (Fairness Intervention with Corrective Sampling) that emphasizes minority samples and filters generated outputs to align with desired distributions, demonstrating effectiveness across various datasets.
To capture the relationship between samples and labels, conditional generative models often inherit spurious correlations from the training dataset. This can result in label-conditional distributions that are imbalanced with respect to another latent attribute. To mitigate this issue, which we call spurious causality of conditional generation, we propose a general two-step strategy. (a) Fairness Intervention (FI): emphasize the minority samples that are hard to generate due to the spurious correlation in the training dataset. (b) Corrective Sampling (CS): explicitly filter the generated samples and ensure that they follow the desired latent attribute distribution. We have designed the fairness intervention to work for various degrees of supervision on the spurious attribute, including unsupervised, weakly-supervised, and semi-supervised scenarios. Our experimental results demonstrate that FICS can effectively resolve spurious causality of conditional generation across various datasets.