Counterfactual Augmentation for Multimodal Learning Under Presentation Bias
This addresses bias in real-world ML systems for practitioners, but it is incremental as it builds on existing causal methods for bias correction.
The paper tackles the problem of presentation bias in machine learning systems caused by feedback loops, proposing counterfactual augmentation to correct it, and shows it yields better downstream performance compared to uncorrected models and existing methods.
In real-world machine learning systems, labels are often derived from user behaviors that the system wishes to encourage. Over time, new models must be trained as new training examples and features become available. However, feedback loops between users and models can bias future user behavior, inducing a presentation bias in the labels that compromises the ability to train new models. In this paper, we propose counterfactual augmentation, a novel causal method for correcting presentation bias using generated counterfactual labels. Our empirical evaluations demonstrate that counterfactual augmentation yields better downstream performance compared to both uncorrected models and existing bias-correction methods. Model analyses further indicate that the generated counterfactuals align closely with true counterfactuals in an oracle setting.