Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection
This work addresses bias elimination in machine learning for fairness applications, offering a more efficient and cleaner approach than existing methods, though it is incremental in improving upon INLP.
The paper tackles the problem of removing protected attributes from embedding spaces while preserving other information, introducing Mean Projection (MP) and Tukey Median Projection (TMP) as methods that use a single projection to achieve this, with MP showing less impact on the overall space compared to the iterative INLP method.
Bias elimination and recent probing studies attempt to remove specific information from embedding spaces. Here it is important to remove as much of the target information as possible, while preserving any other information present. INLP is a popular recent method which removes specific information through iterative nullspace projections. Multiple iterations, however, increase the risk that information other than the target is negatively affected. We introduce two methods that find a single targeted projection: Mean Projection (MP, more efficient) and Tukey Median Projection (TMP, with theoretical guarantees). Our comparison between MP and INLP shows that (1) one MP projection removes linear separability based on the target and (2) MP has less impact on the overall space. Further analysis shows that applying random projections after MP leads to the same overall effects on the embedding space as the multiple projections of INLP. Applying one targeted (MP) projection hence is methodologically cleaner than applying multiple (INLP) projections that introduce random effects.