Towards Universal Debiasing for Language Models-based Tabular Data Generation
This work addresses fairness issues in LLM-based tabular data generation for high-stakes applications, offering a scalable solution, though it appears incremental as it builds on existing methods like direct preference optimization.
The authors tackled the problem of historical biases in tabular datasets that cause large language models (LLMs) to exacerbate fairness issues during data generation, and they introduced a universal debiasing framework that effectively balances fairness and utility, as demonstrated through extensive experiments.
Large language models (LLMs) have achieved promising results in tabular data generation. However, inherent historical biases in tabular datasets often cause LLMs to exacerbate fairness issues, particularly when multiple advantaged and protected features are involved. In this work, we introduce a universal debiasing framework that minimizes group-level dependencies by simultaneously reducing the mutual information between advantaged and protected attributes. By leveraging the autoregressive structure and analytic sampling distributions of LLM-based tabular data generators, our approach efficiently computes mutual information, reducing the need for cumbersome numerical estimations. Building on this foundation, we propose two complementary methods: a direct preference optimization (DPO)-based strategy, namely UDF-DPO, that integrates seamlessly with existing models, and a targeted debiasing technique, namely UDF-MIX, that achieves debiasing without tuning the parameters of LLMs. Extensive experiments demonstrate that our framework effectively balances fairness and utility, offering a scalable and practical solution for debiasing in high-stakes applications.