Differentially Private Tabular Data Synthesis using Large Language Models
This addresses the problem of enabling private data sharing for data analysts and researchers, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles the challenge of generating realistic synthetic tabular data with differential privacy by introducing DP-LLMTGen, a framework that uses fine-tuned large language models, and demonstrates it outperforms existing mechanisms across multiple datasets and privacy settings.
Synthetic tabular data generation with differential privacy is a crucial problem to enable data sharing with formal privacy. Despite a rich history of methodological research and development, developing differentially private tabular data generators that can provide realistic synthetic datasets remains challenging. This paper introduces DP-LLMTGen -- a novel framework for differentially private tabular data synthesis that leverages pretrained large language models (LLMs). DP-LLMTGen models sensitive datasets using a two-stage fine-tuning procedure with a novel loss function specifically designed for tabular data. Subsequently, it generates synthetic data through sampling the fine-tuned LLMs. Our empirical evaluation demonstrates that DP-LLMTGen outperforms a variety of existing mechanisms across multiple datasets and privacy settings. Additionally, we conduct an ablation study and several experimental analyses to deepen our understanding of LLMs in addressing this important problem. Finally, we highlight the controllable generation ability of DP-LLMTGen through a fairness-constrained generation setting.