DP-TBART: A Transformer-based Autoregressive Model for Differentially Private Tabular Data Generation
This work addresses the need for improved deep learning-based methods for generating differentially private synthetic tabular data, offering a viable alternative to traditional marginal-based approaches, though it appears incremental as it builds on existing transformer and autoregressive models.
The paper tackles the problem of generating differentially private synthetic tabular data, where deep learning methods have lagged behind marginal-based approaches, and presents DP-TBART, a transformer-based autoregressive model that achieves competitive performance, even outperforming state-of-the-art methods in some settings.
The generation of synthetic tabular data that preserves differential privacy is a problem of growing importance. While traditional marginal-based methods have achieved impressive results, recent work has shown that deep learning-based approaches tend to lag behind. In this work, we present Differentially-Private TaBular AutoRegressive Transformer (DP-TBART), a transformer-based autoregressive model that maintains differential privacy and achieves performance competitive with marginal-based methods on a wide variety of datasets, capable of even outperforming state-of-the-art methods in certain settings. We also provide a theoretical framework for understanding the limitations of marginal-based approaches and where deep learning-based approaches stand to contribute most. These results suggest that deep learning-based techniques should be considered as a viable alternative to marginal-based methods in the generation of differentially private synthetic tabular data.