Is API Access to LLMs Useful for Generating Private Synthetic Tabular Data?
This work addresses the challenge of privacy-preserving data synthesis for data analysts when model weights are unavailable, but it is incremental as it adapts existing methods and shows limited practical impact.
The paper tackled the problem of generating differentially private synthetic tabular data using only API access to large language models (LLMs), rather than requiring model weights, and found that this approach does not consistently improve data quality compared to existing baselines, with results showing no significant gains in utility metrics.
Differentially private (DP) synthetic data is a versatile tool for enabling the analysis of private data. Recent advancements in large language models (LLMs) have inspired a number of algorithm techniques for improving DP synthetic data generation. One family of approaches uses DP finetuning on the foundation model weights; however, the model weights for state-of-the-art models may not be public. In this work we propose two DP synthetic tabular data algorithms that only require API access to the foundation model. We adapt the Private Evolution algorithm (Lin et al., 2023; Xie et al., 2024) -- which was designed for image and text data -- to the tabular data domain. In our extension of Private Evolution, we define a query workload-based distance measure, which may be of independent interest. We propose a family of algorithms that use one-shot API access to LLMs, rather than adaptive queries to the LLM. Our findings reveal that API-access to powerful LLMs does not always improve the quality of DP synthetic data compared to established baselines that operate without such access. We provide insights into the underlying reasons and propose improvements to LLMs that could make them more effective for this application.