VP-NTK: Exploring the Benefits of Visual Prompting in Differentially Private Data Synthesis
This work addresses the challenge of generating useful synthetic data under privacy constraints, particularly for high-resolution images, representing an incremental advance in DP generative models.
The paper tackled the problem of low utility in differentially private synthetic data for high-resolution images by combining visual prompting with a DP-NTK generator, achieving an accuracy improvement from 0.644±0.044 to 0.769.
Differentially private (DP) synthetic data has become the de facto standard for releasing sensitive data. However, many DP generative models suffer from the low utility of synthetic data, especially for high-resolution images. On the other hand, one of the emerging techniques in parameter efficient fine-tuning (PEFT) is visual prompting (VP), which allows well-trained existing models to be reused for the purpose of adapting to subsequent downstream tasks. In this work, we explore such a phenomenon in constructing captivating generative models with DP constraints. We show that VP in conjunction with DP-NTK, a DP generator that exploits the power of the neural tangent kernel (NTK) in training DP generative models, achieves a significant performance boost, particularly for high-resolution image datasets, with accuracy improving from 0.644$\pm$0.044 to 0.769. Lastly, we perform ablation studies on the effect of different parameters that influence the overall performance of VP-NTK. Our work demonstrates a promising step forward in improving the utility of DP synthetic data, particularly for high-resolution images.