Fast Fourier Transform-Based Spectral and Temporal Gradient Filtering for Differential Privacy
This addresses the problem of balancing privacy and utility in machine learning for data-sensitive applications, representing an incremental improvement over existing methods.
The paper tackled the accuracy loss in differentially private machine learning by introducing the FFT-Enhanced Kalman Filter, which improved gradient quality through frequency-domain filtering and achieved higher test accuracy on datasets like MNIST and CIFAR-10 with per-iteration complexity O(d log d).
Differential Privacy (DP) has emerged as a key framework for protecting sensitive data in machine learning, but standard DP-SGD often suffers from significant accuracy loss due to injected noise. To address this limitation, we introduce the FFT-Enhanced Kalman Filter (FFTKF), a differentially private optimization method that improves gradient quality while preserving $(\varepsilon, δ)$-DP guarantees. FFTKF applies frequency-domain filtering to shift privacy noise into less informative high-frequency components, preserving the low-frequency gradient signals that carry most learning information. A scalar-gain Kalman filter with a finite-difference Hessian approximation further refines the denoised gradients. The method has per-iteration complexity $\mathcal{O}(d \log d)$ and achieves higher test accuracy than DP-SGD and DiSK on MNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet with CNNs, Wide ResNets, and Vision Transformers. Theoretical analysis shows that FFTKF ensures equivalent privacy while delivering a stronger privacy--utility trade-off through reduced variance and controlled bias.