Privacy Amplification for BandMF via $b$-Min-Sep Subsampling
For practitioners of DP-SGD seeking tighter privacy guarantees, this work offers a practical subsampling method that improves privacy amplification without sacrificing structural properties.
The paper proposes a new subsampling scheme, $b$-min-sep, for BandMF (DP-SGD with banded correlation noise) that provides stronger privacy amplification than cyclic Poisson subsampling in the mid-to-low noise regime, with near-exact privacy analysis via Monte Carlo accounting.
We study privacy amplification for BandMF, i.e., DP-SGD with correlated noise across iterations via a banded correlation matrix. We propose $b$-min-sep subsampling, a new subsampling scheme that generalizes Poisson and balls-in-bins subsampling, extends prior practical batching strategies for BandMF, and enables stronger privacy amplification than cyclic Poisson while preserving the structural properties needed for analysis. We give a near-exact privacy analysis using Monte Carlo accounting, based on a dynamic program that leverages the Markovian structure in the subsampling procedure. We show that $b$-min-sep matches cyclic Poisson subsampling in the high noise regime and achieves strictly better guarantees in the mid-to-low noise regime, with experimental results that bolster our claims. We further show that unlike previous BandMF subsampling schemes, our $b$-min-sep subsampling naturally extends to the multi-attribution user-level privacy setting.