Hidden State Differential Private Mini-Batch Block Coordinate Descent for Multi-convexity Optimization
This work addresses privacy concerns in machine learning for practitioners dealing with multi-convex problems, offering incremental improvements in privacy analysis.
The paper tackles the problem of differential privacy guarantees under hidden state assumptions for multi-convex optimization, introducing the DP-MBCD algorithm that provides tighter privacy loss bounds and applies to non-convex problems like matrix factorization and neural network training.
We investigate the differential privacy (DP) guarantees under the hidden state assumption (HSA) for multi-convex problems. Recent analyses of privacy loss under the hidden state assumption have relied on strong assumptions such as convexity, thereby limiting their applicability to practical problems. In this paper, we introduce the Differential Privacy Mini-Batch Block Coordinate Descent (DP-MBCD) algorithm, accompanied by the privacy loss accounting methods under the hidden state assumption. Our proposed methods apply to a broad range of classical non-convex problems which are or can be converted to multi-convex problems, such as matrix factorization and neural network training. In addition to a tighter bound for privacy loss, our theoretical analysis is also compatible with proximal gradient descent and adaptive calibrated noise scenarios.