Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant
This work addresses security concerns for machine learning practitioners by improving efficiency in encrypted data training, though it is incremental as it builds on existing gradient methods.
The paper tackled the problem of slow convergence in privacy-preserving logistic regression training by introducing a quadratic gradient variant, which enhanced optimization algorithms like NAG, AdaGrad, and Adam to achieve state-of-the-art convergence rates and enabled homomorphic training with comparable performance in only four iterations.
Training logistic regression over encrypted data has emerged as a prominent approach to addressing security concerns in recent years. In this paper, we introduce an efficient gradient variant, termed the \textit{quadratic gradient}, which is specifically designed for privacy-preserving logistic regression while remaining equally effective in plaintext optimization. By incorporating this quadratic gradient, we enhance Nesterov's Accelerated Gradient (NAG), Adaptive Gradient (AdaGrad), and Adam algorithms. We evaluate these enhanced algorithms across various datasets, with experimental results demonstrating state-of-the-art convergence rates that significantly outperform traditional first-order gradient methods. Furthermore, we apply the enhanced NAG method to implement homomorphic logistic regression training, achieving comparable performance within only four iterations. The proposed quadratic-gradient approach offers a unified framework that synergizes the advantages of first-order gradient methods and second-order Newton-type methods, suggesting broad applicability to diverse numerical optimization tasks.