Privacy-Preserving Logistic Regression Training with A Faster Gradient Variant
This work addresses security concerns in machine learning for privacy-sensitive domains, offering an incremental improvement over existing methods.
The paper tackles the problem of training logistic regression on encrypted data by introducing a quadratic gradient variant, which enhances several optimization algorithms and achieves state-of-the-art convergence rates, with homomorphic training reaching comparable performance in only four iterations.
Training logistic regression over encrypted data has emerged as a prominent approach to addressing security concerns in recent years. In this paper, we introduce an efficient gradient variant, termed the \textit{quadratic gradient}, which is specifically designed for privacy-preserving logistic regression while remaining equally effective in plaintext optimization. By incorporating this quadratic gradient, we enhance Nesterov's Accelerated Gradient (NAG), Adaptive Gradient (AdaGrad), and Adam algorithms. We evaluate these enhanced algorithms across various datasets, with experimental results demonstrating state-of-the-art convergence rates that significantly outperform traditional first-order gradient methods. Furthermore, we apply the enhanced NAG method to implement homomorphic logistic regression training, achieving comparable performance within only four iterations. The proposed quadratic-gradient approach offers a unified framework that synergizes the advantages of first-order gradient methods and second-order Newton-type methods, suggesting broad applicability to diverse numerical optimization tasks.