Modular Distributed Nonconvex Learning with Error Feedback
This addresses efficient and robust distributed optimization for nonconvex problems in machine learning, though it appears incremental as it builds on existing methods like ADMM and gradient approaches.
The paper tackles distributed nonconvex learning with compressed communications by proposing a modular algorithm combining ADMM and gradient methods with error feedback, guaranteeing almost sure asymptotic convergence to stationary points, as validated by numerical simulations in nonconvex classification.
In this paper, we design a novel distributed learning algorithm using stochastic compressed communications. In detail, we pursue a modular approach, merging ADMM and a gradient-based approach, benefiting from the robustness of the former and the computational efficiency of the latter. Additionally, we integrate a stochastic integral action (error feedback) enabling almost sure rejection of the compression error. We analyze the resulting method in nonconvex scenarios and guarantee almost sure asymptotic convergence to the set of stationary points of the problem. This result is obtained using system-theoretic tools based on stochastic timescale separation. We corroborate our findings with numerical simulations in nonconvex classification.