Learning to accelerate distributed ADMM using graph neural networks
This work addresses optimization efficiency for large-scale machine learning and control applications, representing an incremental improvement through learned hyperparameters.
The paper tackled the slow convergence and hyperparameter sensitivity of distributed ADMM by learning adaptive step sizes and communication weights using a graph neural network, resulting in improved convergence speed and solution quality compared to standard ADMM.
Distributed optimization is fundamental in large-scale machine learning and control applications. Among existing methods, the Alternating Direction Method of Multipliers (ADMM) has gained popularity due to its strong convergence guarantees and suitability for decentralized computation. However, ADMM often suffers from slow convergence and sensitivity to hyperparameter choices. In this work, we show that distributed ADMM iterations can be naturally represented within the message-passing framework of graph neural networks (GNNs). Building on this connection, we propose to learn adaptive step sizes and communication weights by a graph neural network that predicts the hyperparameters based on the iterates. By unrolling ADMM for a fixed number of iterations, we train the network parameters end-to-end to minimize the final iterates error for a given problem class, while preserving the algorithm's convergence properties. Numerical experiments demonstrate that our learned variant consistently improves convergence speed and solution quality compared to standard ADMM. The code is available at https://github.com/paulhausner/learning-distributed-admm.