Bundle Network: a Machine Learning-Based Bundle Method
This work addresses the need for automated parameter tuning in optimization methods for researchers and practitioners in machine learning and operations research, representing an incremental improvement over classical bundle methods.
The paper tackles the problem of tuning regularization parameters in convex non-smooth minimization by introducing Bundle Network, a learning-based algorithm that automatically adjusts parameters from data and replaces iterative optimization with a recurrent neural model, resulting in consistent outperformance of traditional methods on Lagrangian dual relaxations of Multi-Commodity Network Design and Generalized Assignment problems.
This paper presents Bundle Network, a learning-based algorithm inspired by the Bundle Method for convex non-smooth minimization problems. Unlike classical approaches that rely on heuristic tuning of a regularization parameter, our method automatically learns to adjust it from data. Furthermore, we replace the iterative resolution of the optimization problem that provides the search direction-traditionally computed as a convex combination of gradients at visited points-with a recurrent neural model equipped with an attention mechanism. By leveraging the unrolled graph of computation, our Bundle Network can be trained end-to-end via automatic differentiation. Experiments on Lagrangian dual relaxations of the Multi-Commodity Network Design and Generalized Assignment problems demonstrate that our approach consistently outperforms traditional methods relying on grid search for parameter tuning, while generalizing effectively across datasets.