Improving Optimization Bounds using Machine Learning: Decision Diagrams meet Deep Reinforcement Learning
This work addresses the challenge of improving optimization bounds for combinatorial problems like Maximum Independent Set and Maximum Cut, offering a novel machine learning approach that is incremental in applying reinforcement learning to a known bottleneck.
The paper tackles the problem of finding tight bounds for discrete optimization problems by using deep reinforcement learning to optimize variable orderings in decision diagrams, resulting in generally tighter bounds that outperform existing ordering methods on synthetic instances.
Finding tight bounds on the optimal solution is a critical element of practical solution methods for discrete optimization problems. In the last decade, decision diagrams (DDs) have brought a new perspective on obtaining upper and lower bounds that can be significantly better than classical bounding mechanisms, such as linear relaxations. It is well known that the quality of the bounds achieved through this flexible bounding method is highly reliant on the ordering of variables chosen for building the diagram, and finding an ordering that optimizes standard metrics is an NP-hard problem. In this paper, we propose an innovative and generic approach based on deep reinforcement learning for obtaining an ordering for tightening the bounds obtained with relaxed and restricted DDs. We apply the approach to both the Maximum Independent Set Problem and the Maximum Cut Problem. Experimental results on synthetic instances show that the deep reinforcement learning approach, by achieving tighter objective function bounds, generally outperforms ordering methods commonly used in the literature when the distribution of instances is known. To the best knowledge of the authors, this is the first paper to apply machine learning to directly improve relaxation bounds obtained by general-purpose bounding mechanisms for combinatorial optimization problems.