Proximal basin hopping: global optimization with guarantees
Provides a theoretically-guaranteed global optimization method for practitioners needing reliable solutions in high-dimensional problems.
Proximal Basin Hopping (PBH) is a new global optimization algorithm that provably converges to the global minimizer with high probability using finite samples. It outperforms theoretically-backed algorithms on synthetic hard functions and real problems like fitting scaling laws for deep learning, with the performance gap increasing in higher dimensions.
Global optimization is a challenging problem, with plenty of algorithms displaying empirical success, but scarce theoretical backing. In this work, we propose a new theoretical framework called Proximal Basin Hopping (PBH), carefully tailored to combine proximal optimization and local minimization. We use it to construct a practical algorithm that converges to the global minimizer with high probability, when using a finite amount of samples. Proximal Basin Hopping outperforms well known algorithms with theoretical backing on standard synthetic hard functions, and real problems such as fitting scaling laws for deep learning. Furthermore, the higher the dimension, the better the performance gap.