The Mixing method: low-rank coordinate descent for semidefinite programming with diagonal constraints
This work expands the scope and scale of problems solvable using semidefinite programming methods, with applications in maximum cut, maximum satisfiability, and learning word embeddings.
The paper tackles structured semidefinite programming with diagonal constraints by proposing the Mixing method, a low-rank coordinate descent approach that achieves an order of magnitude or better improvement in optimization performance over state-of-the-art methods and is proven to converge to the global optimum almost surely under random initialization.
In this paper, we propose a low-rank coordinate descent approach to structured semidefinite programming with diagonal constraints. The approach, which we call the Mixing method, is extremely simple to implement, has no free parameters, and typically attains an order of magnitude or better improvement in optimization performance over the current state of the art. We show that the method is strictly decreasing, converges to a critical point, and further that for sufficient rank all non-optimal critical points are unstable. Moreover, we prove that with a step size, the Mixing method converges to the global optimum of the semidefinite program almost surely in a locally linear rate under random initialization. This is the first low-rank semidefinite programming method that has been shown to achieve a global optimum on the spherical manifold without assumption. We apply our algorithm to two related domains: solving the maximum cut semidefinite relaxation, and solving a maximum satisfiability relaxation (we also briefly consider additional applications such as learning word embeddings). In all settings, we demonstrate substantial improvement over the existing state of the art along various dimensions, and in total, this work expands the scope and scale of problems that can be solved using semidefinite programming methods.