Relaxation via Separable Estimators: Arithmetic and Implementation
This work provides a new tool for global optimization practitioners who need tighter convex relaxations, though the increased computational cost limits its immediate applicability.
The paper introduces superposition relaxation, a new arithmetic for generating separable underestimating and overestimating functions for multivariate factorable functions. It demonstrates that these relaxations can be consistently tighter than McCormick relaxations, including for neural networks, but at a higher computational cost.
This article presents an arithmetic, called superposition relaxation, for bracketing the graph of a multivariate factorable function on a compact domain between a pair of underestimating and overestimating functions that are both separable. Propagation rules are established for affine and nonlinear composition operations, with a focus on exploiting global monotonicity and convexity properties in the composition. The local convergence properties of this arithmetic are also analyzed in both the pointwise and Hausdorff sense, including conditions under which quadratic pointwise convergence propagates through composition. Parameterizations of the univariate summands in a superposition relaxation either as piecewise-constant or continuous piecewise-linear functions are discussed for a practical implementation. It is shown through numerical case studies that superposition relaxations can be consistently tighter than McCormick relaxations, including for the relaxation of artificial neural networks. But superposition relaxations also incur a higher computational cost than McCormick relaxations. Further investigations are thus warranted as applications in global optimization seek to balance a relaxation's tightness with its computational cost.