Approximation of optimization problems with constraints through kernel Sum-Of-Squares
This work addresses a foundational challenge in optimization and machine learning for researchers and practitioners dealing with constrained problems, but it appears incremental as it builds on existing kernel Sum-Of-Squares methods.
The authors tackled the problem of handling infinite inequality constraints in infinite-dimensional spaces by proposing a unified theorem for kernel Sum-Of-Squares approximations, proving convergence guarantees and enabling the use of scattering inequalities to mitigate dimensionality issues in sampling constraints.
Handling an infinite number of inequality constraints in infinite-dimensional spaces occurs in many fields, from global optimization to optimal transport. These problems have been tackled individually in several previous articles through kernel Sum-Of-Squares (kSoS) approximations. We propose here a unified theorem to prove convergence guarantees for these schemes. Pointwise inequalities are turned into equalities within a class of nonnegative kSoS functions. Assuming further that the functions appearing in the problem are smooth, focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints. Our approach is illustrated in learning vector fields with side information, here the invariance of a set.