Convexity Certificates from Hessians
This provides a more powerful tool for certifying convexity in machine learning and optimization, though it is incremental as it builds on existing Hessian computation methods.
The paper tackles the problem of certifying convexity for differentiable functions by implementing a Hessian-based approach, showing it is at least as powerful as the disciplined convex programming (DCP) approach and can certify a larger class of differentiable functions.
The Hessian of a differentiable convex function is positive semidefinite. Therefore, checking the Hessian of a given function is a natural approach to certify convexity. However, implementing this approach is not straightforward since it requires a representation of the Hessian that allows its analysis. Here, we implement this approach for a class of functions that is rich enough to support classical machine learning. For this class of functions, it was recently shown how to compute computational graphs of their Hessians. We show how to check these graphs for positive semidefiniteness. We compare our implementation of the Hessian approach with the well-established disciplined convex programming (DCP) approach and prove that the Hessian approach is at least as powerful as the DCP approach for differentiable functions. Furthermore, we show for a state-of-the-art implementation of the DCP approach that, for differentiable functions, the Hessian approach is actually more powerful. That is, it can certify the convexity of a larger class of differentiable functions.