Cubic-Regularized Newton for Spectral Constrained Matrix Optimization and its Application to Fairness
This work addresses matrix optimization problems with spectral constraints, offering incremental improvements in methods for applications like fairness in data analysis.
The paper tackles spectral constrained matrix optimization by reformulating it as an unconstrained problem and solving it with a cubic-regularized Newton method, achieving improved convergence and demonstrating advantages in fair covariance estimation on synthetic and real datasets.
Matrix functions are utilized to rewrite smooth spectral constrained matrix optimization problems as smooth unconstrained problems over the set of symmetric matrices which are then solved via the cubic-regularized Newton method. A second-order chain rule identity for matrix functions is proven to compute the higher-order derivatives to implement cubic-regularized Newton, and a new convergence analysis is provided for cubic-regularized Newton for matrix vector spaces. We demonstrate the applicability of our approach by conducting numerical experiments on both synthetic and real datasets. In our experiments, we formulate a new model for estimating fair and robust covariance matrices in the spirit of the Tyler's M-estimator (TME) model and demonstrate its advantage.