MM Algorithms for Distance Covariance based Sufficient Dimension Reduction and Sufficient Variable Selection
This work addresses a computational bottleneck for researchers using distance covariance methods in dimension reduction, offering an incremental improvement in algorithm efficiency.
The authors tackled the challenge of optimizing a nonsmooth and nonconvex objective function in distance covariance-based sufficient dimension reduction by formulating it as a difference of convex functions program and developing a majorization-minimization algorithm with Riemannian Newton steps, resulting in drastically improved computation efficiency and robustness in simulations.
Sufficient dimension reduction (SDR) using distance covariance (DCOV) was recently proposed as an approach to dimension-reduction problems. Compared with other SDR methods, it is model-free without estimating link function and does not require any particular distributions on predictors (see Sheng and Yin, 2013, 2016). However, the DCOV-based SDR method involves optimizing a nonsmooth and nonconvex objective function over the Stiefel manifold. To tackle the numerical challenge, we novelly formulate the original objective function equivalently into a DC (Difference of Convex functions) program and construct an iterative algorithm based on the majorization-minimization (MM) principle. At each step of the MM algorithm, we inexactly solve the quadratic subproblem on the Stiefel manifold by taking one iteration of Riemannian Newton's method. The algorithm can also be readily extended to sufficient variable selection (SVS) using distance covariance. We establish the convergence property of the proposed algorithm under some regularity conditions. Simulation studies show our algorithm drastically improves the computation efficiency and is robust across various settings compared with the existing method. Supplemental materials for this article are available.