Adaptive Cubic Regularized Second-Order Latent Factor Analysis Model
This work addresses optimization instabilities in latent factor analysis for real-world applications like recommendation systems, representing an incremental improvement over prior methods.
The paper tackles the challenge of optimizing second-order latent factor models for high-dimensional incomplete data by proposing the ACRSLF model, which incorporates adaptive cubic regularization and multi-Hessian-vector products, resulting in faster convergence and higher representation accuracy compared to existing methods.
High-dimensional and incomplete (HDI) data, characterized by massive node interactions, have become ubiquitous across various real-world applications. Second-order latent factor models have shown promising performance in modeling this type of data. Nevertheless, due to the bilinear and non-convex nature of the SLF model's objective function, incorporating a damping term into the Hessian approximation and carefully tuning associated parameters become essential. To overcome these challenges, we propose a new approach in this study, named the adaptive cubic regularized second-order latent factor analysis (ACRSLF) model. The proposed ACRSLF adopts the two-fold ideas: 1) self-tuning cubic regularization that dynamically mitigates non-convex optimization instabilities; 2) multi-Hessian-vector product evaluation during conjugate gradient iterations for precise second-order information assimilation. Comprehensive experiments on two industrial HDI datasets demonstrate that the ACRSLF converges faster and achieves higher representation accuracy than the advancing optimizer-based LFA models.