Convex Factorization Machine for Regression
This work addresses the issue of poor local optima in FMs for researchers and practitioners in machine learning, offering a convex alternative that is incremental but with specific gains.
The authors tackled the non-convex optimization problem in Factorization Machines (FMs) by proposing a convex variant called CFM, which achieves globally optimal solutions and shows competitive results on synthetic and MovieLens datasets while outperforming a state-of-the-art tensor factorization method in a toxicogenomics prediction task.
We propose the convex factorization machine (CFM), which is a convex variant of the widely used Factorization Machines (FMs). Specifically, we employ a linear+quadratic model and regularize the linear term with the $\ell_2$-regularizer and the quadratic term with the trace norm regularizer. Then, we formulate the CFM optimization as a semidefinite programming problem and propose an efficient optimization procedure with Hazan's algorithm. A key advantage of CFM over existing FMs is that it can find a globally optimal solution, while FMs may get a poor locally optimal solution since the objective function of FMs is non-convex. In addition, the proposed algorithm is simple yet effective and can be implemented easily. Finally, CFM is a general factorization method and can also be used for other factorization problems including including multi-view matrix factorization and tensor completion problems. Through synthetic and movielens datasets, we first show that the proposed CFM achieves results competitive to FMs. Furthermore, in a toxicogenomics prediction task, we show that CFM outperforms a state-of-the-art tensor factorization method.