Fast local linear regression with anchor regularization
This work addresses the need for efficient and interpretable local regression methods, particularly for applications in domains like finance and biomedical, though it appears incremental as it builds on existing network Lasso approaches.
The paper tackles the problem of training local models for regression by proposing FALL, a fast anchor regularized local linear method that achieves comparable accuracy to state-of-the-art network Lasso while reducing training time by two orders of magnitude.
Regression is an important task in machine learning and data mining. It has several applications in various domains, including finance, biomedical, and computer vision. Recently, network Lasso, which estimates local models by making clusters using the network information, was proposed and its superior performance was demonstrated. In this study, we propose a simple yet effective local model training algorithm called the fast anchor regularized local linear method (FALL). More specifically, we train a local model for each sample by regularizing it with precomputed anchor models. The key advantage of the proposed algorithm is that we can obtain a closed-form solution with only matrix multiplication; additionally, the proposed algorithm is easily interpretable, fast to compute and parallelizable. Through experiments on synthetic and real-world datasets, we demonstrate that FALL compares favorably in terms of accuracy with the state-of-the-art network Lasso algorithm with significantly smaller training time (two orders of magnitude).