Convex-constrained Sparse Additive Modeling and Its Extensions
This work addresses the challenge of enhancing sparse additive models with shape constraints for researchers in statistics and machine learning, representing an incremental improvement over existing methods.
The paper tackles the problem of high-dimensional nonparametric regression by integrating shape constraints like convexity into sparse additive models, resulting in a method called SDCAM that estimates continuous functions without smoothness assumptions and shows competitive or improved performance in experiments.
Sparse additive modeling is a class of effective methods for performing high-dimensional nonparametric regression. In this work we show how shape constraints such as convexity/concavity and their extensions, can be integrated into additive models. The proposed sparse difference of convex additive models (SDCAM) can estimate most continuous functions without any a priori smoothness assumption. Motivated by a characterization of difference of convex functions, our method incorporates a natural regularization functional to avoid overfitting and to reduce model complexity. Computationally, we develop an efficient backfitting algorithm with linear per-iteration complexity. Experiments on both synthetic and real data verify that our method is competitive against state-of-the-art sparse additive models, with improved performance in most scenarios.