On Learning High Dimensional Structured Single Index Models
This addresses the challenge of scalable semi-parametric modeling for high-dimensional data, offering an incremental improvement over existing methods by extending SIMs to structured high-dimensional settings.
The paper tackles the problem of efficiently learning Single Index Models (SIMs) in high dimensions with structural constraints, proposing algorithms that achieve superior predictive performance compared to generalized linear models and competitive results with single-layer neural networks at lower computational cost.
Single Index Models (SIMs) are simple yet flexible semi-parametric models for machine learning, where the response variable is modeled as a monotonic function of a linear combination of features. Estimation in this context requires learning both the feature weights and the nonlinear function that relates features to observations. While methods have been described to learn SIMs in the low dimensional regime, a method that can efficiently learn SIMs in high dimensions, and under general structural assumptions, has not been forthcoming. In this paper, we propose computationally efficient algorithms for SIM inference in high dimensions with structural constraints. Our general approach specializes to sparsity, group sparsity, and low-rank assumptions among others. Experiments show that the proposed method enjoys superior predictive performance when compared to generalized linear models, and achieves results comparable to or better than single layer feedforward neural networks with significantly less computational cost.