Lifting high-dimensional nonlinear models with Gaussian regressors
This addresses a limitation in high-dimensional nonlinear regression for signal processing and statistics, offering a solution for cases like even link functions where prior methods fail, though it is incremental in extending convex optimization techniques.
The paper tackles the problem of recovering a structured signal from high-dimensional data with nonlinear link functions, where existing methods fail when the proportionality constant is zero, such as for even functions. It proposes a convex recovery method that lifts the problem to a higher-dimensional space, achieving error bounds that incorporate nonlinearity and geometry through simple parameters.
We study the problem of recovering a structured signal $\mathbf{x}_0$ from high-dimensional data $\mathbf{y}_i=f(\mathbf{a}_i^T\mathbf{x}_0)$ for some nonlinear (and potentially unknown) link function $f$, when the regressors $\mathbf{a}_i$ are iid Gaussian. Brillinger (1982) showed that ordinary least-squares estimates $\mathbf{x}_0$ up to a constant of proportionality $μ_\ell$, which depends on $f$. Recently, Plan & Vershynin (2015) extended this result to the high-dimensional setting deriving sharp error bounds for the generalized Lasso. Unfortunately, both least-squares and the Lasso fail to recover $\mathbf{x}_0$ when $μ_\ell=0$. For example, this includes all even link functions. We resolve this issue by proposing and analyzing an alternative convex recovery method. In a nutshell, our method treats such link functions as if they were linear in a lifted space of higher-dimension. Interestingly, our error analysis captures the effect of both the nonlinearity and the problem's geometry in a few simple summary parameters.