A Note On Nonlinear Regression Under L2 Loss
This addresses a fundamental challenge in machine learning for researchers and practitioners, but appears incremental as it builds on known issues without broad empirical validation.
The paper tackles the non-convex optimization problem in traditional nonlinear regression under L2 loss by demonstrating the existence of a convex model, potentially enabling easier training for complex systems.
We investigate the nonlinear regression problem under L2 loss (square loss) functions. Traditional nonlinear regression models often result in non-convex optimization problems with respect to the parameter set. We show that a convex nonlinear regression model exists for the traditional least squares problem, which can be a promising towards designing more complex systems with easier to train models.