Overparameterized Multiple Linear Regression as Hyper-Curve Fitting
This work addresses a methodological issue in regression analysis for researchers dealing with overparameterized datasets, offering a novel perspective but appearing incremental as it builds on existing linear regression frameworks.
The paper tackles the problem of applying fixed-effect multiple linear regression to overparameterized datasets by showing it is equivalent to fitting data with a hyper-curve parameterized by a single scalar, which allows for exact predictions even when nonlinear dependencies violate model assumptions. It demonstrates this approach on synthetic and experimental data, highlighting its utility for regularization in noisy predictor scenarios.
The paper shows that the application of the fixed-effect multiple linear regression model to an overparameterized dataset is equivalent to fitting the data with a hyper-curve parameterized by a single scalar parameter. This equivalence allows for a predictor-focused approach, where each predictor is described by a function of the chosen parameter. It is proven that a linear model will produce exact predictions even in the presence of nonlinear dependencies that violate the model assumptions. Parameterization in terms of the dependent variable and the monomial basis in the predictor function space are applied here to both synthetic and experimental data. The hyper-curve approach is especially suited for the regularization of problems with noise in predictor variables and can be used to remove noisy and "improper" predictors from the model.