Regularized Nonlinear Regression for Simultaneously Selecting and Estimating Key Model Parameters
This work addresses model parameter estimation in system identification, particularly for biomechanical models with limited data, offering an incremental improvement through regularization and optimization enhancements.
The paper tackles the problem of poor identifiability in system identification with limited observations by proposing a method to simultaneously select and estimate key model parameters using L1-regularized nonlinear regression. In simulations, it reduces parameter variances by 96.1%, and in experiments, it maintains variance accounted for above 82.5% while reducing variances by 71.1% and speeding up optimization by 54 times.
In system identification, estimating parameters of a model using limited observations results in poor identifiability. To cope with this issue, we propose a new method to simultaneously select and estimate sensitive parameters as key model parameters and fix the remaining parameters to a set of typical values. Our method is formulated as a nonlinear least squares estimator with L1-regularization on the deviation of parameters from a set of typical values. First, we provide consistency and oracle properties of the proposed estimator as a theoretical foundation. Second, we provide a novel approach based on Levenberg-Marquardt optimization to numerically find the solution to the formulated problem. Third, to show the effectiveness, we present an application identifying a biomechanical parametric model of a head position tracking task for 10 human subjects from limited data. In a simulation study, the variances of estimated parameters are decreased by 96.1% as compared to that of the estimated parameters without L1-regularization. In an experimental study, our method improves the model interpretation by reducing the number of parameters to be estimated while maintaining variance accounted for (VAF) at above 82.5%. Moreover, the variances of estimated parameters are reduced by 71.1% as compared to that of the estimated parameters without L1-regularization. Our method is 54 times faster than the standard simplex-based optimization to solve the regularized nonlinear regression.