Consistency of Extreme Learning Machines and Regression under Non-Stationarity and Dependence for ML-Enhanced Moving Objects
This work addresses the challenge of reliable machine learning for autonomous systems in dynamic, non-stationary settings, representing an incremental advance by extending existing consistency results to more complex spatial-temporal scenarios.
The paper tackles the problem of ensuring consistency and asymptotic normality of extreme learning machines and regression methods under non-stationary and dependent data conditions, specifically for moving objects in spatial environments, showing that least squares, ridge regression, and ℓ_s-penalized methods achieve these properties with bounds on prediction error.
Supervised learning by extreme learning machines resp. neural networks with random weights is studied under a non-stationary spatial-temporal sampling design which especially addresses settings where an autonomous object moving in a non-stationary spatial environment collects and analyzes data. The stochastic model especially allows for spatial heterogeneity and weak dependence. As efficient and computationally cheap learning methods (unconstrained) least squares, ridge regression and $\ell_s$-penalized least squares (including the LASSO) are studied. Consistency and asymptotic normality of the least squares and ridge regression estimates as well as corresponding consistency results for the $\ell_s$-penalty are shown under weak conditions. The results also cover bounds for the sample squared predicition error.