Unsupervised learning of observation functions in state-space models by nonparametric moment methods
This addresses a fundamental challenge in state-space modeling for researchers in statistics and machine learning, though it is incremental with limitations such as non-identifiability due to symmetry and stationarity.
The paper tackles the problem of unsupervised learning of non-invertible observation functions in nonlinear state-space models, introducing a nonparametric generalized moment method that leads to convergent estimators for functions ranging from piecewise polynomials to smooth functions.
We investigate the unsupervised learning of non-invertible observation functions in nonlinear state-space models. Assuming abundant data of the observation process along with the distribution of the state process, we introduce a nonparametric generalized moment method to estimate the observation function via constrained regression. The major challenge comes from the non-invertibility of the observation function and the lack of data pairs between the state and observation. We address the fundamental issue of identifiability from quadratic loss functionals and show that the function space of identifiability is the closure of a RKHS that is intrinsic to the state process. Numerical results show that the first two moments and temporal correlations, along with upper and lower bounds, can identify functions ranging from piecewise polynomials to smooth functions, leading to convergent estimators. The limitations of this method, such as non-identifiability due to symmetry and stationarity, are also discussed.