Self-Supervised Inference in State-Space Models
This addresses the problem of inference in nonlinear state-space models for researchers and practitioners, offering a self-supervised approach that is incremental by building on classical methods like the Kalman filter.
The paper tackles approximate inference in state-space models with nonlinear state transitions by using a self-supervised method based on Bayesian updates and neural networks, achieving excellent results on the chaotic Lorenz system and competitive results on audio denoising.
We perform approximate inference in state-space models with nonlinear state transitions. Without parameterizing a generative model, we apply Bayesian update formulas using a local linearity approximation parameterized by neural networks. This comes accompanied by a maximum likelihood objective that requires no supervision via uncorrupt observations or ground truth latent states. The optimization backpropagates through a recursion similar to the classical Kalman filter and smoother. Additionally, using an approximate conditional independence, we can perform smoothing without having to parameterize a separate model. In scientific applications, domain knowledge can give a linear approximation of the latent transition maps, which we can easily incorporate into our model. Usage of such domain knowledge is reflected in excellent results (despite our model's simplicity) on the chaotic Lorenz system compared to fully supervised and variational inference methods. Finally, we show competitive results on an audio denoising experiment.