Towards an Approximation Theory of Observable Operator Models
This work addresses a foundational problem in machine learning for researchers, though it is incremental as it builds on prior unpublished work.
The paper tackles the theoretical challenge of modeling infinite-dimensional stochastic processes with observable operator models (OOMs) by establishing an inner product structure and proving continuity of operators, but identifies a fundamental obstacle in forming a Hilbert space.
Observable operator models (OOMs) offer a powerful framework for modelling stochastic processes, surpassing the traditional hidden Markov models (HMMs) in generality and efficiency. However, using OOMs to model infinite-dimensional processes poses significant theoretical challenges. This article explores a rigorous approach to developing an approximation theory for OOMs of infinite-dimensional processes. Building upon foundational work outlined in an unpublished tutorial [Jae98], an inner product structure on the space of future distributions is rigorously established and the continuity of observable operators with respect to the associated 2-norm is proven. The original theorem proven in this thesis describes a fundamental obstacle in making an infinite-dimensional space of future distributions into a Hilbert space. The presented findings lay the groundwork for future research in approximating observable operators of infinite-dimensional processes, while a remedy to the encountered obstacle is suggested.