Algebraic Reduction of Hidden Markov Models
This provides a method for simplifying HMMs in applications like signal processing or machine learning, but it appears incremental as it extends existing realization theory tools.
The paper tackles the problem of reducing Hidden Markov Models (HMMs) to smaller dimensions while exactly reproducing marginals, using a system-theoretic approach and algebraic representations, resulting in algorithms that produce coarse-grained equivalent HMMs preserving single-time or full multi-time distributions.
The problem of reducing a Hidden Markov Model (HMM) to one of smaller dimension that exactly reproduces the same marginals is tackled by using a system-theoretic approach. Realization theory tools are extended to HMMs by leveraging suitable algebraic representations of probability spaces. We propose two algorithms that return coarse-grained equivalent HMMs obtained by stochastic projection operators: the first returns models that exactly reproduce the single-time distribution of a given output process, while in the second the full (multi-time) distribution is preserved. The reduction method exploits not only the structure of the observed output, but also its initial condition, whenever the latter is known or belongs to a given subclass. Optimal algorithms are derived for a class of HMM, namely observable ones.