Ising Models with Hidden Markov Structure: Applications to Probabilistic Inference in Machine Learning
This provides a structured approach for inference on hierarchical data in machine learning, though it is incremental as it builds on existing hidden Markov models and Ising frameworks.
The paper tackles the problem of probabilistic inference in machine learning by extending hidden Markov models to a bilayer Markov random field on Cayley trees, demonstrating that up to three distinct translation-invariant Gibbs measures exist under certain conditions, which can be applied to tasks like denoising and anomaly detection.
In this paper, we investigate tree-indexed Markov chains (Gibbs measures) defined by a Hamiltonian that couples two Ising layers: hidden spins \(s(x) \in \{\pm 1\}\) and observed spins \(σ(x) \in \{\pm 1\}\) on a Cayley tree. The Hamiltonian incorporates Ising interactions within each layer and site-wise emission couplings between layers, extending hidden Markov models to a bilayer Markov random field. Specifically, we explore translation-invariant Gibbs measures (TIGM) of this Hamiltonian on Cayley trees. Under certain explicit conditions on the model's parameters, we demonstrate that there can be up to three distinct TIGMs. Each of these measures represents an equilibrium state of the spin system. These measures provide a structured approach to inference on hierarchical data in machine learning. They have practical applications in tasks such as denoising, weakly supervised learning, and anomaly detection. The Cayley tree structure is particularly advantageous for exact inference due to its tractability.