Stochastic Collapsed Variational Inference for Hidden Markov Models
This work addresses scalability and accuracy issues in sequential data analysis for researchers and practitioners using HMMs, though it is incremental as it extends existing stochastic collapsed variational inference from topic modeling to HMMs.
The authors tackled the problem of scaling variational inference for hidden Markov models (HMMs) by proposing a stochastic collapsed algorithm that breaks long Markov chains into short subchains, resulting in a method that is scalable to large datasets, memory efficient, and significantly more accurate than existing uncollapsed algorithms, as shown in experiments on two discrete datasets.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm for hidden Markov models, in a sequential data setting. Given a collapsed hidden Markov Model, we break its long Markov chain into a set of short subchains. We propose a novel sum-product algorithm to update the posteriors of the subchains, taking into account their boundary transitions due to the sequential dependencies. Our experiments on two discrete datasets show that our collapsed algorithm is scalable to very large datasets, memory efficient and significantly more accurate than the existing uncollapsed algorithm.