Factorized Asymptotic Bayesian Inference for Factorial Hidden Markov Models
This addresses a challenging simultaneous model selection issue in FHMMs for sequential data modeling, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles the model selection problem in factorial hidden Markov models (FHMMs) by extending Factorized Asymptotic Bayesian (FAB) inference, resulting in significantly outperforming state-of-the-art methods like nonparametric Bayesian iFHMM and Variational FHMM in model selection accuracy, with competitive held-out perplexity.
Factorial hidden Markov models (FHMMs) are powerful tools of modeling sequential data. Learning FHMMs yields a challenging simultaneous model selection issue, i.e., selecting the number of multiple Markov chains and the dimensionality of each chain. Our main contribution is to address this model selection issue by extending Factorized Asymptotic Bayesian (FAB) inference to FHMMs. First, we offer a better approximation of marginal log-likelihood than the previous FAB inference. Our key idea is to integrate out transition probabilities, yet still apply the Laplace approximation to emission probabilities. Second, we prove that if there are two very similar hidden states in an FHMM, i.e. one is redundant, then FAB will almost surely shrink and eliminate one of them, making the model parsimonious. Experimental results show that FAB for FHMMs significantly outperforms state-of-the-art nonparametric Bayesian iFHMM and Variational FHMM in model selection accuracy, with competitive held-out perplexity.