Comparative Analysis of Viterbi Training and Maximum Likelihood Estimation for HMMs
This work addresses parameter estimation in HMMs, offering insights into VT's properties, but it is incremental as it builds on existing methods with limited empirical validation.
The paper compares Viterbi Training (VT) and Maximum Likelihood (ML) estimation for Hidden Markov Models, showing that VT has finite degeneracy, converges faster, and yields sparser models in a solvable case, though it may underperform in general scenarios.
We present an asymptotic analysis of Viterbi Training (VT) and contrast it with a more conventional Maximum Likelihood (ML) approach to parameter estimation in Hidden Markov Models. While ML estimator works by (locally) maximizing the likelihood of the observed data, VT seeks to maximize the probability of the most likely hidden state sequence. We develop an analytical framework based on a generating function formalism and illustrate it on an exactly solvable model of HMM with one unambiguous symbol. For this particular model the ML objective function is continuously degenerate. VT objective, in contrast, is shown to have only finite degeneracy. Furthermore, VT converges faster and results in sparser (simpler) models, thus realizing an automatic Occam's razor for HMM learning. For more general scenario VT can be worse compared to ML but still capable of correctly recovering most of the parameters.