Asymptotically Optimal Sequential Testing with Markovian Data
This work addresses optimal sequential testing for researchers in statistics and machine learning dealing with Markovian data, offering improvements over prior asymptotic or sub-optimal bounds.
The paper tackles the problem of sequential hypothesis testing for data from ergodic Markov chains, establishing a tight non-asymptotic lower bound on expected stopping time and proposing an asymptotically optimal test as the error probability approaches zero.
We study one-sided and $α$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the alternative corresponds to a disjoint set $Q$. We establish a tight non-asymptotic instance-dependent lower bound on the expected stopping time of any valid sequential test under the alternative. Our novel analysis improves the existing lower bounds, which are either asymptotic or provably sub-optimal in this setting. Our lower bound incorporates both the stationary distribution and the transition structure induced by the unknown Markov chain. We further propose an optimal test whose expected stopping time matches this lower bound asymptotically as $α\to 0$. We illustrate the usefulness of our framework through applications to sequential detection of model misspecification in Markov Chain Monte Carlo and to testing structural properties, such as the linearity of transition dynamics, in Markov decision processes. Our findings yield a sharp and general characterization of optimal sequential testing procedures under Markovian dependence.