On the $α$-lazy version of Markov chains in estimation and testing problems
This work addresses a methodological gap for researchers in statistics and machine learning dealing with Markov chain inference, though it appears incremental as it extends existing results to broader chain types.
The paper tackles the problem of performing estimation and identity testing for irreducible Markov chains using a single long trajectory, by simulating an α-lazy version to ensure ergodicity, enabling fully empirical inference and generalizing recent results to include periodic chains.
Given access to a single long trajectory generated by an unknown irreducible Markov chain $M$, we simulate an $α$-lazy version of $M$ which is ergodic. This enables us to generalize recent results on estimation and identity testing that were stated for ergodic Markov chains in a way that allows fully empirical inference. In particular, our approach shows that the pseudo spectral gap introduced by Paulin [2015] and defined for ergodic Markov chains may be given a meaning already in the case of irreducible but possibly periodic Markov chains.