Estimating stationary mass, frequency by frequency
This addresses the challenge of stationary mass estimation in non-i.i.d. settings for researchers in statistics and machine learning, though it is incremental as it extends existing methods to mixing processes.
The paper tackles the problem of estimating the stationary distribution mass for elements with certain frequencies in a trajectory from an exponentially α-mixing stochastic process, showing universal consistency in total variation distance and recovering i.i.d. results as special cases, with simulation studies corroborating theoretical findings.
Suppose we observe a trajectory of length $n$ from an exponentially $α$-mixing stochastic process over a finite but potentially large state space. We consider the problem of estimating the probability mass placed by the stationary distribution of any such process on elements that occur with a certain frequency in the observed sequence. We estimate this vector of probabilities in total variation distance, showing universal consistency in $n$ and recovering known results for i.i.d. sequences as special cases. Our proposed methodology -- implementable in linear time -- carefully combines the plug-in (or empirical) estimator with a recently-proposed modification of the Good--Turing estimator called WingIt, which was originally developed for Markovian sequences. En route to controlling the error of our estimator, we develop new performance bounds on WingIt and the plug-in estimator for exponentially $α$-mixing stochastic processes. Importantly, the extensively used method of Poissonization can no longer be applied in our non i.i.d. setting, and so we develop complementary tools -- including concentration inequalities for a natural self-normalized statistic of mixing sequences -- that may prove independently useful in the design and analysis of estimators for related problems. Simulation studies corroborate our theoretical findings.