Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective
This work provides theoretical foundations for error analysis in time series, which is incremental as it extends existing linear prediction theory to a broader information-theoretic framework.
The paper derived generic variance bounds for estimation and prediction errors in time series analysis using an information-theoretic approach, showing these bounds depend on conditional entropy and linking asymptotic achievability to white Gaussian innovation. For Gaussian processes, the bounds reduce to known formulas like the Kolmogorov-Szegö formula.
In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or past observations. Additionally, we discover that in order to achieve the prediction error bounds asymptotically, the necessary and sufficient condition is that the "innovation" is asymptotically white Gaussian. When restricted to Gaussian processes and 1-step prediction, our bounds are shown to reduce to the Kolmogorov-Szegö formula and Wiener-Masani formula known from linear prediction theory.