Predictive Information Rate in Discrete-time Gaussian Processes
This work provides theoretical insights into information rates for Gaussian processes, which is incremental for researchers in information theory and signal processing.
The authors derived expressions for the predictive information rate (PIR) in autoregressive and moving average Gaussian processes, finding that PIR is maximized for smooth processes with multiple poles at zero frequency and is unbounded for MA processes.
We derive expressions for the predicitive information rate (PIR) for the class of autoregressive Gaussian processes AR(N), both in terms of the prediction coefficients and in terms of the power spectral density. The latter result suggests a duality between the PIR and the multi-information rate for processes with mutually inverse power spectra (i.e. with poles and zeros of the transfer function exchanged). We investigate the behaviour of the PIR in relation to the multi-information rate for some simple examples, which suggest, somewhat counter-intuitively, that the PIR is maximised for very `smooth' AR processes whose power spectra have multiple poles at zero frequency. We also obtain results for moving average Gaussian processes which are consistent with the duality conjectured earlier. One consequence of this is that the PIR is unbounded for MA(N) processes.