How consistent is my model with the data? Information-Theoretic Model Check
This work addresses the fundamental issue of model class consistency in statistical learning and system identification, though it appears incremental as it builds on existing posterior predictive tests.
The authors tackled the problem of evaluating whether a chosen model class can reproduce data similar to observed records, developing an information-theoretic model check applicable to sequential and nonlinear dynamical models. They applied this method to synthetic and real data, comparing it with a classical whiteness test.
The choice of model class is fundamental in statistical learning and system identification, no matter whether the class is derived from physical principles or is a generic black-box. We develop a method to evaluate the specified model class by assessing its capability of reproducing data that is similar to the observed data record. This model check is based on the information-theoretic properties of models viewed as data generators and is applicable to e.g. sequential data and nonlinear dynamical models. The method can be understood as a specific two-sided posterior predictive test. We apply the information-theoretic model check to both synthetic and real data and compare it with a classical whiteness test.