MML is not consistent for Neyman-Scott
This refutes a foundational claim in statistical inference, impacting researchers in information theory and statistics, though it is incremental as it focuses on a specific counterexample.
The paper tackles the problem of whether Strict Minimum Message Length (SMML) provides consistent estimations, showing that for the Neyman-Scott problem, neither SMML nor its approximations are consistent, countering a widely cited claim.
Strict Minimum Message Length (SMML) is an information-theoretic statistical inference method widely cited (but only with informal arguments) as providing estimations that are consistent for general estimation problems. It is, however, almost invariably intractable to compute, for which reason only approximations of it (known as MML algorithms) are ever used in practice. Using novel techniques that allow for the first time direct, non-approximated analysis of SMML solutions, we investigate the Neyman-Scott estimation problem, an oft-cited showcase for the consistency of MML, and show that even with a natural choice of prior neither SMML nor its popular approximations are consistent for it, thereby providing a counterexample to the general claim. This is the first known explicit construction of an SMML solution for a natural, high-dimensional problem.