SMML estimators for exponential families with continuous sufficient statistics
This work addresses theoretical properties of SMML estimators for statisticians and information theorists, but it is incremental as it builds on known results with new proofs and applications.
The paper tackles the problem of characterizing strict minimum message length (SMML) estimators for exponential families with continuous sufficient statistics, showing that their partitions consist of convex polytopes and providing a new proof using calculus of variations, which yields new inequalities and enables construction of an SMML estimator for a 2D normal variable.
The minimum message length principle is an information theoretic criterion that links data compression with statistical inference. This paper studies the strict minimum message length (SMML) estimator for $d$-dimensional exponential families with continuous sufficient statistics, for all $d \ge 1$. The partition of an SMML estimator is shown to consist of convex polytopes (i.e. convex polygons when $d=2$) which can be described explicitly in terms of the assertions and coding probabilities. While this result is known, we give a new proof based on the calculus of variations, and this approach gives some interesting new inequalities for SMML estimators. We also use this result to construct an SMML estimator for a $2$-dimensional normal random variable with known variance and a normal prior on its mean.