Risk-averse estimation, an axiomatic approach to inference, and Wallace-Freeman without MML
This work offers a theoretical foundation for Bayesian inference, addressing researchers in statistics and machine learning, but it is incremental as it builds on existing estimators like MAP and Wallace-Freeman.
The paper tackles the problem of Bayesian point estimation by defining risk-averse estimators and formulating axioms for inference, showing that these axioms uniquely characterize Maximum A Posteriori (MAP) estimation for discrete problems and the Wallace-Freeman estimator for continuous problems, providing a novel justification without approximations or coding.
We define a new class of Bayesian point estimators, which we refer to as risk averse. Using this definition, we formulate axioms that provide natural requirements for inference, e.g. in a scientific setting, and show that for well-behaved estimation problems the axioms uniquely characterise an estimator. Namely, for estimation problems in which some parameter values have a positive posterior probability (such as, e.g., problems with a discrete hypothesis space), the axioms characterise Maximum A Posteriori (MAP) estimation, whereas elsewhere (such as in continuous estimation) they characterise the Wallace-Freeman estimator. Our results provide a novel justification for the Wallace-Freeman estimator, which previously was derived only as an approximation to the information-theoretic Strict Minimum Message Length estimator. By contrast, our derivation requires neither approximations nor coding.