Finite-sample risk bounds for maximum likelihood estimation with arbitrary penalties
This work addresses a theoretical limitation in statistical estimation for researchers, providing incremental improvements in risk bound analysis.
The paper tackles the problem of finite-sample risk bounds for maximum likelihood estimation with arbitrary penalties, showing that a more general inequality holds and deriving exact risk bounds of order 1/n for iid parametric models, improving on previous order (log n)/n bounds.
The MDL two-part coding $ \textit{index of resolvability} $ provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models. However, the bound does not apply to unpenalized maximum likelihood estimation or procedures with exceedingly small penalties. In this paper, we point out a more general inequality that holds for arbitrary penalties. In addition, this approach makes it possible to derive exact risk bounds of order $1/n$ for iid parametric models, which improves on the order $(\log n)/n$ resolvability bounds. We conclude by discussing implications for adaptive estimation.