Bayesian Properties of Normalized Maximum Likelihood and its Fast Computation
This work addresses a theoretical problem in statistical modeling for researchers in data compression and MDL methods, but it is incremental as it builds on existing NML concepts.
The paper tackled the relationship between normalized maximum likelihood (NML) and Bayesian modeling by showing that NML has a Bayes-like representation as a mixture with possibly negative weights, even in finite samples, which speeds up calculations for coding and prediction applications.
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-like representation as a mixture of the component models, even in finite samples, though the weights of linear combination may be both positive and negative. This representation addresses in part the relationship between MDL and Bayes modeling. This representation has the advantage of speeding the calculation of marginals and conditionals required for coding and prediction applications.