Asymptotic Behavior of Bayesian Generalization Error in Multinomial Mixtures
This work addresses a foundational mathematical problem in statistical learning theory for researchers in information engineering, but it is incremental as it builds on existing algebraic geometric methods for singular models.
The paper tackled the mathematical properties of multinomial mixtures, which are singular learning models with non-identifiability and non-positive definite Fisher information matrices, by clarifying their real log canonical thresholds and multiplicities, and elucidated the asymptotic behaviors of generalization error and free energy.
Multinomial mixtures are widely used in the information engineering field, however, their mathematical properties are not yet clarified because they are singular learning models. In fact, the models are non-identifiable and their Fisher information matrices are not positive definite. In recent years, the mathematical foundation of singular statistical models are clarified by using algebraic geometric methods. In this paper, we clarify the real log canonical thresholds and multiplicities of the multinomial mixtures and elucidate their asymptotic behaviors of generalization error and free energy.