Observational nonidentifiability, generalized likelihood and free energy
This addresses a fundamental issue in statistical inference for mixture models, offering a novel solution to observational nonidentifiability, though it appears incremental as it builds on existing likelihood and free energy concepts.
The paper tackles the problem of parameter estimation in mixture models where the observed data model is nonidentifiable, leading to degenerate likelihood maxima and non-unique predictions. It introduces a generalized likelihood with an effective temperature, resolving nonidentifiability and yielding unique results better than random selection or averaging over degenerate maxima.
We study the parameter estimation problem in mixture models with observational nonidentifiability: the full model (also containing hidden variables) is identifiable, but the marginal (observed) model is not. Hence global maxima of the marginal likelihood are (infinitely) degenerate and predictions of the marginal likelihood are not unique. We show how to generalize the marginal likelihood by introducing an effective temperature, and making it similar to the free energy. This generalization resolves the observational nonidentifiability, since its maximization leads to unique results that are better than a random selection of one degenerate maximum of the marginal likelihood or the averaging over many such maxima. The generalized likelihood inherits many features from the usual likelihood, e.g. it holds the conditionality principle, and its local maximum can be searched for via suitably modified expectation-maximization method. The maximization of the generalized likelihood relates to entropy optimization.