Algorithmic learning of probability distributions from random data in the limit
This work addresses foundational issues in algorithmic learning theory for probability distributions, providing incremental theoretical insights into computability and oracle-based learning.
The paper tackles the problem of learning probability distributions from random data in the limit, showing that a computable partial learner exists for computable probability measures, and characterizes oracles that compute explanatory learners as high oracles, analogous to a known result by Adleman and Blum.
We study the problem of identifying a probability distribution for some given randomly sampled data in the limit, in the context of algorithmic learning theory as proposed recently by Vinanyi and Chater. We show that there exists a computable partial learner for the computable probability measures, while by Bienvenu, Monin and Shen it is known that there is no computable learner for the computable probability measures. Our main result is the characterization of the oracles that compute explanatory learners for the computable (continuous) probability measures as the high oracles. This provides an analogue of a well-known result of Adleman and Blum in the context of learning computable probability distributions. We also discuss related learning notions such as behaviorally correct learning and orther variations of explanatory learning, in the context of learning probability distributions from data.