Misspecified Universal Learning
Provides a theoretical foundation for universal learning under misspecification, a known gap in the literature, but the results are theoretical and not yet validated empirically.
This paper extends universal learning to handle model misspecification under log-loss, deriving the minimax regret and optimal learner for cases where the true distribution lies outside the hypothesis class. The framework unifies online/batch and supervised/unsupervised settings.
This paper addresses the problem of universal learning under model misspecification with log-loss. In this setting, the learner operates with a hypothesis class of models denoted by $Θ$, while the true data-generating process belongs to a broader class $Φ\supset Θ$, and may lie outside the assumed hypothesis space. Classical approaches have characterized the minimax regret and identified optimal universal learners in both the well-specified stochastic and individual deterministic frameworks. The misspecified setting has received comparatively less attention, although several important results have emerged in recent years. Extending these foundations, we analyze the minimax regret in the misspecified setting and derive the corresponding optimal universal learner. We propose this formulation as a unified framework for universal learning, applicable to any form of uncertainty in the data-generating process, across both online and batch data arrival modes, as well as supervised and unsupervised learning tasks.