Computable learning of natural hypothesis classes
This addresses a foundational issue in computational learning theory by clarifying the role of computability in learning, indicating that previous counterexamples are unnatural.
The paper tackles the problem of distinguishing between PAC learnable and computably PAC learnable hypothesis classes, showing that under mild assumptions, any natural hypothesis class that is learnable must also be computably learnable.
This paper is about the recent notion of computably probably approximately correct learning, which lies between the statistical learning theory where there is no computational requirement on the learner and efficient PAC where the learner must be polynomially bounded. Examples have recently been given of hypothesis classes which are PAC learnable but not computably PAC learnable, but these hypothesis classes are unnatural or non-canonical in the sense that they depend on a numbering of proofs, formulas, or programs. We use the on-a-cone machinery from computability theory to prove that, under mild assumptions such as that the hypothesis class can be computably listable, any natural hypothesis class which is learnable must be computably learnable. Thus the counterexamples given previously are necessarily unnatural.