Solomonoff induction
This work addresses foundational issues in theoretical AI and machine learning, showing that Solomonoff induction is incremental and flawed in its computability assumptions.
The paper examines Solomonoff induction as a universal prediction framework, focusing on its computability requirements, and demonstrates its failure through a generalized diagonalization argument, while critiquing its claimed benefits for Occam's razor and machine learning.
This chapter discusses the Solomonoff approach to universal prediction. The crucial ingredient in the approach is the notion of computability, and I present the main idea as an attempt to meet two plausible computability desiderata for a universal predictor. This attempt is unsuccessful, which is shown by a generalization of a diagonalization argument due to Putnam. I then critically discuss purported gains of the approach, in particular it providing a foundation for the methodological principle of Occam's razor, and it serving as a theoretical ideal for the development of machine learning methods.