Comment on Is Complexity an Illusion?
This is an incremental critique of a theoretical framework in machine learning, potentially affecting researchers in formal learning theory.
The paper demonstrates that correct policies do not exist for supervised multi-class classification through mathematical proof and exhaustive search, challenging the formalism proposed by Bennett (2024).
The paper "Is Complexity an Illusion?" (Bennett, 2024) provides a formalism for complexity, learning, inference, and generalization, and introduces a formal definition for a "policy". This reply shows that correct policies do not exist for a simple task of supervised multi-class classification, via mathematical proof and exhaustive search. Implications of this result are discussed, as well as possible responses and amendments to the theory.