Cognitive Training for Language Models: Towards General Capabilities via Cross-Entropy Games
This work addresses the open problem of developing general AI capabilities through automated curriculum learning, potentially impacting the entire field of artificial intelligence if successful.
The paper tackles the problem of automatically building general capabilities in language models by proposing a curriculum learning framework called cognitive training, which uses cross-entropy games to iteratively discover relevant skills, and shows that under certain assumptions, this approach leads to a unique meta-objective for skill acquisition.
Defining a constructive process to build general capabilities for language models in an automatic manner is considered an open problem in artificial intelligence. Towards this, we consider the problem of building a curriculum of tasks that grows a model via relevant skill discovery. We provide a concrete framework for this task, using a family of tasks called cross-entropy games, which we postulate is universal in a suitable sense. We show that if it is possible to grow the curriculum for relevant skill discovery by iterating a greedy optimization algorithm, then, under natural assumptions, there is essentially only one meta-objective possible (up to a few hyperparameters). We call the resulting process cognitive training. We postulate that, given sufficiently capable language models as players and meta-samplers and sufficient training time, cognitive training provides a principled way to relevant skill discovery; and hence to the extent general capabilities are achievable via greedy curriculum learning, cognitive training would be a solution.