Experience-driven discovery of planning strategies
This addresses a gap in understanding cognitive strategy formation for researchers in psychology and AI, but it is incremental as it builds on existing work on strategy learning.
The paper tackles the problem of how humans acquire new planning strategies, proposing metacognitive reinforcement learning as a mechanism and demonstrating that it explains human strategy discovery better than alternatives, though models fit to human data show a slower discovery rate.
One explanation for how people can plan efficiently despite limited cognitive resources is that we possess a set of adaptive planning strategies and know when and how to use them. But how are these strategies acquired? While previous research has studied how individuals learn to choose among existing strategies, little is known about the process of forming new planning strategies. In this work, we propose that new planning strategies are discovered through metacognitive reinforcement learning. To test this, we designed a novel experiment to investigate the discovery of new planning strategies. We then present metacognitive reinforcement learning models and demonstrate their capability for strategy discovery as well as show that they provide a better explanation of human strategy discovery than alternative learning mechanisms. However, when fitted to human data, these models exhibit a slower discovery rate than humans, leaving room for improvement.