Meta-learning of Sequential Strategies
This provides a conceptual foundation for scalable agents in broad domains, but it is incremental as it reviews and recasts existing approaches.
The paper reviews memory-based meta-learning as a tool for building sample-efficient strategies that adapt to tasks within a target class, showing that meta-learned strategies are near-optimal by amortizing Bayes-filtered data through memory dynamics.
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains. To do so, we present basic algorithmic templates for building near-optimal predictors and reinforcement learners which behave as if they had a probabilistic model that allowed them to efficiently exploit task structure. Furthermore, we recast memory-based meta-learning within a Bayesian framework, showing that the meta-learned strategies are near-optimal because they amortize Bayes-filtered data, where the adaptation is implemented in the memory dynamics as a state-machine of sufficient statistics. Essentially, memory-based meta-learning translates the hard problem of probabilistic sequential inference into a regression problem.