Meta-reinforcement learning with minimum attention
This work addresses the problem of enhancing RL stability and efficiency for applications like biological control emulation, though it appears incremental as it builds on existing meta-learning and control principles.
The paper tackled the problem of improving reinforcement learning (RL) by incorporating minimum attention, a principle from control theory, into meta-learning for high-dimensional nonlinear dynamics. The result showed outperforming competence compared to state-of-the-art RL algorithms, including fast adaptation in few shots, variance reduction, and improved energy efficiency.
Minimum attention applies the least action principle in the changes of control concerning state and time, first proposed by Brockett. The involved regularization is highly relevant in emulating biological control, such as motor learning. We apply minimum attention in reinforcement learning (RL) as part of the rewards and investigate its connection to meta-learning and stabilization. Specifically, model-based meta-learning with minimum attention is explored in high-dimensional nonlinear dynamics. Ensemble-based model learning and gradient-based meta-policy learning are alternately performed. Empirically, we show that the minimum attention does show outperforming competence in comparison to the state-of-the-art algorithms in model-free and model-based RL, i.e., fast adaptation in few shots and variance reduction from the perturbations of the model and environment. Furthermore, the minimum attention demonstrates the improvement in energy efficiency.