Meta-Cognitive Reinforcement Learning with Self-Doubt and Recovery
This addresses the issue of robust learning in noisy environments for reinforcement learning practitioners, though it is incremental in building on existing robustness methods.
The paper tackles the problem of reinforcement learning agents lacking the ability to reason about their own learning reliability, which can lead to overreaction or catastrophic failure under noise. The result is a meta-cognitive framework that achieves higher average returns and reduces late-stage training failures in benchmarks with reward corruption.
Robust reinforcement learning methods typically focus on suppressing unreliable experiences or corrupted rewards, but they lack the ability to reason about the reliability of their own learning process. As a result, such methods often either overreact to noise by becoming overly conservative or fail catastrophically when uncertainty accumulates. In this work, we propose a meta-cognitive reinforcement learning framework that enables an agent to assess, regulate, and recover its learning behavior based on internally estimated reliability signals. The proposed method introduces a meta-trust variable driven by Value Prediction Error Stability (VPES), which modulates learning dynamics via fail-safe regulation and gradual trust recovery. Experiments on continuous-control benchmarks with reward corruption demonstrate that recovery-enabled meta-cognitive control achieves higher average returns and significantly reduces late-stage training failures compared to strong robustness baselines.