Tracking Emotions: Intrinsic Motivation Grounded on Multi-Level Prediction Error Dynamics
This work addresses the challenge of dynamic goal selection and exploration-exploitation balance in cognitive agents, though it appears incremental in the context of intrinsic motivation research.
The paper tackles the problem of how artificial agents can be intrinsically motivated to seek new experiences by tracking prediction error dynamics, and it shows that their architecture outperforms fixed-goal and greedy intrinsic motivation approaches.
How do cognitive agents decide what is the relevant information to learn and how goals are selected to gain this knowledge? Cognitive agents need to be motivated to perform any action. We discuss that emotions arise when differences between expected and actual rates of progress towards a goal are experienced. Therefore, the tracking of prediction error dynamics has a tight relationship with emotions. Here, we suggest that the tracking of prediction error dynamics allows an artificial agent to be intrinsically motivated to seek new experiences but constrained to those that generate reducible prediction error.We present an intrinsic motivation architecture that generates behaviors towards self-generated and dynamic goals and that regulates goal selection and the balance between exploitation and exploration through multi-level monitoring of prediction error dynamics. This new architecture modulates exploration noise and leverages computational resources according to the dynamics of the overall performance of the learning system. Additionally, it establishes a possible solution to the temporal dynamics of goal selection. The results of the experiments presented here suggest that this architecture outperforms intrinsic motivation approaches where exploratory noise and goals are fixed and a greedy strategy is applied.