Continuously evolving rewards in an open-ended environment
This addresses the limitation of fixed reward functions in open-ended environments, enabling more adaptive AI agents, though it is incremental as it builds on existing dynamic reward concepts.
The paper tackled the problem of agents adapting to changing environments by proposing RULE, an algorithm for dynamically updating rewards, and demonstrated that a population of entities successfully abandoned detrimental behaviors and amplified beneficial ones in a simplified ecosystem setting.
Unambiguous identification of the rewards driving behaviours of entities operating in complex open-ended real-world environments is difficult, partly because goals and associated behaviours emerge endogenously and are dynamically updated as environments change. Reproducing such dynamics in models would be useful in many domains, particularly where fixed reward functions limit the adaptive capabilities of agents. Simulation experiments described assess a candidate algorithm for the dynamic updating of rewards, RULE: Reward Updating through Learning and Expectation. The approach is tested in a simplified ecosystem-like setting where experiments challenge entities' survival, calling for significant behavioural change. The population of entities successfully demonstrate the abandonment of an initially rewarded but ultimately detrimental behaviour, amplification of beneficial behaviour, and appropriate responses to novel items added to their environment. These adjustment happen through endogenous modification of the entities' underlying reward function, during continuous learning, without external intervention.