Should I Stay or Should I Go: Coordinating Biological Needs with Continuously-updated Assessments of the Environment
This work addresses the challenge of balancing exploration and safety in adaptive systems, but it is incremental as it builds on existing motivational and learning frameworks without introducing a new paradigm.
The paper tackles the problem of how autonomous adaptive systems coordinate internal biological needs with continuous environmental assessments, resulting in the Wanderer model that adaptively modulates exploration based on satiation and predator avoidance, with behaviors like decreased exploration after food or predator encounters and associative learning between neutral and salient features.
This paper presents Wanderer, a model of how autonomous adaptive systems coordinate internal biological needs with moment-by-moment assessments of the probabilities of events in the external world. The extent to which Wanderer moves about or explores its environment reflects the relative activations of two competing motivational sub-systems: one represents the need to acquire energy and it excites exploration, and the other represents the need to avoid predators and it inhibits exploration. The environment contains food, predators, and neutral stimuli. Wanderer responds to these events in a way that is adaptive in the short turn, and reassesses the probabilities of these events so that it can modify its long term behaviour appropriately. When food appears, Wanderer be-comes satiated and exploration temporarily decreases. When a predator appears, Wanderer both decreases exploration in the short term, and becomes more "cautious" about exploring in the future. Wanderer also forms associations between neutral features and salient ones (food and predators) when they are present at the same time, and uses these associations to guide its behaviour.