Balancing New Against Old Information: The Role of Surprise in Learning
This work addresses the challenge of learning in complex, changing environments, potentially providing a framework to study human and animal behavior in response to surprising events, though it appears incremental as it builds on existing surprise concepts.
The authors tackled the problem of balancing new and old information in learning by proposing a surprise measure that incorporates data likelihood and belief entropy, and found that surprise-minimizing learning dynamically adjusts this balance without needing temporal statistics knowledge, applied to dynamic decision-making and maze exploration tasks.
Surprise describes a range of phenomena from unexpected events to behavioral responses. We propose a measure of surprise and use it for surprise-driven learning. Our surprise measure takes into account data likelihood as well as the degree of commitment to a belief via the entropy of the belief distribution. We find that surprise-minimizing learning dynamically adjusts the balance between new and old information without the need of knowledge about the temporal statistics of the environment. We apply our framework to a dynamic decision-making task and a maze exploration task. Our surprise minimizing framework is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes and could eventually provide a framework to study the behavior of humans and animals encountering surprising events.