Bayesian Inverse Reinforcement Learning for Collective Animal Movement
This work addresses the challenge of understanding collective behavior in animal groups, but it is incremental as it applies existing methods to new data.
The authors tackled the problem of inferring local rules governing collective animal movement by applying Bayesian inverse reinforcement learning to a self-propelled particle simulation and guppy data, recovering true costs in simulation and finding guppies value collective movement over targeted shelter movement.
Agent-based methods allow for defining simple rules that generate complex group behaviors. The governing rules of such models are typically set a priori and parameters are tuned from observed behavior trajectories. Instead of making simplifying assumptions across all anticipated scenarios, inverse reinforcement learning provides inference on the short-term (local) rules governing long term behavior policies by using properties of a Markov decision process. We use the computationally efficient linearly-solvable Markov decision process to learn the local rules governing collective movement for a simulation of the self propelled-particle (SPP) model and a data application for a captive guppy population. The estimation of the behavioral decision costs is done in a Bayesian framework with basis function smoothing. We recover the true costs in the SPP simulation and find the guppies value collective movement more than targeted movement toward shelter.