Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning
This work provides a mechanistic understanding of how memory length affects olfactory search efficiency in dynamic flows, relevant to robotics and animal behavior research.
The authors used reinforcement learning agents to study olfactory navigation in unsteady flows, finding that agents develop a flow-assisted casting strategy that adapts trajectory geometry and concentration thresholds, with search efficiency showing a non-monotonic dependence on memory length explained by a sector-search model.
In dynamic flow fields, various animals exhibit remarkable odor search capabilities despite relying on stochastic detections. Interestingly, there exists an optimal time window for integrating these detections that maximizes search efficiency. To understand the underlying mechanism, we investigate the navigation performance of Reinforcement Learning (RL) agents in unsteady flows under varying memory lengths and flow conditions. Without any predefined models, the agents develop a flow-assisted casting strategy and adaptively adjust both the geometry of their search trajectories and the concentration threshold for initiating casting to maximize the success rate. The agent's average speed toward the odor source exhibits a non-monotonic dependence on memory length, which can be explained by the "sector-search" model.