Learning Behavior Representations Through Multi-Timescale Bootstrapping
This addresses the challenge of analyzing free and naturalistic behavior for researchers in fields like robotics or neuroscience, though it is incremental as it builds on existing representation learning methods.
The authors tackled the problem of representing natural behavior with unpredictable and multi-timescale dynamics by introducing BAMS, a multi-scale representation learning model, which ranked 3rd overall and 1st on two subtasks in the MABe 2022 challenge.
Natural behavior consists of dynamics that are both unpredictable, can switch suddenly, and unfold over many different timescales. While some success has been found in building representations of behavior under constrained or simplified task-based conditions, many of these models cannot be applied to free and naturalistic settings due to the fact that they assume a single scale of temporal dynamics. In this work, we introduce Bootstrap Across Multiple Scales (BAMS), a multi-scale representation learning model for behavior: we combine a pooling module that aggregates features extracted over encoders with different temporal receptive fields, and design a set of latent objectives to bootstrap the representations in each respective space to encourage disentanglement across different timescales. We first apply our method on a dataset of quadrupeds navigating in different terrain types, and show that our model captures the temporal complexity of behavior. We then apply our method to the MABe 2022 Multi-agent behavior challenge, where our model ranks 3rd overall and 1st on two subtasks, and show the importance of incorporating multi-timescales when analyzing behavior.