RatBodyFormer: Rat Body Surface from Keypoints
This work addresses automated rat behavior analysis for scientific studies by providing a method to model body surface movement, but it appears incremental as it builds on existing keypoint detection methods.
The paper tackles the problem of reconstructing the rat body surface from sparse keypoints to capture subtle behaviors like curling and stretching, introducing RatBodyFormer, which achieves this by learning from a new dataset (RatDome) and using masked-learning, though no concrete performance numbers are provided.
Analyzing rat behavior lies at the heart of many scientific studies. Past methods for automated rodent modeling have focused on 3D pose estimation from keypoints, e.g., face and appendages. The pose, however, does not capture the rich body surface movement encoding the subtle rat behaviors like curling and stretching. The body surface lacks features that can be visually defined, evading these established keypoint-based methods. In this paper, we introduce the first method for reconstructing the rat body surface as a dense set of points by learning to predict it from the sparse keypoints that can be detected with past methods. Our method consists of two key contributions. The first is RatDome, a novel multi-camera system for rat behavior capture, and a large-scale dataset captured with it that consists of pairs of 3D keypoints and 3D body surface points. The second is RatBodyFormer, a novel network to transform detected keypoints to 3D body surface points. RatBodyFormer is agnostic to the exact locations of the 3D body surface points in the training data and is trained with masked-learning. We experimentally validate our framework with a number of real-world experiments. Our results collectively serve as a novel foundation for automated rat behavior analysis.