Dual-stream spatiotemporal networks with feature sharing for monitoring animals in the home cage
This work addresses automated monitoring of animal behavior for researchers, representing an incremental improvement with a novel feature-sharing method.
The paper tackles mouse behavioral classification in home-cage settings by introducing a dual-stream spatiotemporal network with feature sharing, achieving 86.47% prediction accuracy on a public dataset.
This paper presents a spatiotemporal deep learning approach for mouse behavioural classification in the home-cage. Using a series of dual-stream architectures with assorted modifications to increase performance, we introduce a novel feature sharing approach that jointly processes the streams at regular intervals throughout the network. To investigate the efficacy of this approach, models were evaluated by dissociating the streams and training/testing in the same rigorous manner as the main classifiers. Using an annotated, publicly available dataset of a singly-housed mice, we achieve prediction accuracy of 86.47% using an ensemble of a Inception-based network and an attention-based network, both of which utilize this feature sharing. We also demonstrate through ablation studies that for all models, the feature-sharing architectures consistently perform better than conventional ones having separate streams. The best performing models were further evaluated on other activity datasets, both mouse and human. Future work will investigate the effectiveness of feature sharing to behavioural classification in the unsupervised anomaly detection domain.