Deep Multi-stream Network for Video-based Calving Sign Detection
This work addresses the practical need for interpretable and efficient calving detection systems for farmers, though it is incremental as it builds on existing deep learning methods with domain-specific adaptations.
The paper tackled the problem of automatically detecting calving signs from video for livestock management by proposing a deep multi-stream network that incorporates expert knowledge, resulting in a significant improvement over end-to-end systems and reduced detection errors in experiments with videos of 15 cows.
We have designed a deep multi-stream network for automatically detecting calving signs from video. Calving sign detection from a camera, which is a non-contact sensor, is expected to enable more efficient livestock management. As large-scale, well-developed data cannot generally be assumed when establishing calving detection systems, the basis for making the prediction needs to be presented to farmers during operation, so black-box modeling (also known as end-to-end modeling) is not appropriate. For practical operation of calving detection systems, the present study aims to incorporate expert knowledge into a deep neural network. To this end, we propose a multi-stream calving sign detection network in which multiple calving-related features are extracted from the corresponding feature extraction networks designed for each attribute with different characteristics, such as a cow's posture, rotation, and movement, known as calving signs, and are then integrated appropriately depending on the cow's situation. Experimental comparisons conducted using videos of 15 cows demonstrated that our multi-stream system yielded a significant improvement over the end-to-end system, and the multi-stream architecture significantly contributed to a reduction in detection errors. In addition, the distinctive mixture weights we observed helped provide interpretability of the system's behavior.