Under the Cover Infant Pose Estimation using Multimodal Data
This work addresses a specific challenge in healthcare and home monitoring for infants, but it is incremental as it adapts existing methods to a new dataset.
The paper tackles the problem of infant pose estimation during sleep under heavy occlusions from blankets, by introducing a novel multimodal dataset (SMaL) and using transfer learning with hierarchical pretraining, achieving 86% joint detection within 25mm and a mean error of 16.9mm.
Infant pose monitoring during sleep has multiple applications in both healthcare and home settings. In a healthcare setting, pose detection can be used for region of interest detection and movement detection for noncontact based monitoring systems. In a home setting, pose detection can be used to detect sleep positions which has shown to have a strong influence on multiple health factors. However, pose monitoring during sleep is challenging due to heavy occlusions from blanket coverings and low lighting. To address this, we present a novel dataset, Simultaneously-collected multimodal Mannequin Lying pose (SMaL) dataset, for under the cover infant pose estimation. We collect depth and pressure imagery of an infant mannequin in different poses under various cover conditions. We successfully infer full body pose under the cover by training state-of-art pose estimation methods and leveraging existing multimodal adult pose datasets for transfer learning. We demonstrate a hierarchical pretraining strategy for transformer-based models to significantly improve performance on our dataset. Our best performing model was able to detect joints under the cover within 25mm 86% of the time with an overall mean error of 16.9mm. Data, code and models publicly available at https://github.com/DanielKyr/SMaL