CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities
This work addresses the challenge of clinical utility in sleep analysis by enhancing robustness to noisy or missing data, though it appears incremental as it builds on existing multimodal fusion approaches.
The paper tackles the problem of automated sleep staging with imperfect or missing multimodal data, proposing CoRe-Sleep, a fusion framework that achieves state-of-the-art performance on the SHHS-1 dataset, showing improved robustness and positive influence on unimodal testing.
Sleep abnormalities can have severe health consequences. Automated sleep staging, i.e. labelling the sequence of sleep stages from the patient's physiological recordings, could simplify the diagnostic process. Previous work on automated sleep staging has achieved great results, mainly relying on the EEG signal. However, often multiple sources of information are available beyond EEG. This can be particularly beneficial when the EEG recordings are noisy or even missing completely. In this paper, we propose CoRe-Sleep, a Coordinated Representation multimodal fusion network that is particularly focused on improving the robustness of signal analysis on imperfect data. We demonstrate how appropriately handling multimodal information can be the key to achieving such robustness. CoRe-Sleep tolerates noisy or missing modalities segments, allowing training on incomplete data. Additionally, it shows state-of-the-art performance when testing on both multimodal and unimodal data using a single model on SHHS-1, the largest publicly available study that includes sleep stage labels. The results indicate that training the model on multimodal data does positively influence performance when tested on unimodal data. This work aims at bridging the gap between automated analysis tools and their clinical utility.