Self-Supervised Dynamical System Representations for Physiological Time-Series
This work addresses the challenge of learning effective representations from noisy physiological data for applications in healthcare, though it is incremental by building on existing SSL methods with a novel dynamical systems approach.
The paper tackled the problem of self-supervised learning for physiological time-series by proposing PULSE, a pretraining framework based on dynamical systems to extract system parameters and discard noise, which improved class distinction, label efficiency, and transfer learning in real-world datasets.
The effectiveness of self-supervised learning (SSL) for physiological time series depends on the ability of a pretraining objective to preserve information about the underlying physiological state while filtering out unrelated noise. However, existing strategies are limited due to reliance on heuristic principles or poorly constrained generative tasks. To address this limitation, we propose a pretraining framework that exploits the information structure of a dynamical systems generative model across multiple time-series. This framework reveals our key insight that class identity can be efficiently captured by extracting information about the generative variables related to the system parameters shared across similar time series samples, while noise unique to individual samples should be discarded. Building on this insight, we propose PULSE, a cross-reconstruction-based pretraining objective for physiological time series datasets that explicitly extracts system information while discarding non-transferrable sample-specific ones. We establish theory that provides sufficient conditions for the system information to be recovered, and empirically validate it using a synthetic dynamical systems experiment. Furthermore, we apply our method to diverse real-world datasets, demonstrating that PULSE learns representations that can broadly distinguish semantic classes, increase label efficiency, and improve transfer learning.