Martingale-Consistent Self-Supervised Learning
For practitioners deploying SSL under partial information, this work provides a principled way to prevent systematic drift in predictions, improving reliability.
Self-supervised learning (SSL) often fails under partial observation because coarse and refined view predictions are inconsistent. The authors introduce a martingale-consistent SSL framework that enforces coherence, improving robustness and calibration across time-series, tabular, and image benchmarks.
Self-supervised learning (SSL) is often deployed under changing information, such as shorter histories, missing features, or partially observed images. In these settings, predictions from coarse and refined views should be coherent: before refinement, the coarse-view prediction should match the average prediction expected after refinement. Martingales formalize this coherence principle, but standard SSL objectives do not enforce it. Unlike invariance objectives that pull views together, martingale consistency constrains only the expected refined prediction, allowing predictions to update as information is revealed while preventing systematic drift. We introduce a martingale-consistent SSL framework that closes this gap, with practical prediction- and latent-space variants and an unbiased two-sample Monte Carlo estimator based on stochastic refinement. We evaluate the approach on synthetic and real time-series, tabular, and image benchmarks under partial-observation regimes, in both semi-self-supervised and fully label-free settings. Across these experiments, our framework improves robustness and calibration under partial observation, yielding more stable representations as information is revealed.