Implicit Data Synthesis for Contrastive Unsupervised Data Augmentation
For scientists working with unlabeled scientific observations where data-space augmentations are inappropriate, this method offers a way to apply contrastive learning without altering data structure.
The authors propose a contrastive learning method that generates synthetic samples by perturbing network weights instead of data, preserving data structure for scientific observations. Applied to radar meteor data with a SimCLR pipeline, it shows performance gains under matched protocols.
Scientific observations generate large quantities of unlabeled data which is laborious to hand-label, making unsupervised learning techniques valuable for processing datasets. Among these approaches, contrastive learning provides a convenient mechanism for extracting structural representations from unannotated datasets. For natural imagery, the general approach is to use a variety of data-space augmentation methods in order to generate synthetic samples; however, for scientific observations data-space perturbations can fundamentally alter the underlying data. Our proposed method is to generate contrastive samples by perturbing the network weights rather than the underlying data, thus more closely preserving the structure of the data. We demonstrate this technique using a SimCLR-based pipeline applied over radar observations of meteors, and show performance gains under matched protocols.