SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
It addresses the problem of leveraging self-supervised learning for tabular datasets, which is incremental as it adapts existing techniques to a new domain.
The paper tackles the lack of self-supervised contrastive learning methods for tabular data by proposing SCARF, which improves classification accuracy on 69 real-world datasets, including in noisy and semi-supervised settings.
Self-supervised contrastive representation learning has proved incredibly successful in the vision and natural language domains, enabling state-of-the-art performance with orders of magnitude less labeled data. However, such methods are domain-specific and little has been done to leverage this technique on real-world tabular datasets. We propose SCARF, a simple, widely-applicable technique for contrastive learning, where views are formed by corrupting a random subset of features. When applied to pre-train deep neural networks on the 69 real-world, tabular classification datasets from the OpenML-CC18 benchmark, SCARF not only improves classification accuracy in the fully-supervised setting but does so also in the presence of label noise and in the semi-supervised setting where only a fraction of the available training data is labeled. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders. We conduct comprehensive ablations, detailing the importance of a range of factors.