Self-Supervision for Tackling Unsupervised Anomaly Detection: Pitfalls and Opportunities
This vision paper addresses challenges in applying SSL to anomaly detection, which is incremental as it synthesizes existing literature and proposes directions rather than introducing new methods.
The paper examines how self-supervised learning (SSL) strategies impact unsupervised anomaly detection (AD) performance, highlighting the importance of hyperparameter tuning and model selection, and identifies future opportunities like designing new pretext tasks and leveraging foundation models.
Self-supervised learning (SSL) is a growing torrent that has recently transformed machine learning and its many real world applications, by learning on massive amounts of unlabeled data via self-generated supervisory signals. Unsupervised anomaly detection (AD) has also capitalized on SSL, by self-generating pseudo-anomalies through various data augmentation functions or external data exposure. In this vision paper, we first underline the importance of the choice of SSL strategies on AD performance, by presenting evidences and studies from the AD literature. Equipped with the understanding that SSL incurs various hyperparameters (HPs) to carefully tune, we present recent developments on unsupervised model selection and augmentation tuning for SSL-based AD. We then highlight emerging challenges and future opportunities; on designing new pretext tasks and augmentation functions for different data modalities, creating novel model selection solutions for systematically tuning the SSL HPs, as well as on capitalizing on the potential of pretrained foundation models on AD through effective density estimation.