Fast Unsupervised Deep Outlier Model Selection with Hypernetworks
This addresses a critical bottleneck for practitioners in anomaly detection by enabling faster and more effective tuning of deep models without supervision, though it is incremental as it builds on existing hypernetwork and meta-learning techniques.
The paper tackles the challenge of hyperparameter tuning and model selection for unsupervised deep outlier detection, where labeled anomalies are unavailable, by introducing HYPER, a method using hypernetworks and meta-learning that achieves high performance and significant efficiency gains across 35 outlier detection tasks.
Outlier detection (OD) finds many applications with a rich literature of numerous techniques. Deep neural network based OD (DOD) has seen a recent surge of attention thanks to the many advances in deep learning. In this paper, we consider a critical-yet-understudied challenge with unsupervised DOD, that is, effective hyperparameter (HP) tuning/model selection. While several prior work report the sensitivity of OD models to HPs, it becomes ever so critical for the modern DOD models that exhibit a long list of HPs. We introduce HYPER for tuning DOD models, tackling two fundamental challenges: (1) validation without supervision (due to lack of labeled anomalies), and (2) efficient search of the HP/model space (due to exponential growth in the number of HPs). A key idea is to design and train a novel hypernetwork (HN) that maps HPs onto optimal weights of the main DOD model. In turn, HYPER capitalizes on a single HN that can dynamically generate weights for many DOD models (corresponding to varying HPs), which offers significant speed-up. In addition, it employs meta-learning on historical OD tasks with labels to train a proxy validation function, likewise trained with our proposed HN efficiently. Extensive experiments on 35 OD tasks show that HYPER achieves high performance against 8 baselines with significant efficiency gains.