Toward Unsupervised Outlier Model Selection
This addresses the understudied problem of selecting outlier detection models without labels, which is incremental as it builds on meta-learning with a novel similarity measure.
The paper tackles the problem of unsupervised outlier model selection (UOMS) by proposing ELECT, a meta-learning approach that uses performance-based dataset similarity to transfer knowledge from historical datasets. Experiments show that ELECT significantly outperforms various baselines, including no selection and meta-feature-based strategies.
Today there exists no shortage of outlier detection algorithms in the literature, yet the complementary and critical problem of unsupervised outlier model selection (UOMS) is vastly understudied. In this work we propose ELECT, a new approach to select an effective candidate model, i.e. an outlier detection algorithm and its hyperparameter(s), to employ on a new dataset without any labels. At its core, ELECT is based on meta-learning; transferring prior knowledge (e.g. model performance) on historical datasets that are similar to the new one to facilitate UOMS. Uniquely, it employs a dataset similarity measure that is performance-based, which is more direct and goal-driven than other measures used in the past. ELECT adaptively searches for similar historical datasets, as such, it can serve an output on-demand, being able to accommodate varying time budgets. Extensive experiments show that ELECT significantly outperforms a wide range of basic UOMS baselines, including no model selection (always using the same popular model such as iForest) as well as more recent selection strategies based on meta-features.