Robust learning of data anomalies with analytically-solvable entropic outlier sparsification
This addresses the need for efficient and robust anomaly detection in data analysis, though it is incremental as it builds on existing regularization and mixture model concepts.
The authors tackled the problem of robust anomaly detection in both unsupervised and supervised learning by proposing Entropic Outlier Sparsification (EOS), which provides an analytically-solvable method with linear cost scaling independent of data dimension, achieving competitive performance on synthetic and biomedical datasets.
Entropic Outlier Sparsification (EOS) is proposed as a robust computational strategy for the detection of data anomalies in a broad class of learning methods, including the unsupervised problems (like detection of non-Gaussian outliers in mostly-Gaussian data) and in the supervised learning with mislabeled data. EOS dwells on the derived analytic closed-form solution of the (weighted) expected error minimization problem subject to the Shannon entropy regularization. In contrast to common regularization strategies requiring computational costs that scale polynomial with the data dimension, identified closed-form solution is proven to impose additional iteration costs that depend linearly on statistics size and are independent of data dimension. Obtained analytic results also explain why the mixtures of spherically-symmetric Gaussians - used heuristically in many popular data analysis algorithms - represent an optimal choice for the non-parametric probability distributions when working with squared Euclidean distances, combining expected error minimality, maximal entropy/unbiasedness, and a linear cost scaling. The performance of EOS is compared to a range of commonly-used tools on synthetic problems and on partially-mislabeled supervised classification problems from biomedicine.