A Multivariate Extreme Value Theory Approach to Anomaly Clustering and Visualization
This work addresses anomaly detection and visualization for monitoring system health, particularly in aeronautics, but is incremental as it builds on existing extreme value theory methods.
The paper tackles the problem of identifying and visualizing anomalies in complex systems by developing a novel mixture model based on multivariate extreme value theory, which assigns posterior probabilities to anomaly types and enables clustering and 2D visualization, demonstrating effectiveness on simulated and real aeronautics datasets.
In a wide variety of situations, anomalies in the behaviour of a complex system, whose health is monitored through the observation of a random vector X = (X1,. .. , X d) valued in R d , correspond to the simultaneous occurrence of extreme values for certain subgroups $α$ $\subset$ {1,. .. , d} of variables Xj. Under the heavy-tail assumption, which is precisely appropriate for modeling these phenomena, statistical methods relying on multivariate extreme value theory have been developed in the past few years for identifying such events/subgroups. This paper exploits this approach much further by means of a novel mixture model that permits to describe the distribution of extremal observations and where the anomaly type $α$ is viewed as a latent variable. One may then take advantage of the model by assigning to any extreme point a posterior probability for each anomaly type $α$, defining implicitly a similarity measure between anomalies. It is explained at length how the latter permits to cluster extreme observations and obtain an informative planar representation of anomalies using standard graph-mining tools. The relevance and usefulness of the clustering and 2-d visual display thus designed is illustrated on simulated datasets and on real observations as well, in the aeronautics application domain.