Factor Analysis of Mixed Data for Anomaly Detection
This addresses the problem of identifying anomalies like fraud or health risks in mixed data for practitioners, but it is incremental as it builds on existing factor analysis methods.
The paper tackles anomaly detection in high-dimensional mixed data by proposing a kurtosis-weighted Factor Analysis of Mixed Data for anomaly detection (FAMDAD) to create a continuous embedding for scoring, and shows that the approach is highly accurate across simulated and real datasets.
Anomaly detection aims to identify observations that deviate from the typical pattern of data. Anomalous observations may correspond to financial fraud, health risks, or incorrectly measured data in practice. We show detecting anomalies in high-dimensional mixed data is enhanced through first embedding the data then assessing an anomaly scoring scheme. We focus on unsupervised detection and the continuous and categorical (mixed) variable case. We propose a kurtosis-weighted Factor Analysis of Mixed Data for anomaly detection, FAMDAD, to obtain a continuous embedding for anomaly scoring. We illustrate that anomalies are highly separable in the first and last few ordered dimensions of this space, and test various anomaly scoring experiments within this subspace. Results are illustrated for both simulated and real datasets, and the proposed approach (FAMDAD) is highly accurate for high-dimensional mixed data throughout these diverse scenarios.