Modeling Heterogeneous Statistical Patterns in High-dimensional Data by Adversarial Distributions: An Unsupervised Generative Framework
This work addresses the problem of modeling intrinsic clusters with heterogeneous statistical patterns in high-dimensional data for applications like fraud detection and anomaly detection, where label collection is prohibitive.
This paper proposes FIRD, an unsupervised generative framework that uses adversarial distributions to model and disentangle heterogeneous statistical patterns in high-dimensional data. It effectively distinguishes synchronized fraudsters from normal users and achieves over 5% average AUC improvement on anomaly detection datasets compared to SOTA methods.
Since the label collecting is prohibitive and time-consuming, unsupervised methods are preferred in applications such as fraud detection. Meanwhile, such applications usually require modeling the intrinsic clusters in high-dimensional data, which usually displays heterogeneous statistical patterns as the patterns of different clusters may appear in different dimensions. Existing methods propose to model the data clusters on selected dimensions, yet globally omitting any dimension may damage the pattern of certain clusters. To address the above issues, we propose a novel unsupervised generative framework called FIRD, which utilizes adversarial distributions to fit and disentangle the heterogeneous statistical patterns. When applying to discrete spaces, FIRD effectively distinguishes the synchronized fraudsters from normal users. Besides, FIRD also provides superior performance on anomaly detection datasets compared with SOTA anomaly detection methods (over 5% average AUC improvement). The significant experiment results on various datasets verify that the proposed method can better model the heterogeneous statistical patterns in high-dimensional data and benefit downstream applications.