StyleDiff: Attribute Comparison Between Unlabeled Datasets in Latent Disentangled Space
This addresses the challenge for developers in ensuring reliable ML systems by identifying dataset discrepancies, though it is incremental as it builds on existing generative models.
The paper tackles the problem of dataset mismatches in machine learning by proposing StyleDiff, a method that compares unlabeled datasets in latent disentangled space to detect differences in image attributes, achieving accurate detection with a computational complexity of O(d N log N).
One major challenge in machine learning applications is coping with mismatches between the datasets used in the development and those obtained in real-world applications. These mismatches may lead to inaccurate predictions and errors, resulting in poor product quality and unreliable systems. In this study, we propose StyleDiff to inform developers of the differences between the two datasets for the steady development of machine learning systems. Using disentangled image spaces obtained from recently proposed generative models, StyleDiff compares the two datasets by focusing on attributes in the images and provides an easy-to-understand analysis of the differences between the datasets. The proposed StyleDiff performs in $O (d N\log N)$, where $N$ is the size of the datasets and $d$ is the number of attributes, enabling the application to large datasets. We demonstrate that StyleDiff accurately detects differences between datasets and presents them in an understandable format using, for example, driving scenes datasets.