A data-centric approach for improving ambiguous labels with combined semi-supervised classification and clustering
This addresses the issue of subjective annotations in datasets for deep learning practitioners, offering an incremental improvement by enhancing existing SSL methods for better label handling.
The paper tackles the problem of ambiguous labels in real-world datasets by proposing a data-centric method called DC3, which combines semi-supervised classification and clustering to automatically relabel ambiguous data, resulting in an average 7.6% better F1-score for classifications and 7.9% lower inner distance of clusters across multiple SSL algorithms and datasets.
Consistently high data quality is essential for the development of novel loss functions and architectures in the field of deep learning. The existence of such data and labels is usually presumed, while acquiring high-quality datasets is still a major issue in many cases. In real-world datasets we often encounter ambiguous labels due to subjective annotations by annotators. In our data-centric approach, we propose a method to relabel such ambiguous labels instead of implementing the handling of this issue in a neural network. A hard classification is by definition not enough to capture the real-world ambiguity of the data. Therefore, we propose our method "Data-Centric Classification & Clustering (DC3)" which combines semi-supervised classification and clustering. It automatically estimates the ambiguity of an image and performs a classification or clustering depending on that ambiguity. DC3 is general in nature so that it can be used in addition to many Semi-Supervised Learning (SSL) algorithms. On average, this results in a 7.6% better F1-Score for classifications and 7.9% lower inner distance of clusters across multiple evaluated SSL algorithms and datasets. Most importantly, we give a proof-of-concept that the classifications and clusterings from DC3 are beneficial as proposals for the manual refinement of such ambiguous labels. Overall, a combination of SSL with our method DC3 can lead to better handling of ambiguous labels during the annotation process.