Human-Corrected Labels Learning: Enhancing Labels Quality via Human Correction of VLMs Discrepancies
This work addresses label quality issues in weak supervision scenarios for machine learning practitioners, though it is incremental as it builds on existing VLM-based annotation methods.
The paper tackles the problem of low-quality and noisy labels generated by Vision-Language Models (VLMs) for data annotation by proposing Human-Corrected Labels (HCL), a setting that uses human correction only for instances with VLM discrepancies, resulting in higher-quality annotations and reduced labor costs, with extensive experiments showing superior classification performance and robustness to label noise.
Vision-Language Models (VLMs), with their powerful content generation capabilities, have been successfully applied to data annotation processes. However, the VLM-generated labels exhibit dual limitations: low quality (i.e., label noise) and absence of error correction mechanisms. To enhance label quality, we propose Human-Corrected Labels (HCLs), a novel setting that efficient human correction for VLM-generated noisy labels. As shown in Figure 1(b), HCL strategically deploys human correction only for instances with VLM discrepancies, achieving both higher-quality annotations and reduced labor costs. Specifically, we theoretically derive a risk-consistent estimator that incorporates both human-corrected labels and VLM predictions to train classifiers. Besides, we further propose a conditional probability method to estimate the label distribution using a combination of VLM outputs and model predictions. Extensive experiments demonstrate that our approach achieves superior classification performance and is robust to label noise, validating the effectiveness of HCL in practical weak supervision scenarios. Code https://github.com/Lilianach24/HCL.git