Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles
This addresses the challenge of reducing manual labeling effort in semi-supervised learning for computer vision, though it is incremental as it builds on existing auto-annotation and filtering techniques.
The paper tackles the problem of performance degradation from inaccurate auto-annotations in semi-supervised learning by proposing a filtering method based on ensemble consensus to predict annotation quality. It shows that using this filtered data can achieve state-of-the-art performance in semantic segmentation with only 30% of the original manually labeled data.
Auto-annotation by ensemble of models is an efficient method of learning on unlabeled data. Wrong or inaccurate annotations generated by the ensemble may lead to performance degradation of the trained model. To deal with this problem we propose filtering the auto-labeled data using a trained model that predicts the quality of the annotation from the degree of consensus between ensemble models. Using semantic segmentation as an example, we show the advantage of the proposed auto-annotation filtering over training on data contaminated with inaccurate labels. Moreover, our experimental results show that in the case of semantic segmentation, the performance of a state-of-the-art model can be achieved by training it with only a fraction (30$\%$) of the original manually labeled data set, and replacing the rest with the auto-annotated, quality filtered labels.