A Comparison of Human and Machine Learning Errors in Face Recognition
This work addresses the problem of optimizing human oversight in high-stakes machine learning applications like face recognition, but it is incremental as it focuses on comparing existing methods without introducing new techniques.
The study compared errors made by two automated face recognition systems and human annotators in a demographically balanced user study, finding important differences in their mistake patterns and suggesting strategies for human-machine collaboration to improve accuracy.
Machine learning applications in high-stakes scenarios should always operate under human oversight. Developing an optimal combination of human and machine intelligence requires an understanding of their complementarities, particularly regarding the similarities and differences in the way they make mistakes. We perform extensive experiments in the area of face recognition and compare two automated face recognition systems against human annotators through a demographically balanced user study. Our research uncovers important ways in which machine learning errors and human errors differ from each other, and suggests potential strategies in which human-machine collaboration can improve accuracy in face recognition.