Decoupling "when to update" from "how to update"
It addresses noisy labels for deep learning practitioners, but appears incremental as it builds on existing methods for handling label noise.
The paper tackles the problem of noisy labels in deep learning by proposing a meta algorithm that decouples 'when to update' from 'how to update', achieving state-of-the-art results on a gender classification task mined from the LFW dataset and a textual genderizing service.
Deep learning requires data. A useful approach to obtain data is to be creative and mine data from various sources, that were created for different purposes. Unfortunately, this approach often leads to noisy labels. In this paper, we propose a meta algorithm for tackling the noisy labels problem. The key idea is to decouple "when to update" from "how to update". We demonstrate the effectiveness of our algorithm by mining data for gender classification by combining the Labeled Faces in the Wild (LFW) face recognition dataset with a textual genderizing service, which leads to a noisy dataset. While our approach is very simple to implement, it leads to state-of-the-art results. We analyze some convergence properties of the proposed algorithm.