Agreement-based Learning
This addresses model selection challenges in machine learning, particularly for deep learning applications, though it appears incremental as it builds on existing ensemble and agreement methods.
The paper tackles the model selection problem in machine learning by proposing an agreement-based learning framework that couples multiple models to encourage prediction agreement during training, resulting in a learning algorithm that significantly outperforms alternatives and improves with unlabeled data.
Model selection is a problem that has occupied machine learning researchers for a long time. Recently, its importance has become evident through applications in deep learning. We propose an agreement-based learning framework that prevents many of the pitfalls associated with model selection. It relies on coupling the training of multiple models by encouraging them to agree on their predictions while training. In contrast with other model selection and combination approaches used in machine learning, the proposed framework is inspired by human learning. We also propose a learning algorithm defined within this framework which manages to significantly outperform alternatives in practice, and whose performance improves further with the availability of unlabeled data. Finally, we describe a number of potential directions for developing more flexible agreement-based learning algorithms.