Local Contrast Learning
This addresses the challenge of overfitting in parametric models due to lack of training samples, with potential applications in domains requiring few-shot learning, though it appears incremental as it builds on existing deep learning approaches for small data.
The paper tackles the problem of learning deep models from small data, specifically one-shot classification, by proposing Local Contrast Learning (LCL) based on human cognitive behavior of contrasting objects in context. The result is a 122-layer deep model trained on a tiny dataset (60 classes, 20 samples per class) that achieved 97.99% accuracy on Omniglot, outperforming human performance and the state-of-the-art Bayesian Program Learning method.
Learning a deep model from small data is yet an opening and challenging problem. We focus on one-shot classification by deep learning approach based on a small quantity of training samples. We proposed a novel deep learning approach named Local Contrast Learning (LCL) based on the key insight about a human cognitive behavior that human recognizes the objects in a specific context by contrasting the objects in the context or in her/his memory. LCL is used to train a deep model that can contrast the recognizing sample with a couple of contrastive samples randomly drawn and shuffled. On one-shot classification task on Omniglot, the deep model based LCL with 122 layers and 1.94 millions of parameters, which was trained on a tiny dataset with only 60 classes and 20 samples per class, achieved the accuracy 97.99% that outperforms human and state-of-the-art established by Bayesian Program Learning (BPL) trained on 964 classes. LCL is a fundamental idea which can be applied to alleviate parametric model's overfitting resulted by lack of training samples.