Data-Efficient Challenges in Visual Inductive Priors: A Retrospective
This addresses data efficiency challenges in computer vision, but it is incremental as it builds on existing workshop formats and methods.
The paper tackles the problem of training deep learning models for computer vision with limited data by organizing a workshop series with data-impaired challenges, where participants achieved success using methods like large model ensembles and heavy data augmentation.
Deep Learning requires large amounts of data to train models that work well. In data-deficient settings, performance can be degraded. We investigate which Deep Learning methods benefit training models in a data-deficient setting, by organizing the "VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning" workshop series, featuring four editions of data-impaired challenges. These challenges address the problem of training deep learning models for computer vision tasks with limited data. Participants are limited to training models from scratch using a low number of training samples and are not allowed to use any form of transfer learning. We aim to stimulate the development of novel approaches that incorporate prior knowledge to improve the data efficiency of deep learning models. Successful challenge entries make use of large model ensembles that mix Transformers and CNNs, as well as heavy data augmentation. Novel prior knowledge-based methods contribute to success in some entries.