The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck
This work is incremental, providing a structured overview for practitioners and researchers to navigate scattered methods for reducing data dependency in ML.
The paper tackles the data bottleneck problem in machine learning by proposing a taxonomy of existing methods to address the need for large datasets, aiming to raise awareness and inspire more efficient resource use and novel strategies.
Machine learning (ML) is revolutionizing the world, affecting almost every field of science and industry. Recent algorithms (in particular, deep networks) are increasingly data-hungry, requiring large datasets for training. Thus, the dominant paradigm in ML today involves constructing large, task-specific datasets. However, obtaining quality datasets of such magnitude proves to be a difficult challenge. A variety of methods have been proposed to address this data bottleneck problem, but they are scattered across different areas, and it is hard for a practitioner to keep up with the latest developments. In this work, we propose a taxonomy of these methods. Our goal is twofold: (1) We wish to raise the community's awareness of the methods that already exist and encourage more efficient use of resources, and (2) we hope that such a taxonomy will contribute to our understanding of the problem, inspiring novel ideas and strategies to replace current annotation-heavy approaches.