Distance in Latent Space as Novelty Measure
This addresses the data collection bottleneck for practitioners in deep learning, though it is incremental as it builds on existing active learning and self-supervised techniques.
The paper tackles the problem of expensive data labeling in deep learning by proposing a method to intelligently select diverse samples for dataset construction, resulting in equal or superior performance with fewer labeled examples.
Deep Learning performs well when training data densely covers the experience space. For complex problems this makes data collection prohibitively expensive. We propose to intelligently select samples when constructing data sets in order to best utilize the available labeling budget. The selection methodology is based on the presumption that two dissimilar samples are worth more than two similar samples in a data set. Similarity is measured based on the Euclidean distance between samples in the latent space produced by a DNN. By using a self-supervised method to construct the latent space, it is ensured that the space fits the data well and that any upfront labeling effort can be avoided. The result is more efficient, diverse, and balanced data set, which produce equal or superior results with fewer labeled examples.