Self-reinforcing Unsupervised Matching
This addresses the challenge of reducing human labor in data annotation for deep learning models when dealing with rare or unexpected modalities, though it appears incremental.
The paper tackles the problem of annotating images in emerging modalities without supervision by proposing a self-reinforcing unsupervised matching method, achieving cross-modality matching with only one template in a seen modality.
Remarkable gains in deep learning usually rely on tremendous supervised data. Ensuring the modality diversity for one object in training set is critical for the generalization of cutting-edge deep models, but it burdens human with heavy manual labor on data collection and annotation. In addition, some rare or unexpected modalities are new for the current model, causing reduced performance under such emerging modalities. Inspired by the achievements in speech recognition, psychology and behavioristics, we present a practical solution, self-reinforcing unsupervised matching (SUM), to annotate the images with 2D structure-preserving property in an emerging modality by cross-modality matching. This approach requires no any supervision in emerging modality and only one template in seen modality, providing a possible route towards continual learning.