Enabling On-Device Self-Supervised Contrastive Learning With Selective Data Contrast
This addresses the challenge of continuous model improvement on resource-constrained edge devices, though it appears incremental as it builds on existing contrastive learning methods.
The paper tackles the problem of enabling on-device self-supervised contrastive learning for edge devices with limited storage and non-iid data streams by proposing a framework to automatically select representative data, resulting in greatly improved accuracy and learning speed.
After a model is deployed on edge devices, it is desirable for these devices to learn from unlabeled data to continuously improve accuracy. Contrastive learning has demonstrated its great potential in learning from unlabeled data. However, the online input data are usually none independent and identically distributed (non-iid) and storages of edge devices are usually too limited to store enough representative data from different data classes. We propose a framework to automatically select the most representative data from the unlabeled input stream, which only requires a small data buffer for dynamic learning. Experiments show that accuracy and learning speed are greatly improved.