Sampling Training Data for Continual Learning Between Robots and the Cloud
This work addresses the significant time and cost burden of training perception models for robotic fleets by intelligently sampling data on-board the robot, which is a problem for robot operators and cloud service providers.
This paper introduces HarvestNet, an intelligent sampling algorithm for robots that reduces the volume of training data sent to the cloud. It achieves a 65.7-81.3% reduction in cloud storage, annotation time, and compute time, while improving model accuracy by 1.05-2.58x compared to baselines on tasks like road construction site detection and face recognition.
Today's robotic fleets are increasingly measuring high-volume video and LIDAR sensory streams, which can be mined for valuable training data, such as rare scenes of road construction sites, to steadily improve robotic perception models. However, re-training perception models on growing volumes of rich sensory data in central compute servers (or the "cloud") places an enormous time and cost burden on network transfer, cloud storage, human annotation, and cloud computing resources. Hence, we introduce HarvestNet, an intelligent sampling algorithm that resides on-board a robot and reduces system bottlenecks by only storing rare, useful events to steadily improve perception models re-trained in the cloud. HarvestNet significantly improves the accuracy of machine-learning models on our novel dataset of road construction sites, field testing of self-driving cars, and streaming face recognition, while reducing cloud storage, dataset annotation time, and cloud compute time by between 65.7-81.3%. Further, it is between 1.05-2.58x more accurate than baseline algorithms and scalably runs on embedded deep learning hardware. We provide a suite of compute-efficient perception models for the Google Edge Tensor Processing Unit (TPU), an extended technical report, and a novel video dataset to the research community at https://sites.google.com/view/harvestnet.