On the Query Strategies for Efficient Online Active Distillation
It addresses computational efficiency for real-time model adaptation in Human Pose Estimation, but appears incremental as it evaluates existing strategies rather than introducing new ones.
This paper tackles the problem of efficiently training deep learning models for real-time adaptation by evaluating query strategies for online active distillation in Human Pose Estimation, showing that it enables training lightweight models at the edge with effective adaptation to new contexts.
Deep Learning (DL) requires lots of time and data, resulting in high computational demands. Recently, researchers employ Active Learning (AL) and online distillation to enhance training efficiency and real-time model adaptation. This paper evaluates a set of query strategies to achieve the best training results. It focuses on Human Pose Estimation (HPE) applications, assessing the impact of selected frames during training using two approaches: a classical offline method and a online evaluation through a continual learning approach employing knowledge distillation, on a popular state-of-the-art HPE dataset. The paper demonstrates the possibility of enabling training at the edge lightweight models, adapting them effectively to new contexts in real-time.