End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learning
This work addresses the challenge of joint tuning and real-world deployment for active object tracking, which is important for robotics and autonomous systems, though it is incremental as it builds on existing reinforcement learning methods.
The paper tackles the problem of active object tracking by proposing an end-to-end deep reinforcement learning solution that directly predicts camera control signals from visual frames, eliminating the need for separate tuning and reducing human labeling efforts. The tracker, trained in simulators, shows good generalization to unseen conditions and successfully transfers to real-world scenarios, including deployment on a robot.
We study active object tracking, where a tracker takes visual observations (i.e., frame sequences) as input and produces the corresponding camera control signals as output (e.g., move forward, turn left, etc.). Conventional methods tackle tracking and camera control tasks separately, and the resulting system is difficult to tune jointly. These methods also require significant human efforts for image labeling and expensive trial-and-error system tuning in the real world. To address these issues, we propose, in this paper, an end-to-end solution via deep reinforcement learning. A ConvNet-LSTM function approximator is adopted for the direct frame-to-action prediction. We further propose an environment augmentation technique and a customized reward function, which are crucial for successful training. The tracker trained in simulators (ViZDoom and Unreal Engine) demonstrates good generalization behaviors in the case of unseen object moving paths, unseen object appearances, unseen backgrounds, and distracting objects. The system is robust and can restore tracking after occasional lost of the target being tracked. We also find that the tracking ability, obtained solely from simulators, can potentially transfer to real-world scenarios. We demonstrate successful examples of such transfer, via experiments over the VOT dataset and the deployment of a real-world robot using the proposed active tracker trained in simulation.