Tracking as Online Decision-Making: Learning a Policy from Streaming Videos with Reinforcement Learning
This addresses the problem of robust object tracking in videos for computer vision applications, presenting a novel approach but with incremental improvements in method.
The paper tackles object tracking by formulating it as an online decision-making process using a POMDP and deep reinforcement learning, achieving training on datasets several orders of magnitude larger than past work with sparse rewards.
We formulate tracking as an online decision-making process, where a tracking agent must follow an object despite ambiguous image frames and a limited computational budget. Crucially, the agent must decide where to look in the upcoming frames, when to reinitialize because it believes the target has been lost, and when to update its appearance model for the tracked object. Such decisions are typically made heuristically. Instead, we propose to learn an optimal decision-making policy by formulating tracking as a partially observable decision-making process (POMDP). We learn policies with deep reinforcement learning algorithms that need supervision (a reward signal) only when the track has gone awry. We demonstrate that sparse rewards allow us to quickly train on massive datasets, several orders of magnitude more than past work. Interestingly, by treating the data source of Internet videos as unlimited streams, we both learn and evaluate our trackers in a single, unified computational stream.