Learning reusable concepts across different egocentric video understanding tasks
This addresses the challenge of correlating concepts and leveraging task synergies for robots in video understanding, but appears incremental as it builds on existing multitask learning approaches.
The paper tackles the problem of enabling holistic perception in autonomous systems by learning reusable concepts across different egocentric video understanding tasks, introducing Hier-EgoPack as a unified framework that creates a collection of task perspectives for downstream use.
Our comprehension of video streams depicting human activities is naturally multifaceted: in just a few moments, we can grasp what is happening, identify the relevance and interactions of objects in the scene, and forecast what will happen soon, everything all at once. To endow autonomous systems with such holistic perception, learning how to correlate concepts, abstract knowledge across diverse tasks, and leverage tasks synergies when learning novel skills is essential. In this paper, we introduce Hier-EgoPack, a unified framework able to create a collection of task perspectives that can be carried across downstream tasks and used as a potential source of additional insights, as a backpack of skills that a robot can carry around and use when needed.