Discovering Object-Centric Generalized Value Functions From Pixels
This addresses the challenge of extracting useful features from high-dimensional inputs without hand-crafted tasks, potentially benefiting reinforcement learning applications requiring adaptability.
The paper tackles the problem of automatically learning object-centric representations for control and fast adaptation in deep reinforcement learning, showing competitive performance in both stationary and non-stationary settings with interpretable and invariant learned representations.
Deep Reinforcement Learning has shown significant progress in extracting useful representations from high-dimensional inputs albeit using hand-crafted auxiliary tasks and pseudo rewards. Automatically learning such representations in an object-centric manner geared towards control and fast adaptation remains an open research problem. In this paper, we introduce a method that tries to discover meaningful features from objects, translating them to temporally coherent "question" functions and leveraging the subsequent learned general value functions for control. We compare our approach with state-of-the-art techniques alongside other ablations and show competitive performance in both stationary and non-stationary settings. Finally, we also investigate the discovered general value functions and through qualitative analysis show that the learned representations are not only interpretable but also, centered around objects that are invariant to changes across tasks facilitating fast adaptation.