Enhancing Vision-Based Policies with Omni-View and Cross-Modality Knowledge Distillation for Mobile Robots
This work addresses practical deployment challenges for mobile robots, though it is incremental as it builds on existing knowledge distillation techniques.
The paper tackles the trilemma of limited scene transferability, restricted computation, and sensor costs in vision-based policies for lightweight mobile robots by proposing a knowledge distillation method that transfers knowledge from an omni-view depth policy to a monocular policy, improving navigation performance and scene transfer.
Vision-based policies are widely applied in robotics for tasks such as manipulation and locomotion. On lightweight mobile robots, however, they face a trilemma of limited scene transferability, restricted onboard computation resources, and sensor hardware cost. To address these issues, we propose a knowledge distillation approach that transfers knowledge from an information-rich, appearance invariant omniview depth policy to a lightweight monocular policy. The key idea is to train the student not only to mimic the expert actions but also to align with the latent embeddings of the omni view depth teacher. Experiments demonstrate that omni-view and depth inputs improve the scene transfer and navigation performance, and that the proposed distillation method enhances the performance of a singleview monocular policy, compared with policies solely imitating actions. Real world experiments further validate the effectiveness and practicality of our approach. Code will be released publicly.