The Temporal Trap: Entanglement in Pre-Trained Visual Representations for Visuomotor Policy Learning
This addresses a critical bottleneck for researchers and practitioners in robotics and AI by improving the robustness of visuomotor policies, though it is incremental as it builds on existing pre-trained models.
The paper tackled the problem of temporal entanglement in pre-trained visual representations for visuomotor policy learning, showing a strong correlation between policy success rates and latent space ability to capture task-progression cues, and proposed a disentanglement baseline that mitigates this issue.
The integration of pre-trained visual representations (PVRs) has significantly advanced visuomotor policy learning. However, effectively leveraging these models remains a challenge. We identify temporal entanglement as a critical, inherent issue when using these time-invariant models in sequential decision-making tasks. This entanglement arises because PVRs, optimised for static image understanding, struggle to represent the temporal dependencies crucial for visuomotor control. In this work, we quantify the impact of temporal entanglement, demonstrating a strong correlation between a policy's success rate and the ability of its latent space to capture task-progression cues. Based on these insights, we propose a simple, yet effective disentanglement baseline designed to mitigate temporal entanglement. Our empirical results show that traditional methods aimed at enriching features with temporal components are insufficient on their own, highlighting the necessity of explicitly addressing temporal disentanglement for robust visuomotor policy learning.