Chameleon: Episodic Memory for Long-Horizon Robotic Manipulation
This addresses memory limitations for robots in perceptually confusable environments, representing a novel method rather than an incremental improvement.
The paper tackles the problem of perceptual aliasing in robotic manipulation by introducing Chameleon, a memory system that uses geometry-grounded multimodal tokens and differentiable recall, resulting in improved decision reliability and long-horizon control across tasks.
Robotic manipulation often requires memory: occlusion and state changes can make decision-time observations perceptually aliased, making action selection non-Markovian at the observation level because the same observation may arise from different interaction histories. Most embodied agents implement memory via semantically compressed traces and similarity-based retrieval, which discards disambiguating fine-grained perceptual cues and can return perceptually similar but decision-irrelevant episodes. Inspired by human episodic memory, we propose Chameleon, which writes geometry-grounded multimodal tokens to preserve disambiguating context and produces goal-directed recall through a differentiable memory stack. We also introduce Camo-Dataset, a real-robot UR5e dataset spanning episodic recall, spatial tracking, and sequential manipulation under perceptual aliasing. Across tasks, Chameleon consistently improves decision reliability and long-horizon control over strong baselines in perceptually confusable settings.