NativeMEM: Native Memory Compression for Long-Horizon Robotic Manipulation
For robotic manipulation tasks requiring long-horizon visual memory, NativeMEM provides a practical solution that significantly improves success rates without sacrificing inference speed.
NativeMEM introduces a memory compression scheme that repurposes a VLA model's own vision encoder to compress historical frames into single tokens, enabling long-horizon memory with low latency. It boosts success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining efficiency.
How can pretrained Vision-Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external memory management, which restrains either the memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated memory. At its core is an efficient memory encoding scheme, Native Memory Compression, which repurposes the VLA's own vision encoder to compress each historical frame from each camera view into a single token. Appended to the input sequence, these memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized memory module. To align the memory tokens with the pretrained policy, we first develop a generic memory tokenizer under the supervision of a frozen VLA on memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining low inference latency and GPU memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.