MemoryWAM: Efficient World Action Modeling with Persistent Memory
For robotic manipulation, MemoryWAM addresses the trade-off between memory and efficiency in world action models, enabling robust performance in non-Markovian environments without prohibitive computational costs.
MemoryWAM introduces a world action model with persistent memory that combines recent frames, event-boundary anchors, and compact gist tokens to efficiently handle long-horizon, memory-dependent robotic manipulation tasks, outperforming VLA and WAM baselines while reducing inference latency and GPU memory usage.
Robust robotic manipulation in the real world requires not only an understanding of the current observation, but also memory and dynamics modeling. World action models (WAMs) possess these capabilities by jointly modeling visual foresight and actions conditioned on both current and historical observations, making them a promising paradigm for robotic manipulation. However, existing WAMs face a fundamental trade-off: methods with efficient inference typically condition only on a bounded window of recent observations and therefore struggle in non-Markovian environments, whereas methods that preserve long histories incur time and space costs that grow substantially with sequence length. To address this challenge, we introduce MemoryWAM, a world action model with efficient persistent memory. MemoryWAM uses a hybrid memory design that combines recent frames, event-boundary anchor frames, and compact gist tokens that summarize long-range history. A tailored attention mechanism enables retrieval of both detailed short-term context and compressed long-term context, supporting memory-dependent decision-making with reduced inference latency and GPU memory usage. Across long-horizon, memory-dependent manipulation tasks in both simulation and the real world, MemoryWAM outperforms strong vision-language-action (VLA) and WAM baselines while maintaining favorable computational efficiency.