FACTS: A Factored State-Space Framework For World Modelling
This addresses limitations in current frameworks like Transformers and Mambas for long-term high-dimensional sequence modelling, offering a general-purpose solution with broad applicability.
The paper tackles the problem of efficiently encoding spatial and temporal structures in world modelling for complex systems, proposing the FACTS framework, which outperforms or matches state-of-the-art models in tasks like multivariate time series forecasting and spatial-temporal graph prediction.
World modelling is essential for understanding and predicting the dynamics of complex systems by learning both spatial and temporal dependencies. However, current frameworks, such as Transformers and selective state-space models like Mambas, exhibit limitations in efficiently encoding spatial and temporal structures, particularly in scenarios requiring long-term high-dimensional sequence modelling. To address these issues, we propose a novel recurrent framework, the \textbf{FACT}ored \textbf{S}tate-space (\textbf{FACTS}) model, for spatial-temporal world modelling. The FACTS framework constructs a graph-structured memory with a routing mechanism that learns permutable memory representations, ensuring invariance to input permutations while adapting through selective state-space propagation. Furthermore, FACTS supports parallel computation of high-dimensional sequences. We empirically evaluate FACTS across diverse tasks, including multivariate time series forecasting, object-centric world modelling, and spatial-temporal graph prediction, demonstrating that it consistently outperforms or matches specialised state-of-the-art models, despite its general-purpose world modelling design.