Spatial Reasoning with Denoising Models
This addresses hallucination issues in generative models for spatial domains, offering a novel framework with significant performance gains, though it is incremental in improving existing denoising methods.
The paper tackles the problem of hallucination in generative models for spatial reasoning by introducing Spatial Reasoning Models (SRMs), which use denoising to infer continuous variables, and reports an accuracy increase from <1% to >50% on specific tasks.
We introduce Spatial Reasoning Models (SRMs), a framework to perform reasoning over sets of continuous variables via denoising generative models. SRMs infer continuous representations on a set of unobserved variables, given observations on observed variables. Current generative models on spatial domains, such as diffusion and flow matching models, often collapse to hallucination in case of complex distributions. To measure this, we introduce a set of benchmark tasks that test the quality of complex reasoning in generative models and can quantify hallucination. The SRM framework allows to report key findings about importance of sequentialization in generation, the associated order, as well as the sampling strategies during training. It demonstrates, for the first time, that order of generation can successfully be predicted by the denoising network itself. Using these findings, we can increase the accuracy of specific reasoning tasks from <1% to >50%. Our project website provides additional videos, code, and the benchmark datasets: https://geometric-rl.mpi-inf.mpg.de/srm