GeoAlign: Geometric Feature Realignment for MLLM Spatial Reasoning
For MLLM researchers, this provides a method to improve spatial reasoning without scaling model size, addressing a known bottleneck in geometric feature utilization.
GeoAlign addresses the misalignment between geometric features from 3D foundation models and the spatial reasoning needs of MLLMs by dynamically aggregating multi-layer features. It achieves state-of-the-art performance on VSI-Bench, ScanQA, and SQA3D with a compact 4B model, outperforming larger MLLMs.
Multimodal large language models (MLLMs) have exhibited remarkable performance in various visual tasks, yet still struggle with spatial reasoning. Recent efforts mitigate this by injecting geometric features from 3D foundation models, but rely on static single-layer extractions. We identify that such an approach induces a task misalignment bias: the geometric features naturally evolve towards 3D pretraining objectives, which may contradict the heterogeneous spatial demands of MLLMs, rendering any single layer fundamentally insufficient. To resolve this, we propose GeoAlign, a novel framework that dynamically aggregates multi-layer geometric features to realign with the actual demands. GeoAlign constructs a hierarchical geometric feature bank and leverages the MLLM's original visual tokens as content-aware queries to perform layer-wise sparse routing, adaptively fetching the suitable geometric features for each patch. Extensive experiments on VSI-Bench, ScanQA, and SQA3D demonstrate that our compact 4B model effectively achieves state-of-the-art performance, even outperforming larger existing MLLMs.