Sample-Efficient Post-Training for LEGO Spatial-Physics Reasoning
For researchers in physical reasoning and embodied AI, this work addresses a critical failure mode in LLM-based generation, showing that physical validity alone is insufficient and proposing a method to improve semantic and geometric fidelity.
LLM-based LEGO assembly generation suffers from PhysHack, where physically valid structures are geometrically misaligned or semantically inconsistent. The authors propose PVPO, a sample-efficient reinforcement learning method using model-based data selection, which improves structural and semantic alignment, physical validity, and calibration while reducing reliance on rejection sampling.
LLM-based LEGO assembly generation requires both semantic grounding and physical feasibility. We identify a data-induced failure mode, PhysHack, in which the assemblies satisfy physical-validity constraints while producing structures that are geometrically misaligned, semantically inconsistent, or poorly calibrated. To address this challenge, we propose a model-based data selection approach that uses only a small fraction of the training data while improving physically grounded LEGO assembly generation. Building on the selected trajectories, we introduce PVPO, a sample-efficient reinforcement learning method that couples physical feasibility with voxel-space geometric rewards. Our results show that physical validity alone is an insufficient proxy for reliable physical reasoning: models can learn to generate valid structures without preserving semantic or geometric fidelity. Experiments across model backbones and test-time scaling settings demonstrate that PVPO improves structural and semantic alignment, physical validity, structural stability, and calibration, while reducing reliance on extensive post-hoc rejection sampling. In particular, results on calibration show that PVPO mitigates PhysHack by making test-time selection more predictive of semantic and structural quality.