Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generation
This addresses the challenge of unreliable automated modeling in computational engineering, offering an incremental improvement for structural engineering applications.
The paper tackles the problem of generating physically consistent and simulation-executable structural modeling code using large language models, by introducing a framework with domain-specific data and fine-tuning, resulting in substantial reductions in non-conforming outputs as validated by a new benchmark.
Structural modeling is a fundamental component of computational engineering science, in which even minor physical inconsistencies or specification violations may invalidate downstream simulations. The potential of large language models (LLMs) for automatic generation of modeling code has been demonstrated. However, non-executable or physically inconsistent outputs remain prevalent under stringent engineering constraints. A framework for physics-consistent automatic building modeling is therefore proposed, integrating domain knowledge construction, constraint-oriented model alignment, and verification-driven evaluation. CivilInstruct is introduced as a domain-specific dataset that formalizes structural engineering knowledge and constraint reasoning to enable simulation-ready model generation. A two-stage fine-tuning strategy is further employed to enforce constraint satisfaction and application programming interface compliance, substantially reducing hallucinated and non-conforming outputs. MBEval is presented as a verification-driven benchmark that evaluates executability and structural dynamics consistency through closed-loop validation. Experimental results show consistent improvements over baselines across rigorous verification metrics. Our code is available at https://github.com/Jovanqing/AutoBM.