CEQuest: Benchmarking Large Language Models for Construction Estimation
This work addresses the need for domain-specific evaluation in construction, but it is incremental as it primarily benchmarks existing models without proposing new methods.
The paper tackles the problem of evaluating large language models (LLMs) in the specialized field of construction, introducing the CEQuest benchmark dataset for construction drawing interpretation and estimation, and finds that current LLMs show significant room for improvement in accuracy, execution time, and model size.
Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of general-domain tasks. However, their effectiveness in specialized fields, such as construction, remains underexplored. In this paper, we introduce CEQuest, a novel benchmark dataset specifically designed to evaluate the performance of LLMs in answering construction-related questions, particularly in the areas of construction drawing interpretation and estimation. We conduct comprehensive experiments using five state-of-the-art LLMs, including Gemma 3, Phi4, LLaVA, Llama 3.3, and GPT-4.1, and evaluate their performance in terms of accuracy, execution time, and model size. Our experimental results demonstrate that current LLMs exhibit considerable room for improvement, highlighting the importance of integrating domain-specific knowledge into these models. To facilitate further research, we will open-source the proposed CEQuest dataset, aiming to foster the development of specialized large language models (LLMs) tailored to the construction domain.