Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline
For transportation engineering practitioners, this provides a reproducible framework to adapt general LLMs to domain-specific technical content, addressing the limitation of insufficient exposure to specialized terminology and standards.
This study proposes a systematic approach to develop a customized generative AI agent for transportation engineering by continued pre-training of six LLMs using LoRA on a curated corpus of U.S. transportation manuals. Qwen2.5-7B and LLaMA-3.1-8B achieved the highest domain alignment and response quality, validated by BLEU-4 and ROUGE metrics.
Recent advancements in generative artificial intelligence (AI) and large language models (LLMs) have shown significant promise in automating complex reasoning, summarization, and question-answering tasks. However, the effectiveness of general-purpose LLMs in specialized engineering domains remains limited due to insufficient exposure to technical standards, engineering terminology, and domain-specific semantics. This study proposes a systematic approach to developing a customized generative AI agent for transportation engineering applications. A curated corpus of U.S. transportation manuals, design guidelines, and regulatory documents is used to conduct continued pretraining of six state-of-the-art LLMs through a unified low-rank adaptation (LoRA) framework. The training process is monitored to ensure convergence and model stability. Performance is evaluated using standard natural language processing metrics, including BLEU-4 and ROUGE, with Qwen2.5-7B and LLaMA-3.1-8B demonstrating the highest domain alignment and response quality. Results validate the effectiveness of LoRA-based adaptation in improving LLM performance on technical content interpretation and context-specific reasoning. This work contributes a reproducible development framework for constructing domain-specialized generative AI agents, supporting broader deployment in transportation research, design, planning, and policy analysis.