Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering
This study addresses road safety by improving explainable QA for less-resourced languages like Romanian, though it is incremental as it applies existing methods to new data.
The paper tackled the problem of evaluating Large Language Models (LLMs) on Romanian driving-law question answering with explanation generation, finding that fine-tuned 8B models are competitive with SOTA systems and textual descriptions of images outperform direct visual input.
Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages.