Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law
This provides a cost-effective alternative to manual annotation for adapting LLMs to specialized domains like German law.
The paper tackles the problem of large language models struggling with legal reasoning by developing a synthetic data generation method that produces high-quality German legal question-answer pairs from statutes, resulting in LLMs that significantly outperform baselines on German legal QA tasks.
Large language models (LLMs) often struggle in specialized domains such as legal reasoning due to limited expert knowledge, resulting in factually incorrect outputs or hallucinations. This paper presents an effective method for adapting advanced LLMs to German legal question answering through a novel synthetic data generation approach. In contrast to costly human-annotated resources or unreliable synthetic alternatives, our approach systematically produces high-quality, diverse, and legally accurate question-answer pairs directly from authoritative German statutes. Using rigorous automated filtering methods and parameter-efficient fine-tuning techniques, we demonstrate that LLMs adapted with our synthetic dataset significantly outperform their baseline counterparts on German legal question answering tasks. Our results highlight the feasibility of using carefully designed synthetic data as a robust alternative to manual annotation in high-stakes, knowledge-intensive domains.