Bilingual BSARD: Extending Statutory Article Retrieval to Dutch
This work addresses legal information retrieval for bilingual countries like Belgium, but it is incremental as it builds on an existing dataset and focuses on benchmarking.
The authors tackled the challenge of statutory article retrieval in multilingual legal contexts by extending the BSARD dataset to include Dutch, creating bBSARD. They benchmarked retrieval models and found that BM25 remains competitive with zero-shot dense models, and fine-tuned small models can match or surpass proprietary ones.
Statutory article retrieval plays a crucial role in making legal information more accessible to both laypeople and legal professionals. Multilingual countries like Belgium present unique challenges for retrieval models due to the need for handling legal issues in multiple languages. Building on the Belgian Statutory Article Retrieval Dataset (BSARD) in French, we introduce the bilingual version of this dataset, bBSARD. The dataset contains parallel Belgian statutory articles in both French and Dutch, along with legal questions from BSARD and their Dutch translation. Using bBSARD, we conduct extensive benchmarking of retrieval models available for Dutch and French. Our benchmarking setup includes lexical models, zero-shot dense models, and fine-tuned small foundation models. Our experiments show that BM25 remains a competitive baseline compared to many zero-shot dense models in both languages. We also observe that while proprietary models outperform open alternatives in the zero-shot setting, they can be matched or surpassed by fine-tuning small language-specific models. Our dataset and evaluation code are publicly available.