Optimizing Legal Document Retrieval in Vietnamese with Semi-Hard Negative Mining
This work addresses the challenge of precision in legal document retrieval for specialized domains, though it is incremental as it builds on existing retrieval methods with optimizations.
The paper tackled the problem of legal document retrieval in Vietnamese by introducing a two-stage retrieval and re-ranking framework with semi-hard negative mining, achieving a top-three position in the SoICT Hackathon 2024 with a lightweight, competitive approach.
Large Language Models (LLMs) face significant challenges in specialized domains like law, where precision and domain-specific knowledge are critical. This paper presents a streamlined two-stage framework consisting of Retrieval and Re-ranking to enhance legal document retrieval efficiency and accuracy. Our approach employs a fine-tuned Bi-Encoder for rapid candidate retrieval, followed by a Cross-Encoder for precise re-ranking, both optimized through strategic negative example mining. Key innovations include the introduction of the Exist@m metric to evaluate retrieval effectiveness and the use of semi-hard negatives to mitigate training bias, which significantly improved re-ranking performance. Evaluated on the SoICT Hackathon 2024 for Legal Document Retrieval, our team, 4Huiter, achieved a top-three position. While top-performing teams employed ensemble models and iterative self-training on large bge-m3 architectures, our lightweight, single-pass approach offered a competitive alternative with far fewer parameters. The framework demonstrates that optimized data processing, tailored loss functions, and balanced negative sampling are pivotal for building robust retrieval-augmented systems in legal contexts.