Query-driven Relevant Paragraph Extraction from Legal Judgments
This work addresses a domain-specific problem for legal professionals by providing incremental improvements in retrieval methods for legal texts.
The paper tackled the problem of extracting relevant paragraphs from lengthy legal judgments based on queries, constructing a dataset from the European Court of Human Rights and evaluating retrieval models, finding a significant performance gap between fine-tuned and zero-shot approaches, with fine-tuned models showing improvements but still struggling with unseen legal queries.
Legal professionals often grapple with navigating lengthy legal judgements to pinpoint information that directly address their queries. This paper focus on this task of extracting relevant paragraphs from legal judgements based on the query. We construct a specialized dataset for this task from the European Court of Human Rights (ECtHR) using the case law guides. We assess the performance of current retrieval models in a zero-shot way and also establish fine-tuning benchmarks using various models. The results highlight the significant gap between fine-tuned and zero-shot performance, emphasizing the challenge of handling distribution shift in the legal domain. We notice that the legal pre-training handles distribution shift on the corpus side but still struggles on query side distribution shift, with unseen legal queries. We also explore various Parameter Efficient Fine-Tuning (PEFT) methods to evaluate their practicality within the context of information retrieval, shedding light on the effectiveness of different PEFT methods across diverse configurations with pre-training and model architectures influencing the choice of PEFT method.