Exploiting LLMs' Reasoning Capability to Infer Implicit Concepts in Legal Information Retrieval
This addresses limitations in legal retrieval systems for handling real-life queries, offering a domain-specific improvement.
The paper tackled the problem of legal information retrieval by using large language models' reasoning capabilities to infer implicit concepts, improving retrieval accuracy on COLIEE datasets and outperforming all participating teams in the 2022 and 2023 competitions.
Statutory law retrieval is a typical problem in legal language processing, that has various practical applications in law engineering. Modern deep learning-based retrieval methods have achieved significant results for this problem. However, retrieval systems relying on semantic and lexical correlations often exhibit limitations, particularly when handling queries that involve real-life scenarios, or use the vocabulary that is not specific to the legal domain. In this work, we focus on overcoming this weaknesses by utilizing the logical reasoning capabilities of large language models (LLMs) to identify relevant legal terms and facts related to the situation mentioned in the query. The proposed retrieval system integrates additional information from the term--based expansion and query reformulation to improve the retrieval accuracy. The experiments on COLIEE 2022 and COLIEE 2023 datasets show that extra knowledge from LLMs helps to improve the retrieval result of both lexical and semantic ranking models. The final ensemble retrieval system outperformed the highest results among all participating teams in the COLIEE 2022 and 2023 competitions.