GAIus: Combining Genai with Legal Clauses Retrieval for Knowledge-based Assistant
This work addresses the need for accurate and explainable legal assistants in underrepresented languages, though it is incremental as it builds on existing retrieval and LLM methods.
The paper tackles the problem of improving legal information retrieval for non-English and non-Chinese contexts by introducing gAIus, an LLM-based agent that retrieves knowledge from the Polish Civil Code, resulting in a 419% improvement over GPT-3.5-turbo-0125 and boosting GPT-4o-mini's score from 31% to 86% on a Polish legal exam dataset.
In this paper we discuss the capability of large language models to base their answer and provide proper references when dealing with legal matters of non-english and non-chinese speaking country. We discuss the history of legal information retrieval, the difference between case law and statute law, its impact on the legal tasks and analyze the latest research in this field. Basing on that background we introduce gAIus, the architecture of the cognitive LLM-based agent, whose responses are based on the knowledge retrieved from certain legal act, which is Polish Civil Code. We propose a retrieval mechanism which is more explainable, human-friendly and achieves better results than embedding-based approaches. To evaluate our method we create special dataset based on single-choice questions from entrance exams for law apprenticeships conducted in Poland. The proposed architecture critically leveraged the abilities of used large language models, improving the gpt-3.5-turbo-0125 by 419%, allowing it to beat gpt-4o and lifting gpt-4o-mini score from 31% to 86%. At the end of our paper we show the possible future path of research and potential applications of our findings.