Nguyen Ha Thanh

Nguyen Ha Thanh, Bui Minh Quan, Chau Nguyen et al.

We summarize the evaluation of the first Automated Legal Question Answering Competition (ALQAC 2021). The competition this year contains three tasks, which aims at processing the statute law document, which are Legal Text Information Retrieval (Task 1), Legal Text Entailment Prediction (Task 2), and Legal Text Question Answering (Task 3). The final goal of these tasks is to build a system that can automatically determine whether a particular statement is lawful. There is no limit to the approaches of the participating teams. This year, there are 5 teams participating in Task 1, 6 teams participating in Task 2, and 5 teams participating in Task 3. There are in total 36 runs submitted to the organizer. In this paper, we summarize each team's approaches, official results, and some discussion about the competition. Only results of the teams who successfully submit their approach description paper are reported in this paper.

Nguyen Ha Thanh

Deep learning is a powerful approach in recovering lost information as well as harder inverse function computation problems. When applied in natural language processing, this approach is essentially making use of context as a mean to recover information through likelihood maximization. Not long ago, a linguistic study called Tieq Viet was controversial among both researchers and society. We find this a great example to demonstrate the ability of deep learning models to recover lost information. In the proposal of Tieq Viet, some consonants in the standard Vietnamese are replaced. A sentence written in this proposal can be interpreted into multiple sentences in the standard version, with different meanings. The hypothesis that we want to test is whether a deep learning model can recover the lost information if we translate the text from Vietnamese to Tieq Viet.

Wachara Fungwacharakorn, Nguyen Ha Thanh, May Myo Zin et al.

This paper presents a novel approach termed Layer-of-Thoughts Prompting (LoT), which utilizes constraint hierarchies to filter and refine candidate responses to a given query. By integrating these constraints, our method enables a structured retrieval process that enhances explainability and automation. Existing methods have explored various prompting techniques but often present overly generalized frameworks without delving into the nuances of prompts in multi-turn interactions. Our work addresses this gap by focusing on the hierarchical relationships among prompts. We demonstrate that the efficacy of thought hierarchy plays a critical role in developing efficient and interpretable retrieval algorithms. Leveraging Large Language Models (LLMs), LoT significantly improves the accuracy and comprehensibility of information retrieval tasks.

Zhihan Cao, Fumihito Nishino, Hiroaki Yamada et al.

We introduce JBE-QA, a Japanese Bar Exam Question-Answering dataset to evaluate large language models' legal knowledge. Derived from the multiple-choice (tanto-shiki) section of the Japanese bar exam (2015-2024), JBE-QA provides the first comprehensive benchmark for Japanese legal-domain evaluation of LLMs. It covers the Civil Code, the Penal Code, and the Constitution, extending beyond the Civil Code focus of prior Japanese resources. Each question is decomposed into independent true/false judgments with structured contextual fields. The dataset contains 3,464 items with balanced labels. We evaluate 26 LLMs, including proprietary, open-weight, Japanese-specialised, and reasoning models. Our results show that proprietary models with reasoning enabled perform best, and the Constitution questions are generally easier than the Civil Code or the Penal Code questions.