From Citations to Criticality: Predicting Legal Decision Influence in the Multilingual Swiss Jurisprudence
This work addresses the need for efficient case triage in overwhelmed court systems, though it is incremental as it builds on existing multilingual models with a new dataset.
The paper tackles the problem of court case prioritization by introducing the Criticality Prediction dataset, which uses algorithmic labeling to create a large resource for evaluating case influence, and finds that fine-tuned models outperform large language models in this domain-specific task.
Many court systems are overwhelmed all over the world, leading to huge backlogs of pending cases. Effective triage systems, like those in emergency rooms, could ensure proper prioritization of open cases, optimizing time and resource allocation in the court system. In this work, we introduce the Criticality Prediction dataset, a novel resource for evaluating case prioritization. Our dataset features a two-tier labeling system: (1) the binary LD-Label, identifying cases published as Leading Decisions (LD), and (2) the more granular Citation-Label, ranking cases by their citation frequency and recency, allowing for a more nuanced evaluation. Unlike existing approaches that rely on resource-intensive manual annotations, we algorithmically derive labels leading to a much larger dataset than otherwise possible. We evaluate several multilingual models, including both smaller fine-tuned models and large language models in a zero-shot setting. Our results show that the fine-tuned models consistently outperform their larger counterparts, thanks to our large training set. Our results highlight that for highly domain-specific tasks like ours, large training sets are still valuable.