The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunal
It provides a benchmark for employment-related dispute resolution, addressing access to justice through technological innovation, but is incremental as it applies existing methods to a new dataset.
This paper tackles the problem of predicting case outcomes in the UK Employment Tribunal by creating the CLC-UKET dataset of approximately 19,000 cases with automatic LLM annotations, and finds that finetuned transformer models outperform zero-shot and few-shot LLMs on this task.
This paper explores the intersection of technological innovation and access to justice by developing a benchmark for predicting case outcomes in the UK Employment Tribunal (UKET). To address the challenge of extensive manual annotation, the study employs a large language model (LLM) for automatic annotation, resulting in the creation of the CLC-UKET dataset. The dataset consists of approximately 19,000 UKET cases and their metadata. Comprehensive legal annotations cover facts, claims, precedent references, statutory references, case outcomes, reasons and jurisdiction codes. Facilitated by the CLC-UKET data, we examine a multi-class case outcome prediction task in the UKET. Human predictions are collected to establish a performance reference for model comparison. Empirical results from baseline models indicate that finetuned transformer models outperform zero-shot and few-shot LLMs on the UKET prediction task. The performance of zero-shot LLMs can be enhanced by integrating task-related information into few-shot examples. We hope that the CLC-UKET dataset, along with human annotations and empirical findings, can serve as a valuable benchmark for employment-related dispute resolution.