Towards Explainability in Legal Outcome Prediction Models
This addresses the need for explainability in legal NLP to enable real-world use by legal practitioners, though it is incremental in improving understanding rather than solving the core prediction task.
The paper tackles the problem of explainability in legal outcome prediction models by proposing a novel method to identify the precedent used by these models, and finds that while models predict outcomes reasonably well, their use of precedent differs from human judges.
Current legal outcome prediction models - a staple of legal NLP - do not explain their reasoning. However, to employ these models in the real world, human legal actors need to be able to understand the model's decisions. In the case of common law, legal practitioners reason towards the outcome of a case by referring to past case law, known as precedent. We contend that precedent is, therefore, a natural way of facilitating explainability for legal NLP models. In this paper, we contribute a novel method for identifying the precedent employed by legal outcome prediction models. Furthermore, by developing a taxonomy of legal precedent, we are able to compare human judges and neural models with respect to the different types of precedent they rely on. We find that while the models learn to predict outcomes reasonably well, their use of precedent is unlike that of human judges.