Towards Trustworthy Legal AI through LLM Agents and Formal Reasoning
This addresses the need for trustworthy AI in legal decision-making by combining natural language flexibility with symbolic rigor, though it is incremental as it builds on existing LLM and formal reasoning methods.
The paper tackles the problem of making legal AI trustworthy by ensuring verifiable justifications, presenting L4L, a framework that integrates LLM agents with formal reasoning to enforce alignment with statutory laws, and experiments show it outperforms baselines on legal benchmarks while providing auditable justifications.
Legal decisions should be logical and based on statutory laws. While large language models(LLMs) are good at understanding legal text, they cannot provide verifiable justifications. We present L4L, a solver-centric framework that enforces formal alignment between LLM-based legal reasoning and statutory laws. The framework integrates role-differentiated LLM agents with SMT-backed verification, combining the flexibility of natural language with the rigor of symbolic reasoning. Our approach operates in four stages: (1) Statute Knowledge Building, where LLMs autoformalize legal provisions into logical constraints and validate them through case-level testing; (2) Dual Fact-and-Statute Extraction, in which the prosecutor-and defense-aligned agents independently map case narratives to argument tuples; (3) Solver-Centric Adjudication, where SMT solvers check the legal admissibility and consistency of the arguments against the formalized statute knowledge; (4) Judicial Rendering, in which a judge agent integrates solver-validated reasoning with statutory interpretation and similar precedents to produce a legally grounded verdict. Experiments on public legal benchmarks show that L4L consistently outperforms baselines, while providing auditable justifications that enable trustworthy legal AI.