Adaptive Cost-Efficient Evaluation for Reliable Patent Claim Validation
This addresses the cost-efficiency and reliability challenge in patent validation for legal and AI applications, representing an incremental improvement over existing methods.
The paper tackles the problem of automated patent claim validation, which requires zero-defect tolerance, by proposing ACE, a hybrid framework that routes high-uncertainty claims to an expert LLM using predictive entropy, achieving an F1 score of 94.95% and reducing operational costs by 78% compared to standalone LLM deployments.
Automated validation of patent claims demands zero-defect tolerance, as even a single structural flaw can render a claim legally defective. Existing evaluation paradigms suffer from a rigidity-resource dilemma: lightweight encoders struggle with nuanced legal dependencies, while exhaustive verification via Large Language Models (LLMs) is prohibitively costly. To bridge this gap, we propose ACE (Adaptive Cost-efficient Evaluation), a hybrid framework that uses predictive entropy to route only high-uncertainty claims to an expert LLM. The expert then executes a Chain of Patent Thought (CoPT) protocol grounded in 35 U.S.C. statutory standards. This design enables ACE to handle long-range legal dependencies more effectively while preserving efficiency. ACE achieves the best F1 among the evaluated methods at 94.95\%, while reducing operational costs by 78\% compared to standalone LLM deployments. We also construct ACE-40k, a 40,000-claim benchmark with MPEP-grounded error annotations, to facilitate further research.