Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models
This addresses the tedious and data-scarce process of financial auditing for auditors, though it is incremental as it builds on existing transformer and LLM techniques.
The authors tackled the problem of matching financial documents to legal requirements without fine-tuning by proposing ZeroShotALI, a two-step system combining a custom BERT-based model and an LLM, which achieved significant performance improvements over existing methods.
Auditing financial documents is a very tedious and time-consuming process. As of today, it can already be simplified by employing AI-based solutions to recommend relevant text passages from a report for each legal requirement of rigorous accounting standards. However, these methods need to be fine-tuned regularly, and they require abundant annotated data, which is often lacking in industrial environments. Hence, we present ZeroShotALI, a novel recommender system that leverages a state-of-the-art large language model (LLM) in conjunction with a domain-specifically optimized transformer-based text-matching solution. We find that a two-step approach of first retrieving a number of best matching document sections per legal requirement with a custom BERT-based model and second filtering these selections using an LLM yields significant performance improvements over existing approaches.