Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis
This addresses the challenge for financial analysts in processing long 10-K reports, representing an incremental improvement through refined retrieval strategies.
This paper tackled the problem of extracting information from lengthy financial reports by developing a Retrieval-Augmented Generation system for question-answering, finding that neural reranking significantly improved answer correctness from 33.5% to 49.0% and reduced error rates from 35.3% to 22.5%.
Financial analysts face significant challenges extracting information from lengthy 10-K reports, which often exceed 100 pages. This paper presents a Retrieval-Augmented Generation (RAG) system designed to answer questions about S&P 500 financial reports and evaluates the impact of neural reranking on system performance. Our pipeline employs hybrid search combining full-text and semantic retrieval, followed by an optional reranking stage using a cross-encoder model. We conduct systematic evaluation using the FinDER benchmark dataset, comprising 1,500 queries across five experimental groups. Results demonstrate that reranking significantly improves answer quality, achieving 49.0 percent correctness for scores of 8 or above compared to 33.5 percent without reranking, representing a 15.5 percentage point improvement. Additionally, the error rate for completely incorrect answers decreases from 35.3 percent to 22.5 percent. Our findings emphasize the critical role of reranking in financial RAG systems and demonstrate performance improvements over baseline methods through modern language models and refined retrieval strategies.