Gold-Guided Programmatic Distillation for Financial Reasoning over Hybrid Tables and Text
For practitioners needing reliable numerical reasoning in finance, this provides a method to train smaller models that surpass larger ones, but the approach is incremental as it builds on programmatic distillation.
The paper tackles financial question answering over hybrid tables and text, where LLMs struggle with arithmetic errors. Using execution-verified Python programs for distillation, a 7B student model achieves 87.00 EM / 87.18 F1 on TAT-QA, outperforming the 72B teacher (78.46 EM) and strong baselines.
Financial question answering over hybrid tabular and textual data may require multi-source reasoning and precise numerical computation. While large language models (LLMs) can generate intermediate reasoning steps, natural-language rationales remain prone to arithmetic errors, making them an unreliable supervision source for distillation. Building on programmatic distillation, we develop an approach that transfers reliable numerical reasoning from a large teacher model to a compact student using execution-verified Python programs instead of free-form textual rationales. It leverages gold derivations to guide teacher-side program synthesis and retains only programs that execute correctly and produce the gold answer, ensuring high-quality supervision. We further introduce an iterative recovery stage that revisits teacher-failed examples, enabling the student to recover and incorporate newly verified programs into training. Experiments on TAT-QA show that our framework is highly effective for hybrid financial reasoning. Our best 7B student achieves 87.00 EM / 87.18 F1 on the test set, substantially outperforming the 72B teacher (78.46 EM) as well as traditional and strong LLM-based baselines, including TAGOP and TAT-LLM. These results demonstrate that execution-verified programmatic distillation provides an effective and extensible framework for training smaller models to perform reliable numerical reasoning.