SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
This benchmark addresses the gap in Arabic financial NLP for trustworthy finance and Islamic-finance assistants, providing a standardized evaluation resource.
The paper introduces SAHM, a benchmark and instruction-tuning dataset for Arabic financial NLP and Shari'ah-compliant reasoning, containing 14,380 expert-verified instances across seven tasks. Evaluation of 19 LLMs shows that Arabic fluency does not reliably translate to evidence-grounded financial reasoning, with models performing better on recognition tasks than on generation and causal reasoning.
English financial NLP has progressed rapidly through benchmarks for sentiment, document understanding, and financial question answering, while Arabic financial NLP remains comparatively under-explored despite strong practical demand for trustworthy finance and Islamic-finance assistants. We introduce SAHM, a document-grounded benchmark and instruction-tuning dataset for Arabic financial NLP and Shari'ah-compliant reasoning. SAHM contains 14,380 expert-verified instances spanning seven tasks: AAOIFI standards QA, fatwa-based QA/MCQ, accounting and business exams, financial sentiment analysis, extractive summarization, and event-cause reasoning, curated from authentic regulatory, juristic, and corporate sources. We evaluate 19 strong open and proprietary LLMs using task-specific metrics and rubric-based scoring for open-ended outputs, and find that Arabic fluency does not reliably translate to evidence-grounded financial reasoning: models are substantially stronger on recognition-style tasks than on generation and causal reasoning, with the largest gaps on event-cause reasoning. We release the benchmark, evaluation framework, and an instruction-tuned model to support future research on trustworthy Arabic financial NLP.