LLM-AR: LLM-powered Automated Reasoning Framework
This addresses the need for interpretable and reliable AI in venture capital decision-making, though it is an incremental improvement combining existing methods.
The paper tackles the problem of unreliable LLM accuracy in high-stakes decision-making by developing LLM-AR, a framework that distills LLM-generated heuristics into probabilistic rules for predicting startup success, achieving 59.5% precision and 8.7% recall on unseen data.
Large language models (LLMs) can already identify patterns and reason effectively, yet their variable accuracy hampers adoption in high-stakes decision-making applications. In this paper, we study this issue from a venture capital perspective by predicting idea-stage startup success based on founder traits. (i) To build a reliable prediction model, we introduce LLM-AR, a pipeline inspired by neural-symbolic systems that distils LLM-generated heuristics into probabilistic rules executed by the ProbLog automated-reasoning engine. (ii) An iterative policy-evolution loop incorporates association-rule mining to progressively refine the prediction rules. On unseen folds, LLM-AR achieves 59.5% precision and 8.7% recall, 5.9x the random baseline precision, while exposing every decision path for human inspection. The framework is interpretable and tunable via hyperparameters, showing promise to extend into other domains.