UPAR: A Kantian-Inspired Prompting Framework for Enhancing Large Language Model Capabilities
This work addresses the problem of enhancing explainability and accuracy in LLM inference for researchers and practitioners, offering a novel framework that integrates existing methods, though it is incremental in building on prior prompting techniques.
The paper tackles the lack of a unified epistemological foundation for prompting large language models by proposing UPAR, a Kantian-inspired framework that structures inference into Understand, Plan, Act, and Reflect phases, resulting in accuracy improvements from 22.92% to 58.33% on a GSM8K subset and from 67.91% to 75.40% on a causal judgment task.
Large Language Models (LLMs) have demonstrated impressive inferential capabilities, with numerous research endeavors devoted to enhancing this capacity through prompting. Despite these efforts, a unified epistemological foundation is still conspicuously absent. Drawing inspiration from Kant's a priori philosophy, we propose the UPAR prompting framework, designed to emulate the structure of human cognition within LLMs. The UPAR framework is delineated into four phases: "Understand", "Plan", "Act", and "Reflect", enabling the extraction of structured information from complex contexts, prior planning of solutions, execution according to plan, and self-reflection. This structure significantly augments the explainability and accuracy of LLM inference, producing a human-understandable and inspectable inferential trajectory. Furthermore, our work offers an epistemological foundation for existing prompting techniques, allowing for a possible systematic integration of these methods. With GPT-4, our approach elevates the accuracy from COT baseline of 22.92% to 58.33% in a challenging subset of GSM8K, and from 67.91% to 75.40% in the causal judgment task. Without using few-shot examples or external tools, UPAR significantly outperforms existing prompting methods on SCIBENCH, a challenging dataset containing collegiate-level mathematics, chemistry, and physics scientific problems.