Hint of Pseudo Code (HoPC): Zero-Shot Step by Step Pseudo Code Reasoning Prompting
This work addresses the problem of inconsistent reasoning in language models for complex tasks, offering a more robust zero-shot method that is incremental over existing techniques like Chain of Thought and Program of Thought.
The paper tackles the challenge of improving zero-shot reasoning in large language models for multi-step tasks by introducing Hint of Pseudo Code (HoPC), a prompting technique that enhances problem decomposition and semantic code reasoning without requiring an interpreter, achieving competitive performance on benchmarks like StrategyQA.
Prompting a language model (LM) is an increasingly important research topic for better utilization of large language models (LLMs). While simple prompting is effective for single-step questions, it fails to activate the correct knowledge path for multi-step reasoning tasks consistently. The few-shot Chain of Thought (CoT), serves as an advanced prompting strategy that explains and demonstrates the reasoning process to the LLM, outperforming simple prompting in challenging reasoning tasks such as arithmetic and common-sense reasoning. The Program of Thought (PoT) aims to generate text and programming language solutions for multi-step reasoning problems. In zero-shot CoT, the prompt is simply ``Let's think step by step'', which is overly simplistic and does not adequately demonstrate a robust reasoning process for complex reasoning challenges. Additionally, PoT requires an extra interpreter to execute the answer and struggles with semantic reasoning problems like StrategyQA. This paper introduces a novel Hint of Pseudo Code (HoPC) prompting technique that does not require extra interpreter as in PoT and incorporates a more powerful zero-shot problem decomposition and semantic code reasoning capabilities than zero-shot CoT. It consists of three components: problem decomposition, semantic code reasoning, and answer extraction. We prompt these components as hints in a sequential, step by step manner, making it easy to tailor and explain for various tasks.