It's Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoning
This is an incremental analysis for AI researchers, highlighting limitations in LLM reasoning that could affect applications like medical diagnoses.
The study tackled the problem of large language models (LLMs) struggling with process of elimination reasoning using chain-of-thought prompting, finding that this strategy always underperforms choosing correct answers and has lower agreement than self-consistency.
Chain-of-thought (COT) prompting can help large language models (LLMs) reason toward correct answers, but its efficacy in reasoning toward incorrect answers is unexplored. This process of elimination (PoE), when used with COT, can enhance self-consistency, interpretability, and tasks such as medical diagnoses of exclusion. Thus, we propose PoE with COT, where LLMs must reason toward incorrect options on multiple-choice questions. We evaluate the ability of GPT-3.5, LLaMA-2, and Falcon to perform PoE with COT on a total of four commonsense and scientific reasoning datasets. We find that the strategy of PoE always underperforms the strategy of choosing the correct answer. The agreement of these strategies is also lower than the self-consistency of each strategy. To study these issues further, we conduct error analyses and give suggestions for future work.