Empowering Multi-step Reasoning across Languages via Tree-of-Thoughts
This addresses a barrier for non-English users in complex reasoning tasks, representing an incremental improvement over existing prompting methods.
The paper tackles the problem of enabling multi-step reasoning in non-English languages for Large Language Models, proposing Cross-lingual Tree-of-Thoughts (Cross-ToT) which significantly outperforms existing methods by reducing interactions and achieving state-of-the-art performance.
Reasoning methods, best exemplified by the well-known Chain-of-Thought (CoT), empower the reasoning abilities of Large Language Models (LLMs) by eliciting them to solve complex tasks in a step-by-step manner. Although they are achieving significant success, the ability to deliver multi-step reasoning remains limited to English because of the imbalance in the distribution of pre-training data, which makes other languages a barrier. In this paper, we propose Cross-lingual Tree-of-Thoughts (Cross-ToT), a method for aligning Cross-lingual CoT reasoning across languages. The proposed method, through a self-consistent cross-lingual prompting mechanism inspired by the Tree-of-Thoughts approach, provides multi-step reasoning paths in different languages that, during the steps, lead to the final solution. Experimental evaluations show that our method significantly outperforms existing prompting methods by reducing the number of interactions and achieving state-of-the-art performance.