Batch-of-Thought: Cross-Instance Learning for Enhanced LLM Reasoning
For LLM reasoning systems, BoT addresses the inefficiency of independent query processing by leveraging shared patterns and consistency constraints, offering a training-free method to enhance performance and reduce costs.
Batch-of-Thought (BoT) processes related queries jointly to enable cross-instance learning, improving LLM reasoning accuracy and confidence calibration while reducing inference costs by up to 61% across six benchmarks.
Current Large Language Model reasoning systems process queries independently, discarding valuable cross-instance signals such as shared reasoning patterns and consistency constraints. We introduce Batch-of-Thought (BoT), a training-free method that processes related queries jointly to enable cross-instance learning. By performing comparative analysis across batches, BoT identifies high-quality reasoning templates, detects errors through consistency checks, and amortizes computational costs. We instantiate BoT within a multi-agent reflection architecture (BoT-R), where a Reflector performs joint evaluation to unlock mutual information gain unavailable in isolated processing. Experiments across three model families and six benchmarks demonstrate that BoT-R consistently improves accuracy and confidence calibration while reducing inference costs by up to 61%. Our theoretical and experimental analysis reveals when and why batch-aware reasoning benefits LLM systems. Our code is available at https://github.com/xuanyang19/BoT