Self-Consistency

“Although self-consistency can enhance output stability, it also often leads to hallucinations or invalid queries due to semantic errors since sampling at higher temperatures increases randomness instead of improving diversity.”

“It improves accuracy without retraining but suffers from high inference cost and lacks intra-chain correction. The mistakes made early in a chain propagate to the end since errors are never revised mid-way.”

“self-consistency must be inferred multiple times, burdening deployment budgets”

“However, these prevailing strategies are inherently bounded by the quality of the set of candidates. They are limited to produce a solution that transcends the quality of candidate proposals, which becomes particularly problematic when all candidates are flawed.”

“outperforming prior refinement methods~wang2023selfconsistency”

“However, these structured reasoning methods universally require generating longer sequences or processing multiple reasoning paths, leading directly to a substantial increase in inference cost”

“However, as self-consistency is fundamentally an extension of CoT, it is unclear whether self-consistency also improves performance on non-math questions that involve the recall of encyclopedic knowledge”

“Early SC approaches focused on majority voting for final answers”

“While effective, it uses a fixed number of calls and can fail when the correct answer is infrequent.”

“While effective, SC's primary drawback is its high computational cost.”

“However, despite its effectiveness, SC incurs significant computational costs at inference time due to its requirement for multiple sampling iterations.”

“these studies predominantly focus on short tasks (e.g., math problems, brief QA), where stochastic sampling produces independent errors. This assumption breaks down in long settings where systemic errors emerge due to position bias.”