APCD: Adaptive Path-Contrastive Decoding for Reliable Large Language Model Generation
For LLM practitioners, APCD offers a principled method to enhance generation reliability without sacrificing speed, though the gains are incremental over existing multi-path approaches.
APCD introduces a multi-path decoding framework that reduces hallucinations in LLMs by adaptively branching only when uncertainty is high and regulating path interactions based on distribution divergence. It achieves improved factual accuracy across eight benchmarks while maintaining efficiency.
Large language models (LLMs) often suffer from hallucinations due to error accumulation in autoregressive decoding, where suboptimal early token choices misguide subsequent generation. Although multi-path decoding can improve robustness by exploring alternative trajectories, existing methods lack principled strategies for determining when to branch and how to regulate inter-path interactions. We propose Adaptive Path-Contrastive Decoding (APCD), a multi-path decoding framework that improves output reliability through adaptive exploration and controlled path interaction. APCD consists of two components: (1) Entropy-Driven Path Expansion, which delays branching until predictive uncertainty - measured by Shannon entropy over top candidate tokens - indicates multiple plausible continuations; and (2) Divergence-Aware Path Contrast, which encourages diverse reasoning trajectories while dynamically attenuating inter-path influence as prediction distributions diverge. Experiments on eight benchmarks demonstrate improved factual accuracy while maintaining decoding efficiency. Our code is available at https://github.com/zty-king/APCD.