LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers
This work addresses reliability issues in knowledge-intensive tasks for users of large language models, representing an incremental improvement over existing decoding-time strategies.
The paper tackled the problem of factual errors in large language models by introducing a token-aware, layer-localized contrastive decoding method that aligns token types with influential transformer layers, achieving consistent improvements in factuality across multiple models and benchmarks without additional training.
Large language models (LLMs) excel at natural language understanding and generation but remain vulnerable to factual errors, limiting their reliability in knowledge-intensive tasks. While decoding-time strategies provide a promising efficient solution without training, existing methods typically treat token-level and layer-level signals in isolation, overlooking the joint dynamics between them. In this work, we introduce a token-aware, layer-localized contrastive decoding method that aligns specific token types with their most influential transformer layers to improve factual generation. Through empirical attention analysis, we identify two key patterns: punctuation tokens receive dominant attention in early layers, while conceptual tokens govern semantic reasoning in intermediate layers. By selectively suppressing attention to these token types at their respective depths, we achieve the induction of controlled factual degradation and derive contrastive signals to guide the final factual decoding. Our method requires no additional training or model modification, and experiments demonstrate that our method consistently improves factuality across multiple LLMs and various benchmarks.