Training-free Truthfulness Detection via Value Vectors in LLMs
This work addresses the challenge of scalable and interpretable truthfulness detection for LLM users, though it is incremental as it builds on prior training-free methods by extending analysis to MLP modules.
The paper tackled the problem of detecting factual correctness in large language model outputs by identifying truthfulness-related patterns in MLP module value vectors, resulting in a method that significantly outperformed existing training-free and log-likelihood baselines on the NoVo benchmark.
Large language models often generate factually incorrect outputs, motivating efforts to detect the truthfulness of their content. Most existing approaches rely on training probes over internal activations, but these methods suffer from scalability and generalization issues. A recent training-free method, NoVo, addresses this challenge by exploiting statistical patterns from the model itself. However, it focuses exclusively on attention mechanisms, potentially overlooking the MLP module-a core component of Transformer models known to support factual recall. In this paper, we show that certain value vectors within MLP modules exhibit truthfulness-related statistical patterns. Building on this insight, we propose TruthV, a simple and interpretable training-free method that detects content truthfulness by leveraging these value vectors. On the NoVo benchmark, TruthV significantly outperforms both NoVo and log-likelihood baselines, demonstrating that MLP modules-despite being neglected in prior training-free efforts-encode rich and useful signals for truthfulness detection. These findings offer new insights into how truthfulness is internally represented in LLMs and motivate further research on scalable and interpretable truthfulness detection.