From Dispersion to Attraction: Spectral Dynamics of Hallucination Across Whisper Model Scales
For ASR safety researchers, this provides a theoretical framework linking spectral dynamics to hallucination, but the validation is limited to Whisper models and adversarial stress.
The paper proposes the Spectral Sensitivity Theorem to predict a phase transition in deep networks from signal decay to rank-1 collapse, and validates it on Whisper models (Tiny to Large-v3-Turbo) under adversarial stress, finding that intermediate models show a 13.4% collapse in Cross-Attention rank while large models enter a compression-seeking attractor state with -2.34% rank compression in Self-Attention.
Hallucinations in large ASR models present a critical safety risk. In this work, we propose the \textit{Spectral Sensitivity Theorem}, which predicts a phase transition in deep networks from a dispersive regime (signal decay) to an attractor regime (rank-1 collapse) governed by layer-wise gain and alignment. We validate this theory by analyzing the eigenspectra of activation graphs in Whisper models (Tiny to Large-v3-Turbo) under adversarial stress. Our results confirm the theoretical prediction: intermediate models exhibit \textit{Structural Disintegration} (Regime I), characterized by a $13.4\%$ collapse in Cross-Attention rank. Conversely, large models enter a \textit{Compression-Seeking Attractor} state (Regime II), where Self-Attention actively compresses rank ($-2.34\%$) and hardens the spectral slope, decoupling the model from acoustic evidence.