LLM Safety From Within: Detecting Harmful Content with Internal Representations
Provides a lightweight, efficient, and generalizable harmfulness detector for LLM safety, addressing the bottleneck of existing guard models that ignore internal representations.
SIREN detects harmful content in LLM inputs/outputs using internal safety neurons, outperforming state-of-the-art guard models with 250x fewer trainable parameters and superior generalization.
Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection.