GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization
This addresses reliability and safety issues in LLMs for users and developers, offering a generalizable solution, though it is incremental as it builds on existing detection methods.
The paper tackled the problem of detecting glitch tokens that cause unpredictable behavior in Large Language Models by introducing GlitchMiner, a framework that uses gradient-based discrete optimization to maximize predictive entropy, resulting in consistent outperformance in detection accuracy and query efficiency across ten LLMs from five model families.
Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner]