Robust and Efficient Guardrails with Latent Reasoning
For practitioners deploying LLM safety guardrails, COLAGUARD resolves the trade-off between reasoning quality and inference efficiency, making robust guardrails practical for high-throughput applications.
COLAGUARD transfers multi-step safety reasoning into continuous latent space, achieving 12.9x speedup and 22.4x token reduction over explicit reasoning baselines while improving macro-F1 by 8.24 points over Llama Guard 3 across eight safety benchmarks.
Maintaining the safety of large language models (LLMs) is crucial as they are increasingly deployed in real-world applications. Existing safety guardrails typically rely on single-pass classification or, more recently, distilled reasoning. Reasoning-based guardrails significantly outperform classification-only baselines, but they incur substantial query latency and token overhead that make them impractical for highthroughput deployment. To address this challenge, we propose COLAGUARD, a guardrail model that transfers multi-step safety reasoning into a continuous latent space through a stage-wise training curriculum, enabling direct hidden-state propagation at inference. Evaluated on ten prompt- and response-moderation settings spanning eight safety benchmarks, COLAGUARD improves macro-F1 by 8.24 points over Llama Guard 3 and matches our explicit reasoning baseline, GuardReasoner, in macroF1 while delivering a 12.9X speedup and 22.4X reduction in token usage. Our results suggest that latent reasoning offers a practical alternative to explicit rationale generation for deployable guardrails, jointly improving safety robustness and inference efficiency rather than treating them as competing objectives.