PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarks
For developers of MoE LLMs, PathMark provides a robust method to protect intellectual property, addressing a critical gap where existing watermarking fails due to MoE's dynamic routing.
PathMark is the first watermarking framework for MoE LLMs, achieving >99% verification accuracy with <2% perplexity degradation by actively steering token routing as a covert channel, and showing superior robustness against quantization, fine-tuning, pruning, and adaptive attacks.
Mixture-of-Experts (MoE) large language models represent high-value intellectual property, yet existing watermarking schemes designed for dense models fail on MoE architectures due to architectural mismatch: traditional methods assume watermarked parameters are consistently activated, but MoE's dynamic routing breaks this assumption. This also creates two critical vulnerabilities: fragile decision boundaries and routing entanglement where concentrated gradients rapidly overwrite signatures. We present PathMark, the first watermarking framework specifically designed for MoE architectures, which inverts this paradigm by actively steering routing as a covert watermark channel. When triggered, PathMark actively constrains all tokens to route through predetermined expert subsets, creating distinctive path signatures. Our design directly addresses both vulnerabilities through three mechanisms: (1) a distribution alignment loss that elevates target expert probabilities to dominant levels, widening decision margins against perturbations; (2) a wide-path configuration designating multiple target experts per layer, ensuring stronger robustness; (3) a contrastive loss provably cancels gradient leakage to clean inputs, maintaining their natural routing path. Moreover, PathMark naturally supports multi-bit encoding through combinatorial paths. Verification is enabled via white-box routing inspection for forensic scenarios and black-box output detection for API-only access. Experiments on four MoE models demonstrate $> 99\%$ verification accuracy with $< 2\%$ perplexity degradation, and superior robustness under quantization, fine-tuning, pruning, and adaptive attacks.