Distilled Large Language Model-Driven Dynamic Sparse Expert Activation Mechanism
For industrial defect detection, DS-MoE offers a new approach to handle inter-class similarity and scale variation with improved accuracy and real-time inference.
The paper proposes DS-MoE, a framework combining LLM-driven dynamic routing with sparse Mixture-of-Experts for visual defect recognition. It achieves +13.9, +1.4, and +2.0 pp mAP@0.5:0.95 over YOLOv8/YOLOX on BBMP, aluminum, and PCB datasets.
High inter-class similarity, extreme scale variation, and limited computational budgets hinder reliable visual recognition across diverse real-world data. Existing vision-centric and cross-modal approaches often rely on rigid fusion mechanisms and heavy annotation pipelines, leading to sub-optimal generalization. We propose the Distilled Large Language Model (LLM)-Driven Sparse Mixture-of-Experts (DS-MoE) framework, which integrates text-guided dynamic routing and lightweight multi-scale comprehension. The DS-MoE framework dynamically aligns textual semantics with defect-specific visual patterns through a sparse MoE architecture, where task-relevant experts are adaptively activated based on semantic relevance, resolving inter-class ambiguity. A lightweight MobileSAM encoder enables real-time inference while preserving multi-scale defect details. Extensive experiments on PCB, aluminum foil, and mold defect datasets demonstrate that our framework achieves superior performance compared to existing pure vision models. \textbf{DS-MoE} surpasses YOLOv8/YOLOX with gains of +13.9, +1.4, and +2.0 pp mAP@ 0.5:0.95 on BBMP, aluminum, and PCB, respectively, while also improving precision and recall.