A Lightweight Framework for Trigger-Guided LoRA-Based Self-Adaptation in LLMs
This addresses the limitation of LLMs in continuous adaptation during inference, which is an incremental improvement for enhancing real-time reasoning capabilities.
The paper tackles the problem of large language models being unable to adapt to new data during inference by proposing SAGE, a framework that enables dynamic fine-tuning during reasoning, resulting in excellent accuracy, robustness, and stability on atomic reasoning subtasks.
Large language models are unable to continuously adapt and learn from new data during reasoning at inference time. To address this limitation, we propose that complex reasoning tasks be decomposed into atomic subtasks and introduce SAGE, a trigger-guided dynamic fine-tuning framework that enables adaptive updates during reasoning at inference time. SAGE consists of three key components: (1) a Trigger module that detects reasoning failures through multiple evaluation metrics in real time; (2) a Trigger Buffer module that clusters anomaly samples using a streaming clustering process with HDBSCAN, followed by stability checks and similarity-based merging; and (3) a Lora Store module that dynamically optimizes parameter updates with an adapter pool for knowledge retention. Evaluation results show that SAGE demonstrates excellent accuracy, robustness, and stability on the atomic reasoning subtask through dynamic knowledge updating during test time.