R-Log: Incentivizing Log Analysis Capability in LLMs via Reasoning-based Reinforcement Learning
This work addresses the challenge of improving log analysis for software system operators by offering a novel method that enhances generalizability and reduces hallucinations, though it is incremental in building on existing RL and reasoning techniques.
The paper tackles the problem of domain discrepancy and hallucinations in using LLMs for automated log analysis by proposing R-Log, a reasoning-based reinforcement learning approach that outperforms existing methods by up to 228.05% in unseen scenarios and achieves a 5x speedup with 93% efficacy retention.
The growing complexity of log data in modern software systems has prompted the use of Large Language Models (LLMs) for automated log analysis. Current approaches typically rely on direct supervised fine-tuning (SFT) on log-label pairs. However, this exacerbates the domain discrepancy between general-purpose LLMs and specialized log data, causing overfitting. Furthermore, SFT's imbalanced loss computation often allows lengthy contexts to overwhelm critical, concise details in model answers, leading to hallucinations. To address these limitations, we propose R-Log, a novel reasoning-based paradigm that mirrors the structured, step-by-step analytical process of human engineers. This approach enhances generalizability by learning the underlying rules behind conclusions. We further employ Reinforcement Learning (RL) to optimize the model within a simulated O&M environment, thereby reducing hallucinations by directly rewarding correct outcomes. R-Log is first cold-started on a curated dataset of 2k+ reasoning trajectories, guided by 13 strategies from manual O&M practices, to establish an initial reasoning capability. This ability is then refined via RL using a joint reward function. Empirical evaluations on real-world logs show that R-Log outperforms existing methods across five log analysis tasks, particularly in unseen scenarios (by 228.05%). We also designed R-Log-fast with 5x speedup while keeping 93% of the efficacy.