OS-R1: Agentic Operating System Kernel Tuning with Reinforcement Learning
This addresses the challenge of optimizing operating system performance for users and administrators, though it appears incremental as it builds on existing RL and LLM methods for a specific domain.
The paper tackles the problem of inefficient and non-scalable Linux kernel tuning by introducing OS-R1, an agentic framework using rule-based reinforcement learning, which achieves up to 5.6% performance improvement over heuristic methods while maintaining high data efficiency.
Linux kernel tuning is essential for optimizing operating system (OS) performance. However, existing methods often face challenges in terms of efficiency, scalability, and generalization. This paper introduces OS-R1, an agentic Linux kernel tuning framework powered by rule-based reinforcement learning (RL). By abstracting the kernel configuration space as an RL environment, OS-R1 facilitates efficient exploration by large language models (LLMs) and ensures accurate configuration modifications. Additionally, custom reward functions are designed to enhance reasoning standardization, configuration modification accuracy, and system performance awareness of the LLMs. Furthermore, we propose a two-phase training process that accelerates convergence and minimizes retraining across diverse tuning scenarios. Experimental results show that OS-R1 significantly outperforms existing baseline methods, achieving up to 5.6% performance improvement over heuristic tuning and maintaining high data efficiency. Notably, OS-R1 is adaptable across various real-world applications, demonstrating its potential for practical deployment in diverse environments. Our dataset and code are publicly available at https://github.com/LHY-24/OS-R1.