Online Data Selection for Instruction Tuning via Gaussian Processes
For researchers and practitioners in LLM fine-tuning, this work provides a more robust and scalable data selection method that addresses the limitation of batch-constrained optimization.
The paper tackles the problem of online data selection for instruction tuning of LLMs, where existing methods are limited to local optimization within random batches. The proposed GAIA framework uses Gaussian Process regression to model global utility manifolds and achieves significant improvements over state-of-the-art baselines across three datasets.
With Large Language Model (LLM) pre-training and fine-tuning shifting its focus from data volume to data quality, quality data selection has emerged as a critical research topic. Existing online data selection methods for LLM training are typically "batch-constrained", limiting optimization to local utility within random batches. To overcome this, we propose GAIA (Global Adaptive Instruction tuning via GAussian processes), a framework that formulates data valuation as a global estimation process. GAIA employs Gaussian Process regression to model continuous utility manifolds across the semantic space, utilizing an adaptive strategy fusion mechanism to dynamically prioritize high-utility samples. By casting the strategy-posterior update as an instance of the classical fixed-share Hedge framework for tracking the best expert, we inherit a dynamic-regret guarantee that characterizes GAIA's robustness under non-stationary quality scores during training. Empirical evaluations on three datasets demonstrate that GAIA significantly outperforms state-of-the-art baselines like \greats, establishing our method as a scalable and robust solution for efficient instruction tuning.