Learning What Matters: Probabilistic Task Selection via Mutual Information for Model Finetuning
This addresses the challenge for practitioners in efficiently and robustly finetuning LLMs, offering an interpretable and scalable alternative to heuristic-based methods, though it is incremental as it builds on existing finetuning approaches.
The paper tackles the problem of manually selecting task datasets for finetuning large language models by introducing TASKPGM, a framework that optimizes task mixtures using a Markov Random Field and behavioral divergences, resulting in consistent empirical improvements on models like Llama 2 and Mistral across benchmarks such as MMLU and BIGBench.
The performance of finetuned large language models (LLMs) hinges critically on the composition of the training mixture. However, selecting an optimal blend of task datasets remains a largely manual, heuristic driven process, with practitioners often relying on uniform or size based sampling strategies. We introduce TASKPGM, a principled and scalable framework for mixture optimization that selects continuous task proportions by minimizing an energy function over a Markov Random Field (MRF). Task relationships are modeled using behavioral divergences such as Jensen Shannon Divergence and Pointwise Mutual Information computed from the predictive distributions of single task finetuned models. Our method yields a closed form solution under simplex constraints and provably balances representativeness and diversity among tasks. We provide theoretical guarantees, including weak submodularity for budgeted variants, and demonstrate consistent empirical improvements on Llama 2 and Mistral across evaluation suites such as MMLU and BIGBench. Beyond performance, TASKPGM offers interpretable insights into task influence and mixture composition, making it a powerful tool for efficient and robust LLM finetuning.