Adaptive parameter-efficient fine-tuning via Hessian-informed subset selection
This addresses the challenge of efficiently fine-tuning large models for downstream tasks with minimal computational overhead, representing an incremental improvement in PEFT methods.
The paper tackles the problem of selecting the most influential subset of parameters for parameter-efficient fine-tuning (PEFT) in large pre-trained models, proposing AdaPEFT, a Hessian-informed method that formulates this as a multi-task optimization problem and solves it via Pareto optimality, achieving adaptive performance across tasks and models.
Parameter-efficient fine-tuning (PEFT) is a highly effective approach for adapting large pre-trained models to downstream tasks with minimal computational overhead. At the core, PEFT methods freeze most parameters and only trains a small subset (say $<0.1\%$ of total parameters). Notably, different PEFT methods select different subsets, resulting in varying levels of performance. This variation prompts a key question: how to effectively select the most influential subset to train? We formulate the subset selection as a multi-task problem: maximizing the performance and minimizing the number of trainable parameters. We leverage a series of transformations -- including $ε$-constraint method and second-order Taylor approximation -- to arrive at the classical 0-1 knapsack problem, which we solve through the lens of Pareto optimality. Consequently, we propose AdaPEFT, a Hessian-informed PEFT that adapts to various tasks and models, in which the selected subset empirically transfers across training horizons and model sizes.