PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
This addresses the efficiency challenge for practitioners finetuning large models, though it is incremental as it builds on existing LoRA methods.
The paper tackled the problem of efficiently finetuning large models by introducing PLoP, a method for automatically identifying optimal module types for LoRA adapter placement, which consistently outperformed or matched common strategies in experiments on supervised finetuning and reinforcement learning for reasoning.
Low-Rank Adaptation (LoRA) is a widely used finetuning method for large models. Its small memory footprint allows practitioners to adapt large models to specific tasks at a fraction of the cost of full finetuning. Different modifications have been proposed to enhance its efficiency by, for example, setting the learning rate, the rank, and the initialization. Another improvement axis is adapter placement strategy: when using LoRA, practitioners usually pick module types to adapt with LoRA, such as Query and Key modules. Few works have studied the problem of adapter placement, with nonconclusive results: original LoRA paper suggested placing adapters in attention modules, while other works suggested placing them in the MLP modules. Through an intuitive theoretical analysis, we introduce PLoP (Precise LoRA Placement), a lightweight method that allows automatic identification of module types where LoRA adapters should be placed, given a pretrained model and a finetuning task. We demonstrate that PLoP consistently outperforms, and in the worst case competes, with commonly used placement strategies through comprehensive experiments on supervised finetuning and reinforcement learning for reasoning.