Improving LoRA with Variational Learning
This work provides a more efficient and effective method for finetuning large language models, which is significant for practitioners in NLP seeking better performance with lower overhead, though it is incremental as it builds on existing variational and LoRA techniques.
The paper tackles the problem of improving LoRA finetuning for large language models by addressing the limitations of Bayesian methods, such as marginal accuracy gains and high computational costs, using a variational algorithm called IVON. The result shows that IVON, combined with posterior pruning, improves accuracy by 1.3% and reduces expected calibration error by 5.4% on a Llama-3.2-3B model compared to AdamW.
Bayesian methods have recently been used to improve LoRA finetuning and, although they improve calibration, their effect on other metrics (such as accuracy) is marginal and can sometimes even be detrimental. Moreover, Bayesian methods also increase computational overheads and require additional tricks for them to work well. Here, we fix these issues by using a recently proposed variational algorithm called IVON. We show that IVON is easy to implement and has similar costs to AdamW, and yet it can also drastically improve many metrics by using a simple posterior pruning technique. We present extensive results on billion-scale LLMs (Llama and Qwen series) going way beyond the scale of existing applications of IVON. For example, we finetune a Llama-3.2-3B model on a set of commonsense reasoning tasks and improve accuracy over AdamW by 1.3% and reduce ECE by 5.4%, outperforming AdamW and other recent Bayesian methods like Laplace-LoRA and BLoB. Overall, our results show that variational learning with IVON can effectively improve LoRA finetuning.