Parameter-efficient fine-tuning (LoRA family)

LoRA: Low-Rank Adaptation of Large Language Models

194 critique · 201 beaten on benchmarks

AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning

29 critique · 54 beaten on benchmarks

DoRA: Weight-Decomposed Low-Rank Adaptation

16 critique · 66 beaten on benchmarks

PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models

14 critique · 30 beaten on benchmarks

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

7 critique · 28 beaten on benchmarks

Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning

10 critique · 23 beaten on benchmarks

9 critique · 17 beaten on benchmarks

MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models

11 critique · 10 beaten on benchmarks

QLoRA: Efficient Finetuning of Quantized LLMs

5 critique · 14 beaten on benchmarks

Prefix-Tuning: Optimizing Continuous Prompts for Generation

7 critique · 8 beaten on benchmarks

9 critique · 6 beaten on benchmarks

LoRA: Low-Rank Adaptation of Large Language Models

2 critique · 12 beaten on benchmarks