RE-Adapt: Reverse Engineered Adaptation of Large Language Models
This addresses the challenge of adapting large language models to new domains while preserving instruction-following abilities, which is incremental but practical for users needing domain-specific fine-tuning.
The authors tackled the problem of fine-tuning large language models on new domains without degrading pre-existing instruction-tuning by introducing RE-Adapt, which reverse engineers an adapter to isolate learned instruction-following capabilities; the method outperformed other fine-tuning approaches across multiple models and datasets, including with retrieval-augmented generation.
We introduce RE-Adapt, an approach to fine-tuning large language models on new domains without degrading any pre-existing instruction-tuning. We reverse engineer an adapter which isolates what an instruction-tuned model has learned beyond its corresponding pretrained base model. Importantly, this requires no additional data or training. We can then fine-tune the base model on a new domain and readapt it to instruction following with the reverse engineered adapter. RE-Adapt and our low-rank variant LoRE-Adapt both outperform other methods of fine-tuning, across multiple popular LLMs and datasets, even when the models are used in conjunction with retrieval-augmented generation.