Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy
This addresses a challenge in constructing general-purpose AI systems by improving LoRA merging for better multi-task representation, though it is incremental as it builds on existing LoRA methods.
The paper tackles the problem of merging multiple Low-Rank Adaptation (LoRA) modules, which can weaken critical task directions and reduce model performance, by proposing TARA-Merging to align merging weights and preserve task-relevant subspaces, resulting in consistent outperformance across eight vision and six NLI benchmarks.
Merging multiple Low-Rank Adaptation (LoRA) modules is promising for constructing general-purpose systems, yet challenging because LoRA update directions span different subspaces and contribute unevenly. When merged naively, such mismatches can weaken the directions most critical to certain task losses while overemphasizing relatively less important ones, ultimately reducing the model's ability to represent all tasks faithfully. We revisit this problem through two perspectives: subspace coverage, which captures how broadly LoRA directions cover diverse representational directions, and anisotropy, which reflects the imbalance of influence across those directions. We propose TARA-Merging (Task-Rank Anisotropy Alignment), which aligns merging weights using a preference-weighted cross-entropy pseudo-loss while preserving task-relevant LoRA subspaces. This ensures broad subspace coverage and mitigates anisotropy via direction-wise reweighting. Across eight vision and six NLI benchmarks, TARA-Merging consistently outperforms vanilla and LoRA-aware baselines, demonstrating strong robustness and generalization, and highlighting the importance of addressing both subspace coverage and anisotropy in LoRA merging.