Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
This work addresses the memory and latency bottlenecks of on-device fine-tuning for large language models by proposing a more efficient adapter design.
LoRDBA replaces LoRA's low-rank factors with binary sign carriers and channel-wise scales, achieving competitive quality with fp16 LoRA while reducing adapter footprint over 10x and incurring at most 8% prefill latency overhead at rank 16.
On-device adaptation of large language models commonly keeps a quantized base model frozen while training and deploying a small, task-specific LoRA adapter. In the unmerged adapter-mode setting, however, the adapter is more than a compact storage module; it introduces an additional dense floating-point branch, maintains a trainable state for local updates, and acts as a unit of communication and hot-swapping.We introduce LoRDBA, a LoRA-compatible adapter that replaces both low-rank factors with binary sign carriers while representing magnitudes through lightweight, channel-wise scales, converting the dense adapter branch into two sign-accumulation matrix multiplications interleaved with channel-wise scaling. A finite-sample analysis shows that reconstruction quality is governed by the residual-to-magnitude ratio of the original LoRA factors. In adapter-mode experiments, LoRDBA outperforms low-bit baselines at matched model sizes while matching fp16 LoRA quality in selected regimes. The unmerged adapter incurs at most 8% prefill latency overhead at matched rank r=16 despite an over 10x reduction in adapter footprint, with moderate training memory overhead of approximately 1.6x that of fp16 LoRA.