On-device System of Compositional Multi-tasking in Large Language Models
This addresses the problem of efficient compositional multi-tasking for real-world applications with resource constraints, representing an incremental improvement over existing adapter-based methods.
The paper tackles the challenge of simultaneous execution of complex tasks like generating translated summaries from conversations in large language models, proposing a novel approach with a learnable projection layer on combined adapters that demonstrates practical viability in on-device environments with fast performance.
Large language models (LLMs) are commonly adapted for diverse downstream tasks via parameter-efficient fine-tuning techniques such as Low-Rank Adapters (LoRA). While adapters can be combined to handle multiple tasks separately, standard approaches struggle when targeting the simultaneous execution of complex tasks, such as generating a translated summary from a long conversation. To address this challenge, we propose a novel approach tailored specifically for compositional multi-tasking scenarios involving summarization and translation. Our technique involves adding a learnable projection layer on top of the combined summarization and translation adapters. This design enables effective integration while maintaining efficiency through reduced computational overhead compared to alternative strategies requiring extensive retraining or sequential processing. We demonstrate the practical viability of our method within an on-device environment by developing an Android app capable of executing compositional tasks seamlessly. Experimental results indicate our solution performs well and is fast in both cloud-based and on-device implementations, highlighting the potential benefits of adopting our framework in real-world applications demanding high-speed operation alongside resource constraints.