Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
This addresses the need for making scientific information accessible across multiple technical fields for non-expert audiences, representing an incremental improvement over single-domain approaches.
The paper tackles the problem of cross-domain lay paraphrasing by proposing Sci-LoRA, a model that uses a mixture of LoRAs fine-tuned on multiple scientific domains, which significantly outperforms state-of-the-art large language models across twelve domains on five public datasets.
Lay paraphrasing aims to make scientific information accessible to audiences without technical backgrounds. However, most existing studies focus on a single domain, such as biomedicine. With the rise of interdisciplinary research, it is increasingly necessary to comprehend knowledge spanning multiple technical fields. To address this, we propose Sci-LoRA, a model that leverages a mixture of LoRAs fine-tuned on multiple scientific domains. In particular, Sci-LoRA dynamically generates and applies weights for each LoRA, enabling it to adjust the impact of different domains based on the input text, without requiring explicit domain labels. To balance domain-specific knowledge and generalization across various domains, Sci-LoRA integrates information at both the data and model levels. This dynamic fusion enhances the adaptability and performance across various domains. Experimental results across twelve domains on five public datasets show that Sci-LoRA significantly outperforms state-of-the-art large language models and demonstrates flexible generalization and adaptability in cross-domain lay paraphrasing.