SilverSight: A Multi-Task Chinese Financial Large Language Model Based on Adaptive Semantic Space Learning
This work addresses data efficiency and generalization for multi-task learning in specialized domains like finance, but it appears incremental as it builds on existing multi-expert models with a novel adaptation method.
The study tackled the problem of conflicts in model task transfer due to heterogeneous data across specialized fields by introducing the Adaptive Semantic Space Learning (ASSL) framework, which achieved results close to full data training using only 10% of the data and demonstrated strong generalization capabilities.
Large language models (LLMs) are increasingly being applied across various specialized fields, leveraging their extensive knowledge to empower a multitude of scenarios within these domains. However, each field encompasses a variety of specific tasks that require learning, and the diverse, heterogeneous data across these domains can lead to conflicts during model task transfer. In response to this challenge, our study introduces an Adaptive Semantic Space Learning (ASSL) framework, which utilizes the adaptive reorganization of data distributions within the semantic space to enhance the performance and selection efficacy of multi-expert models. Utilizing this framework, we trained a financial multi-task LLM named "SilverSight". Our research findings demonstrate that our framework can achieve results close to those obtained with full data training using only 10% of the data, while also exhibiting strong generalization capabilities.