Efficient Multi-Task Modeling through Automated Fusion of Trained Models
This addresses the problem of inefficient multi-task modeling for AI practitioners by simplifying the process, though it is incremental as it builds on existing single-task models and fusion techniques.
The paper tackles the cumbersome process of designing multi-task models by proposing a method to automatically fuse trained single-task models into a multi-task model, eliminating the need for manual customization of task relationships and model structures, with effectiveness verified on three datasets.
Although multi-task learning is widely applied in intelligent services, traditional multi-task modeling methods often require customized designs based on specific task combinations, resulting in a cumbersome modeling process. Inspired by the rapid development and excellent performance of single-task models, this paper proposes an efficient multi-task modeling method that can automatically fuse trained single-task models with different structures and tasks to form a multi-task model. As a general framework, this method allows modelers to simply prepare trained models for the required tasks, simplifying the modeling process while fully utilizing the knowledge contained in the trained models. This eliminates the need for excessive focus on task relationships and model structure design. To achieve this goal, we consider the structural differences among various trained models and employ model decomposition techniques to hierarchically decompose them into multiple operable model components. Furthermore, we have designed an Adaptive Knowledge Fusion (AKF) module based on Transformer, which adaptively integrates intra-task and inter-task knowledge based on model components. Through the proposed method, we achieve efficient and automated construction of multi-task models, and its effectiveness is verified through extensive experiments on three datasets.