Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning
This work addresses a specific issue in multi-domain learning for action recognition, but it appears incremental as it builds on existing adapter structures without broad application.
The paper tackled the problem of unclear effectiveness in capturing domain-specific information in multi-domain learning architectures using shared and domain-specific LoRAs, and proposed a method to separate them into different subspaces, demonstrating effectiveness in action recognition on datasets like UCF101, Kinetics400, and HMDB51.
Existing architectures of multi-domain learning have two types of adapters: shared LoRA for all domains and domain-specific LoRA for each particular domain. However, it remains unclear whether this structure effectively captures domain-specific information. In this paper, we propose a method that ensures that shared and domain-specific LoRAs exist in different subspaces; specifically, the column and left null subspaces of the pre-trained weights. We apply the proposed method to action recognition with three datasets (UCF101, Kinetics400, and HMDB51) and demonstrate its effectiveness in some cases along with the analysis of the dimensions of LoRA weights.