Model-agnostic Multi-Domain Learning with Domain-Specific Adapters for Action Recognition
This work addresses the challenge of efficiently learning across multiple domains in action recognition, offering a model-agnostic solution that is more effective and efficient than existing approaches, though it is incremental in nature.
The paper tackles the problem of multi-domain learning for action recognition by proposing a model-agnostic method that uses domain-specific adapters inserted between layers of a backbone network, achieving improved effectiveness over multi-head architectures and better efficiency than separate training per domain on datasets like HMDB51, UCF101, and Kinetics-400.
In this paper, we propose a multi-domain learning model for action recognition. The proposed method inserts domain-specific adapters between layers of domain-independent layers of a backbone network. Unlike a multi-head network that switches classification heads only, our model switches not only the heads, but also the adapters for facilitating to learn feature representations universal to multiple domains. Unlike prior works, the proposed method is model-agnostic and doesn't assume model structures unlike prior works. Experimental results on three popular action recognition datasets (HMDB51, UCF101, and Kinetics-400) demonstrate that the proposed method is more effective than a multi-head architecture and more efficient than separately training models for each domain.