EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2021: Team M3EM Technical Report
This work addresses domain adaptation for action recognition in kitchen environments, presenting an incremental improvement through multi-modal integration.
The authors tackled unsupervised domain adaptation for action recognition by proposing a Multi-Modal Mutual Enhancement Module (M3EM) that leverages multiple modalities to improve transferable representations across domains, achieving results in the EPIC-KITCHENS-100 challenge.
In this report, we describe the technical details of our submission to the 2021 EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition. Leveraging multiple modalities has been proved to benefit the Unsupervised Domain Adaptation (UDA) task. In this work, we present Multi-Modal Mutual Enhancement Module (M3EM), a deep module for jointly considering information from multiple modalities to find the most transferable representations across domains. We achieve this by implementing two sub-modules for enhancing each modality using the context of other modalities. The first sub-module exchanges information across modalities through the semantic space, while the second sub-module finds the most transferable spatial region based on the consensus of all modalities.