Actions and Objects Pathways for Domain Adaptation in Video Question Answering
This addresses domain adaptation for video question answering, offering a lightweight solution with minimal training parameters, though it appears incremental as it builds on pretrained models and existing datasets.
The paper tackles out-of-domain generalization in video question answering by introducing the Actions and Objects Pathways (AOPath), which dissociates pretrained features into action and object pathways for domain-agnostic processing without trainable weights, achieving 5% and 4% superior performance over conventional classifiers on out-of-domain and in-domain datasets, respectively.
In this paper, we introduce the Actions and Objects Pathways (AOPath) for out-of-domain generalization in video question answering tasks. AOPath leverages features from a large pretrained model to enhance generalizability without the need for explicit training on the unseen domains. Inspired by human brain, AOPath dissociates the pretrained features into action and object features, and subsequently processes them through separate reasoning pathways. It utilizes a novel module which converts out-of-domain features into domain-agnostic features without introducing any trainable weights. We validate the proposed approach on the TVQA dataset, which is partitioned into multiple subsets based on genre to facilitate the assessment of generalizability. The proposed approach demonstrates 5% and 4% superior performance over conventional classifiers on out-of-domain and in-domain datasets, respectively. It also outperforms prior methods that involve training millions of parameters, whereas the proposed approach trains very few parameters.