Memory Efficient Temporal & Visual Graph Model for Unsupervised Video Domain Adaptation
This work addresses memory efficiency for video domain adaptation, which is a domain-specific problem for researchers and practitioners handling long videos.
The paper tackles the high memory cost of existing video domain adaptation methods by proposing a graph-based approach that models each video as a single graph with attention, reducing memory usage while achieving superior performance compared to state-of-the-art methods.
Existing video domain adaption (DA) methods need to store all temporal combinations of video frames or pair the source and target videos, which are memory cost expensive and can't scale up to long videos. To address these limitations, we propose a memory-efficient graph-based video DA approach as follows. At first our method models each source or target video by a graph: nodes represent video frames and edges represent the temporal or visual similarity relationship between frames. We use a graph attention network to learn the weight of individual frames and simultaneously align the source and target video into a domain-invariant graph feature space. Instead of storing a large number of sub-videos, our method only constructs one graph with a graph attention mechanism for one video, reducing the memory cost substantially. The extensive experiments show that, compared with the state-of-art methods, we achieved superior performance while reducing the memory cost significantly.