Multi-Task Domain Adaptation for Language Grounding with 3D Objects
This work addresses language grounding for 3D objects, which is incremental as it builds on existing methods by incorporating domain adaptation and multi-task learning.
The paper tackles the problem of object-level language grounding with 3D objects by proposing a domain adaptation method to improve cross-modal representation in cross-domain settings, achieving state-of-the-art accuracies of 83.8% and 86.8% in single-view and multi-view benchmarks.
The existing works on object-level language grounding with 3D objects mostly focus on improving performance by utilizing the off-the-shelf pre-trained models to capture features, such as viewpoint selection or geometric priors. However, they have failed to consider exploring the cross-modal representation of language-vision alignment in the cross-domain field. To answer this problem, we propose a novel method called Domain Adaptation for Language Grounding (DA4LG) with 3D objects. Specifically, the proposed DA4LG consists of a visual adapter module with multi-task learning to realize vision-language alignment by comprehensive multimodal feature representation. Experimental results demonstrate that DA4LG competitively performs across visual and non-visual language descriptions, independent of the completeness of observation. DA4LG achieves state-of-the-art performance in the single-view setting and multi-view setting with the accuracy of 83.8% and 86.8% respectively in the language grounding benchmark SNARE. The simulation experiments show the well-practical and generalized performance of DA4LG compared to the existing methods. Our project is available at https://sites.google.com/view/da4lg.