Fine-Grained Alignment in Vision-and-Language Navigation through Bayesian Optimization
This work addresses fine-grained vision negatives in VLN tasks for robotics, representing an incremental improvement over existing contrastive learning methods.
The paper tackles the challenge of fine-grained alignment in Vision-and-Language Navigation by introducing a Bayesian Optimization-based adversarial optimization framework to enhance cross-modal embeddings, leading to improved navigation performance on R2R and REVERIE benchmarks.
This paper addresses the challenge of fine-grained alignment in Vision-and-Language Navigation (VLN) tasks, where robots navigate realistic 3D environments based on natural language instructions. Current approaches use contrastive learning to align language with visual trajectory sequences. Nevertheless, they encounter difficulties with fine-grained vision negatives. To enhance cross-modal embeddings, we introduce a novel Bayesian Optimization-based adversarial optimization framework for creating fine-grained contrastive vision samples. To validate the proposed methodology, we conduct a series of experiments to assess the effectiveness of the enriched embeddings on fine-grained vision negatives. We conduct experiments on two common VLN benchmarks R2R and REVERIE, experiments on the them demonstrate that these embeddings benefit navigation, and can lead to a promising performance enhancement. Our source code and trained models are available at: https://anonymous.4open.science/r/FGVLN.