To Ask or Not to Ask? Detecting Absence of Information in Vision and Language Navigation
This addresses efficiency in VLN for agents by reducing digressions from vague instructions, though it's incremental as it focuses only on detecting when information is absent, not what's missing.
The paper tackles the problem of Vision Language Navigation agents failing to recognize when instructions are incomplete, proposing an attention-based module that improves vagueness estimation performance by 52% in precision-recall balance.
Recent research in Vision Language Navigation (VLN) has overlooked the development of agents' inquisitive abilities, which allow them to ask clarifying questions when instructions are incomplete. This paper addresses how agents can recognize "when" they lack sufficient information, without focusing on "what" is missing, particularly in VLN tasks with vague instructions. Equipping agents with this ability enhances efficiency by reducing potential digressions and seeking timely assistance. The challenge in identifying such uncertain points is balancing between being overly cautious (high recall) and overly confident (high precision). We propose an attention-based instruction-vagueness estimation module that learns associations between instructions and the agent's trajectory. By leveraging instruction-to-path alignment information during training, the module's vagueness estimation performance improves by around 52% in terms of precision-recall balance. In our ablative experiments, we also demonstrate the effectiveness of incorporating this additional instruction-to-path attention network alongside the cross-modal attention networks within the navigator module. Our results show that the attention scores from the instruction-to-path attention network serve as better indicators for estimating vagueness.