FORTE: FOL-guided Optimal Refinement for Text-audio rEtrieval
For researchers in cross-modal retrieval, FORTE offers a novel way to incorporate symbolic reasoning to improve fine-grained semantic alignment, though the gains are incremental over existing methods.
FORTE integrates first-order logic reasoning with parameter-efficient cross-modal alignment to improve text-to-audio retrieval, achieving consistent gains over strong baselines on AudioCaps and Clotho, especially in fine-grained scenarios.
Text-to-audio retrieval has made significant progress with shared embedding models such as CLAP and Pengi, yet they often struggle with fine-grained semantic alignment due to the inherent modality gap between text and audio. In this work, we propose FORTE, a unified framework that integrates structured logical reasoning with parameter-efficient cross-modal alignment to improve retrieval precision. Our approach first transforms queries into first-order logic and refines them via a constrained search that preserves semantic invariance while introducing discriminative attributes. The refined representation is then aligned with audio embeddings using a lightweight projection module, followed by a predicate-aware re-ranking step that enforces logical consistency at inference. Extensive experiments on AudioCaps and Clotho demonstrate consistent improvements over strong baselines, particularly in challenging fine-grained scenarios. Our results highlight the effectiveness of combining symbolic reasoning with representation learning for cross-modal retrieval.