AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images
For researchers in visual localization and multi-sensor fusion, AnyMatch provides a scalable, low-cost method to generate geometrically consistent multi-modal training data, addressing the scarcity of such data.
AnyMatch generates large-scale multi-modal image pairs from single-view images using monocular depth, 3D reprojection, and diffusion-based inpainting, avoiding SfM-MVS errors. Fine-tuning matching networks (LoFTR, EDM, RoMa) on the resulting Any-syn dataset yields substantial performance gains on multi-modal benchmarks.
Multi-modal image matching is essential for visual localization and multi-sensor fusion, but it is hindered by the scarcity of large-scale training data with precise geometric annotations. Existing real-world datasets suffer from prohibitive costs, limited scene diversity, and errors in SfM-MVS pipelines, while synthetic methods struggle to maintain 3D geometric consistency or achieve photorealistic appearance. To address this, we propose AnyMatch, a novel framework that leverages abundant, easily accessible single-view images at minimal cost to generate rich multi-modal training data. AnyMatch integrates monocular depth estimation, 3D reprojection, diffusion-based inpainting, and crossmodal image translation to synthesize multi-view, multi-modal image pairs with 3D geometric fidelity. Crucially, our method provides annotations that strictly adhere to 3D geometric consistency through explicit 3D reprojection, avoiding SfM-MVS error accumulation. Furthermore, AnyMatch offers strong scalability, enabling controllable scene diversity and annotation difficulty via adjustable input and camera parameters. We construct Any-syn, a large-scale synthetic multi-modal dataset using AnyMatch. Experimental results show that matching networks (e.g., LoFTR, EDM, RoMa) fine-tuned on Any-syn achieve substantial performance gains on multi-modal benchmarks, exhibiting superior generalization and robustness compared to models trained on existing data.