U(PM)$^2$:Unsupervised polygon matching with pre-trained models for challenging stereo images
This addresses polygon matching for computer vision and remote sensing applications, offering a low-cost solution with strong performance, though it is incremental as it builds on pre-trained models.
The paper tackles the problem of polygon matching in stereo images, which faces challenges like disparity discontinuity and scale variation, by proposing U(PM)$^2$, an unsupervised method that achieves state-of-the-art accuracy on ScanNet and SceneFlow datasets at competitive speed without training.
Stereo image matching is a fundamental task in computer vision, photogrammetry and remote sensing, but there is an almost unexplored field, i.e., polygon matching, which faces the following challenges: disparity discontinuity, scale variation, training requirement, and generalization. To address the above-mentioned issues, this paper proposes a novel U(PM)$^2$: low-cost unsupervised polygon matching with pre-trained models by uniting automatically learned and handcrafted features, of which pipeline is as follows: firstly, the detector leverages the pre-trained segment anything model to obtain masks; then, the vectorizer converts the masks to polygons and graphic structure; secondly, the global matcher addresses challenges from global viewpoint changes and scale variation based on bidirectional-pyramid strategy with pre-trained LoFTR; finally, the local matcher further overcomes local disparity discontinuity and topology inconsistency of polygon matching by local-joint geometry and multi-feature matching strategy with Hungarian algorithm. We benchmark our U(PM)$^2$ on the ScanNet and SceneFlow datasets using our proposed new metric, which achieved state-of-the-art accuracy at a competitive speed and satisfactory generalization performance at low cost without any training requirement.