BYOCL: Build Your Own Consistent Latent with Hierarchical Representative Latent Clustering
This addresses the problem of inconsistent segmentation across image sequences for users of foundation models, offering a plug-and-play solution without training, though it appears incremental as it builds on existing models like SAM.
The paper tackles the semantic inconsistency issue in single-image segmentation models like SAM when handling image sequences by introducing BYOCL, which outperforms SAM in experiments and reduces time and space consumption through batch processing.
To address the semantic inconsistency issue with SAM or other single-image segmentation models handling image sequences, we introduce BYOCL. This novel model outperforms SAM in extensive experiments, showcasing its Hierarchical prototype capabilities across CLIP and other representations. BYOCL significantly reduces time and space consumption by dividing inputs into smaller batches, achieving exponential time reduction compared to previous methods. Our approach leverages the SAM image encoder for feature extraction, followed by Intra-Batch and Inter-Batch clustering algorithms. Extensive experiments demonstrate that BYOCL far exceeds the previous state-of-the-art single image segmentation model. Our work is the first to apply consistent segmentation using foundation models without requiring training, utilizing plug-and-play modules for any latent space, making our method highly efficientModels are available at \href{https://github.com/cyt1202/BYOCL.git