HCFormer: Unified Image Segmentation with Hierarchical Clustering
This work addresses the need for interpretable and unified architectures in image segmentation for computer vision researchers and practitioners, though it is incremental as it builds on hierarchical clustering methods.
The paper tackles the problem of complex and non-interpretable neural network architectures for image segmentation by proposing HCFormer, a simpler and more interpretable model based on hierarchical clustering. HCFormer achieves comparable or superior accuracy on semantic segmentation (55.5 mIoU on ADE20K), instance segmentation (47.1 AP on COCO), and panoptic segmentation (55.7 PQ on COCO).
Hierarchical clustering is an effective and efficient approach widely used for classical image segmentation methods. However, many existing methods using neural networks generate segmentation masks directly from per-pixel features, complicating the architecture design and degrading the interpretability. In this work, we propose a simpler, more interpretable architecture, called HCFormer. HCFormer accomplishes image segmentation by bottom-up hierarchical clustering and allows us to interpret, visualize, and evaluate the intermediate results as hierarchical clustering results. HCFormer can address semantic, instance, and panoptic segmentation with the same architecture because the pixel clustering is a common approach for various image segmentation tasks. In experiments, HCFormer achieves comparable or superior segmentation accuracy compared to baseline methods on semantic segmentation (55.5 mIoU on ADE20K), instance segmentation (47.1 AP on COCO), and panoptic segmentation (55.7 PQ on COCO).