AIMS: All-Inclusive Multi-Level Segmentation
This addresses the need for multi-level segmentation in image editing applications, though it appears incremental as it builds on existing segmentation work with a unified multi-task approach.
The paper tackles the problem of diverse region-of-interest selections in image editing by proposing a new task called All-Inclusive Multi-Level Segmentation (AIMS), which segments images into part, entity, and relation levels, and demonstrates its effectiveness through experiments showing generalization capacity compared to state-of-the-art methods.
Despite the progress of image segmentation for accurate visual entity segmentation, completing the diverse requirements of image editing applications for different-level region-of-interest selections remains unsolved. In this paper, we propose a new task, All-Inclusive Multi-Level Segmentation (AIMS), which segments visual regions into three levels: part, entity, and relation (two entities with some semantic relationships). We also build a unified AIMS model through multi-dataset multi-task training to address the two major challenges of annotation inconsistency and task correlation. Specifically, we propose task complementarity, association, and prompt mask encoder for three-level predictions. Extensive experiments demonstrate the effectiveness and generalization capacity of our method compared to other state-of-the-art methods on a single dataset or the concurrent work on segmenting anything. We will make our code and training model publicly available.