COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decomposition
This work provides the first benchmark for open tree decomposition, enabling hierarchical image segmentation with unconstrained granularity for computer vision researchers.
The paper introduces COCOTree, a large-scale benchmark for open tree-structured visual decomposition, featuring over 21K images and 1.8M structural nodes with 3.5K unique labels. The automated pipeline using LVLMs and SAM 3 achieves strong alignment with human structural judgment.
We formalize and enable the task of open tree decomposition, which segments an image into hierarchical trees of visual components with unconstrained granularity and flexibility. Specifically, we provide the foundation benchmark for this new paradigm with the following three key contributions. First, we overcome the prohibitively high cognitive and physical bottlenecks of manual annotation by developing a fully automated generation pipeline that synergizes the semantic reasoning of Large Vision-Language Models (LVLMs) with the precise geometric grounding of SAM 3. Second, leveraging this pipeline, we construct COCOTree, a massive-scale benchmark featuring over 21K images and 1.8M structural nodes. By embracing an open-vocabulary space of over 3.5K unique labels, it successfully captures the long-tail distribution of complex physical assemblies. Notably, rigorous human evaluation confirms our generated annotations demonstrate strong alignment with human structural judgment. Third, we establish a standardized evaluation protocol by proposing the Open Tree Quality (OTQ) metric, which jointly assesses mask precision, label accuracy, and structural consistency. We release our dataset and benchmark code at https://github.com/melonkick3090/COCOTree.