Predicting How to Distribute Work Between Algorithms and Humans to Segment an Image Batch
This work addresses the practical challenge of optimizing human-computer collaboration in image segmentation for users needing reliable results, though it is incremental in its application of resource allocation to existing segmentation tasks.
The paper tackles the problem of efficiently allocating a fixed budget of human annotation effort to improve segmentation quality in image batches by predicting when to use humans versus algorithms for coarse and fine-grained segmentations, showing that a mixed approach outperforms using either resource alone across three diverse image modalities.
Foreground object segmentation is a critical step for many image analysis tasks. While automated methods can produce high-quality results, their failures disappoint users in need of practical solutions. We propose a resource allocation framework for predicting how best to allocate a fixed budget of human annotation effort in order to collect higher quality segmentations for a given batch of images and automated methods. The framework is based on a prediction module that estimates the quality of given algorithm-drawn segmentations. We demonstrate the value of the framework for two novel tasks related to predicting how to distribute annotation efforts between algorithms and humans. Specifically, we develop two systems that automatically decide, for a batch of images, when to recruit humans versus computers to create 1) coarse segmentations required to initialize segmentation tools and 2) final, fine-grained segmentations. Experiments demonstrate the advantage of relying on a mix of human and computer efforts over relying on either resource alone for segmenting objects in images coming from three diverse modalities (visible, phase contrast microscopy, and fluorescence microscopy).