Improving Human-Machine Cooperative Visual Search With Soft Highlighting
This addresses the challenge of enhancing visual search tasks like satellite or medical imagery analysis where human expertise is still essential, though it appears incremental compared to existing highlighting methods.
The paper tackled the problem of improving human-machine cooperative visual search by proposing a soft highlighting technique that modulates saliency based on classifier confidence, and reported experiments showing it achieves better performance synergy than hard highlighting.
Advances in machine learning have produced systems that attain human-level performance on certain visual tasks, e.g., object identification. Nonetheless, other tasks requiring visual expertise are unlikely to be entrusted to machines for some time, e.g., satellite and medical imagery analysis. We describe a human-machine cooperative approach to visual search, the aim of which is to outperform either human or machine acting alone. The traditional route to augmenting human performance with automatic classifiers is to draw boxes around regions of an image deemed likely to contain a target. Human experts typically reject this type of hard highlighting. We propose instead a soft highlighting technique in which the saliency of regions of the visual field is modulated in a graded fashion based on classifier confidence level. We report on experiments with both synthetic and natural images showing that soft highlighting achieves a performance synergy surpassing that attained by hard highlighting.