Which Are You In A Photo?
This addresses the challenge of accurate image tagging for mobile users, offering a novel sensing-based approach that is incremental over traditional image recognition methods.
The paper tackles the problem of automatic image tagging by using mobile phone acoustic sensing to identify and correlate people in photos as they are taken, achieving over 85% accuracy for tag correlation within 3 meters.
Automatic image tagging has been a long standing problem, it mainly relies on image recognition techniques of which the accuracy is still not satisfying. This paper attempts to explore out-of-band sensing base on the mobile phone to sense the people in a picture while the picture is being taken and create name tags on-the-fly. The major challenges pertain to two aspects - "Who" and "Which". (1) "Who": discriminating people who are in the picture from those that are not; (2) "Which": correlating each name tag with its corresponding people in the picture. We propose an accurate acoustic scheme applying on the mobile phones, which leverages the Doppler effect of sound wave to address these two challenges. As a proof of concept, we implement the scheme on 7 android phones and take pictures in various real-life scenarios with people positioning in different ways. Extensive experiments show that the accuracy of tag correlation is above 85% within 3m for picturing.