Targeted Mismatch Adversarial Attack: Query with a Flower to Retrieve the Tower
This addresses privacy concerns for users of online visual search engines by enabling query concealment, though it is incremental as it builds on existing adversarial attack methods.
The paper tackles the problem of privacy in visual search by introducing a targeted mismatch attack that generates adversarial images to conceal user queries, achieving identical or very similar retrieval results while making the images look nothing like the intended query.
Access to online visual search engines implies sharing of private user content - the query images. We introduce the concept of targeted mismatch attack for deep learning based retrieval systems to generate an adversarial image to conceal the query image. The generated image looks nothing like the user intended query, but leads to identical or very similar retrieval results. Transferring attacks to fully unseen networks is challenging. We show successful attacks to partially unknown systems, by designing various loss functions for the adversarial image construction. These include loss functions, for example, for unknown global pooling operation or unknown input resolution by the retrieval system. We evaluate the attacks on standard retrieval benchmarks and compare the results retrieved with the original and adversarial image.