You See What I Want You To See: Exploring Targeted Black-Box Transferability Attack for Hash-Based Image Retrieval Systems

Document Type

Conference Paper

Publication Date


Publication Title

Proceedings of the IEEE/CVF Conference on the Computer Vision and Pattern Recognition (CVPR), 2021



Conference Name

IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021, June 19-25, 2021, Virtual


With the large multimedia content online, deep hashing has become a popular method for efficient image retrieval and storage. However, by inheriting the algorithmic backend from softmax classification, these techniques are vulnerable to the well-known adversarial examples as well. The massive collection of online images into the database also opens up new attack vectors. Attackers can embed adversarial images into the database and target specific categories to be retrieved by user queries. In this paper, we start from an adversarial standpoint to explore and enhance the capacity of targeted black-box transferability attack for deep hashing. We motivate this work by a series of empirical studies to see the unique challenges in image retrieval. We study the relations between adversarial subspace and black-box transferability via utilizing random noise as a proxy. Then we develop a new attack that is simultaneously adversarial and robust to noise to enhance transferability. Our experimental results demonstrate about 1.2-3x improvements of black-box transferability compared with the state-of-the-art mechanisms.


The CVPR 2021 paper is the Open Access version, provided by the Computer Vision Foundation. The final published version of the proceedings is in press.

Original Publication Citation

Xiao, Y., & Wang, C. (2021). You see what I want you to see: Exploring targeted black-box transferability attack for hash-based image retrieval systems. IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021 June 19-25, 2021, Virtual. https://openaccess.thecvf.com/content/CVPR2021/html/Xiao_You_See_What_I_Want_You_To_See_Exploring_Targeted_CVPR_2021_paper.html