Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Adversarial Item Promotion on Visually-Aware Recommender Systems by Guided Diffusion

Published: 19 August 2024 Publication History

Abstract

Visually-aware recommender systems have found widespread applications in domains where visual elements significantly contribute to the inference of users’ potential preferences. While the incorporation of visual information holds the promise of enhancing recommendation accuracy and alleviating the cold-start problem, it is essential to point out that the inclusion of item images may introduce substantial security challenges. Some existing works have shown that the item provider can manipulate item exposure rates to its advantage by constructing adversarial images. However, these works cannot reveal the real vulnerability of visually-aware recommender systems because (1) the generated adversarial images are markedly distorted, rendering them easily detected by human observers; and (2) the effectiveness of these attacks is inconsistent and even ineffective in some scenarios or datasets. To shed light on the real vulnerabilities of visually-aware recommender systems when confronted with adversarial images, this article introduces a novel attack method, Item Promotion by Diffusion Generated Image (IPDGI). Specifically, IPDGI employs a guided diffusion model to generate adversarial samples designed to promote the exposure rates of target items (e.g., long-tail items). Taking advantage of accurately modeling benign images’ distribution by diffusion models, the generated adversarial images have high fidelity with original images, ensuring the stealth of our IPDGI. To demonstrate the effectiveness of our proposed methods, we conduct extensive experiments on two commonly used e-commerce recommendation datasets (Amazon Beauty and Amazon Baby) with several typical visually-aware recommender systems. The experimental results show that our attack method significantly improves both the performance of promoting the long-tailed (i.e., unpopular) items and the quality of generated adversarial images.

References

[1]
Christian Bracher, Sebastian Heinz, and Roland Vollgraf. 2016. Fashion DNA: Merging content and sales data for recommendation and article mapping. arXiv:1609.02489. Retrieved from https://doi.org/10.48550/arXiv.1609.02489
[2]
Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 335–344.
[3]
Tong Chen, Hongzhi Yin, Hongxu Chen, Lin Wu, Hao Wang, Xiaofang Zhou, and Xue Li. 2018. Tada: Trend alignment with dual-attention multi-task recurrent neural networks for sales prediction. In Proceedings of the IEEE International Conference on Data Mining (ICDM’18). IEEE, 49–58.
[4]
Yu Cheng, Yunzhu Pan, Jiaqi Zhang, Yongxin Ni, Aixin Sun, and Fajie Yuan. 2023. An image dataset for benchmarking recommender systems with raw pixels. arXiv:2309.06789. Retrieved from https://doi.org/10.48550/arXiv.2309.06789
[5]
Rami Cohen, Oren Sar Shalom, Dietmar Jannach, and Amihood Amir. 2021. A black-box attack model for visually-aware recommender systems. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 94–102.
[6]
Tao Dai, Yan Feng, Dongxian Wu, Bin Chen, Jian Lu, Yong Jiang, and Shu-Tao Xia. 2020. Dipdefend: Deep image prior driven defense against adversarial examples. In Proceedings of the 28th ACM International Conference on Multimedia. 1404–1412.
[7]
Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Siwei Li, Li Chen, Michael E Kounavis, and Duen Horng Chau. 2018. Shield: Fast, practical defense and vaccination for deep learning using jpeg compression. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 196–204.
[8]
Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems 34, 8780–8794.
[9]
Tommaso Di Noia, Daniele Malitesta, and Felice Antonio Merra. 2020. Taamr: Targeted adversarial attack against multimedia recommender systems. In Proceedings of the 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W’20). IEEE, 1–8.
[10]
Hanwen Du, Huanhuan Yuan, Zhen Huang, Pengpeng Zhao, and Xiaofang Zhou. 2023. Sequential recommendation with diffusion models. arXiv:2304.04541. Retrieved from https://doi.org/10.48550/arXiv.2304.04541
[11]
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M. Roy. 2016. A study of the effect of jpg compression on adversarial images. arXiv:1608.00853. Retrieved from https://doi.org/10.48550/arXiv.1608.00853
[12]
Gamaleldin Elsayed, Shreya Shankar, Brian Cheung, Nicolas Papernot, Alexey Kurakin, Ian Goodfellow, and Jascha Sohl-Dickstein. 2018. Adversarial examples that fool both computer vision and time-limited humans. Advances in Neural Information Processing Systems 31, 3910–3920.
[13]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Communications of the ACM 63, 11 (2020), 139–144.
[14]
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv:1412.6572. Retrieved from https://doi.org/10.48550/arXiv.1412.6572
[15]
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. 2017. Countering adversarial images using input transformations. arXiv:1711.00117. Retrieved from https://doi.org/10.48550/arXiv.1711.00117
[16]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[17]
Ruining He and Julian McAuley. 2016. VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
[18]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. 173–182.
[19]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems 30, 25–35.
[20]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33, 6840–6851.
[21]
Vignesh Jagadeesh, Robinson Piramuthu, Anurag Bhardwaj, Wei Di, and Neel Sundaresan. 2014. Large scale visual recommendations from street fashion images. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1925–1934.
[22]
Xiaojun Jia, Xingxing Wei, Xiaochun Cao, and Hassan Foroosh. 2019. Comdefend: An efficient image compression model to defend adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6084–6092.
[23]
Yannis Kalantidis, Lyndon Kennedy, and Li-Jia Li. 2013. Getting the look: Clothing recognition and segmentation for automatic product suggestions in everyday photos. In Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval. 105–112.
[24]
Wang-Cheng Kang, Chen Fang, Zhaowen Wang, and Julian McAuley. 2017. Visually-aware fashion recommendation and design with generative image models. In Proceedings of the IEEE International Conference on Data Mining (ICDM’17). IEEE, 207–216.
[25]
Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https://doi.org/10.48550/arXiv.1412.6980
[26]
Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv:1312.6114. Retrieved from https://doi.org/10.48550/arXiv.1312.6114
[27]
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial Intelligence Safety and Security. Chapman and Hall/CRC, 99–112.
[28]
Chenyi Lei, Dong Liu, Weiping Li, Zheng-Jun Zha, and Houqiang Li. 2016. Comparative deep learning of hybrid representations for image recommendations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2545–2553.
[29]
Yang Li, Tong Chen, Peng-Fei Zhang, and Hongzhi Yin. 2021. Lightweight self-attentive sequential recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 967–977.
[30]
Zihao Li, Aixin Sun, and Chenliang Li. 2023. DiffuRec: A diffusion model for sequential recommendation. arXiv:2304.00686. Retrieved from https://doi.org/10.48550/arXiv.2304.00686
[31]
Zhuoran Liu and Martha Larson. 2021. Adversarial item promotion: Vulnerabilities at the core of top-n recommenders that use images to address cold start. In Proceedings of the Web Conference 2021. 3590–3602.
[32]
Teng Long, Qi Gao, Lili Xu, and Zhangbing Zhou. 2022. A survey on adversarial attacks in computer vision: Taxonomy, visualization and future directions. Computers & Security 121, 102847.
[33]
Måns Magnusson, Michael Andersen, Johan Jonasson, and Aki Vehtari. 2019. Bayesian leave-one-out cross-validation for large data. In International Conference on Machine Learning. PMLR, 4244–4253.
[34]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 43–52.
[35]
Felice Antonio Merra, Vito Walter Anelli, Tommaso Di Noia, Daniele Malitesta, and Alberto Carlo Maria Mancino. 2023. Denoise to protect: A method to robustify visual recommenders from adversaries. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1924–1928.
[36]
James Neve and Ryan McConville. 2020. ImRec: Learning reciprocal preferences using images. In Proceedings of the 14th ACM Conference on Recommender Systems. 170–179.
[37]
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2022. A survey of machine unlearning. arXiv:2209.02299. Retrieved from https://doi.org/10.48550/arXiv.2209.02299
[38]
Thanh Toan Nguyen, Quoc Viet Hung Nguyen, Thanh Tam Nguyen, Thanh Trung Huynh, Thanh Thi Nguyen, Matthias Weidlich, and Hongzhi Yin. 2024. Manipulating recommender systems: A survey of poisoning attacks and countermeasures. arXiv:2404.14942. Retrieved from https://doi.org/10.48550/arXiv.2404.14942
[39]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32, 8026–8037.
[40]
Ruihong Qiu, Jingjing Li, Zi Huang, and Hongzhi Yin. 2019. Rethinking the item order in session-based recommendation with graph neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 579–588.
[41]
Ruihong Qiu, Hongzhi Yin, Zi Huang, and Tong Chen. 2020. Gag: Global attributed graph neural network for streaming session-based recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 669–678.
[42]
Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoc Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. 2023. Semi-decentralized federated ego graph learning for recommendation. In Proceedings of the ACM Web Conference 2023. 339–348.
[43]
Liang Qu, Huaisheng Zhu, Ruiqi Zheng, Yuhui Shi, and Hongzhi Yin. 2021. Imgagn: Imbalanced network embedding via generative adversarial graph networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1390–1398.
[44]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv:1205.2618. Retrieved from https://doi.org/10.48550/arXiv.1205.2618
[45]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 211–252.
[46]
Andrew I. Schein, Alexandrin Popescul, Lyle H. Ungar, and David M. Pennock. 2002. Methods and metrics for cold-start recommendations. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 253–260.
[47]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https://doi.org/10.48550/arXiv.1409.1556
[48]
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning. PMLR, 2256–2265.
[49]
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. 2017. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv:1710.10766. Retrieved from https://doi.org/10.48550/arXiv.1710.10766
[50]
Jinhui Tang, Xiaoyu Du, Xiangnan He, Fajie Yuan, Qi Tian, and Tat-Seng Chua. 2019. Adversarial training towards robust multimedia recommender system. IEEE Transactions on Knowledge and Data Engineering 32, 5 (2019), 855–867.
[51]
Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015. Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proceedings of the IEEE International Conference on Computer Vision. 4642–4650.
[52]
Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, and Tat-Seng Chua. 2023. Diffusion recommender model. arXiv:2304.04971. Retrieved from https://doi.org/10.48550/arXiv.2304.04971
[53]
Lianghao Xia, Chao Huang, Yong Xu, Jiashu Zhao, Dawei Yin, and Jimmy Huang. 2022. Hypergraph contrastive collaborative filtering. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 70–79.
[54]
Weilin Xu, David Evans, and Yanjun Qi. 2017. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv:1704.01155. DOI:
[55]
Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2023. Diffusion models: A comprehensive survey of methods and applications. Computing Surveys 56, 4 (2023), 1–39.
[56]
Hongzhi Yin, Bin Cui, Zi Huang, Weiqing Wang, Xian Wu, and Xiaofang Zhou. 2015. Joint modeling of users’ interests and mobility patterns for point-of-interest recommendation. In Proceedings of the 23rd ACM International Conference on Multimedia. 819–822.
[57]
Hongzhi Yin, Bin Cui, Yizhou Sun, Zhiting Hu, and Ling Chen. 2014. LCARS: A spatial item recommender system. ACM Transactions on Information Systems (TOIS) 32, 3 (2014), 1–37.
[58]
Hongzhi Yin, Liang Qu, Tong Chen, Wei Yuan, Ruiqi Zheng, Jing Long, Xin Xia, Yuhui Shi, and Chengqi Zhang. 2024. On-device recommender systems: A comprehensive survey. arXiv:2401.11441. Retrieved from https://doi.org/10.48550/arXiv.2401.11441
[59]
Minglei Yin, Bin Liu, Neil Zhenqiang Gong, and Xin Li. 2023. Securing visually-aware recommender systems: An adversarial image reconstruction and detection framework. arXiv:2306.07992. Retrieved from https://doi.org/10.48550/arXiv.2306.07992
[60]
Wei Yuan, Quoc Viet Hung Nguyen, Tieke He, Liang Chen, and Hongzhi Yin. 2023a. Manipulating federated recommender systems: poisoning with synthetic users and its countermeasures. arXiv:2304.03054. Retrieved from https://doi.org/10.48550/arXiv.2304.03054
[61]
Wei Yuan, Chaoqun Yang, Liang Qu, Guanhua Ye, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2024. Robust federated contrastive recommender system against model poisoning attack. arXiv:2403.20107. Retrieved from https://doi.org/10.48550/arXiv.2403.20107
[62]
Wei Yuan, Shilong Yuan, Chaoqun Yang, Quoc Viet Hung Nguyen, and Hongzhi Yin. 2023b. Manipulating visually-aware federated recommender systems and its countermeasures. ACM Transactions on Information Systems 42, 3 (2023), 64:1–64:26.
[63]
Xiaohui Zeng, Chenxi Liu, Yu-Siang Wang, Weichao Qiu, Lingxi Xie, Yu-Wing Tai, Chi-Keung Tang, and Alan L. Yuille. 2019. Adversarial attacks beyond the image space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4302–4311.
[64]
Shudong Zhang, Haichang Gao, and Qingxun Rao. 2021a. Defense against adversarial attacks by reconstructing images. IEEE Transactions on Image Processing 30, 6117–6129.
[65]
Shijie Zhang, Hongzhi Yin, Tong Chen, Zi Huang, Lizhen Cui, and Xiangliang Zhang. 2021b. Graph embedding for recommendation against attribute inference attacks. In Proceedings of the Web Conference 2021. 3002–3014.
[66]
Shijie Zhang, Hongzhi Yin, Tong Chen, Zi Huang, Quoc Viet Hung Nguyen, and Lizhen Cui. 2022. Pipattack: Poisoning federated recommender systems for manipulating item promotion. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 1415–1423.
[67]
Wayne Xin Zhao, Zihan Lin, Zhichao Feng, Pengfei Wang, and Ji-Rong Wen. 2022. A revisiting study of appropriate offline evaluation for top-N recommendation algorithms. ACM Transactions on Information Systems 41, 2 (2022), 1–41.
[68]
Ruiqi Zheng, Liang Qu, Tong Chen, Kai Zheng, Yuhui Shi, and Hongzhi Yin. 2024. Poisoning decentralized collaborative recommender system and its countermeasures. arXiv:2404.01177. Retrieved from https://doi.org/10.48550/arXiv.2404.01177
[69]
Ruiqi Zheng, Liang Qu, Bin Cui, Yuhui Shi, and Hongzhi Yin. 2023. Automl for deep recommender systems: A survey. ACM Transactions on Information Systems 41, 4 (2023), 1–38.

Cited By

View all
  • (2024)Domain-aware Multimodal Dialog Systems with Distribution-based User Characteristic ModelingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/370481121:2(1-22)Online publication date: 19-Nov-2024
  • (2024)Fairness and Diversity in Recommender Systems: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/3664928Online publication date: 21-May-2024
  • (2024)PDC-FRS: Privacy-Preserving Data Contribution for Federated Recommender SystemAdvanced Data Mining and Applications10.1007/978-981-96-0850-8_5(65-79)Online publication date: 3-Dec-2024

Index Terms

  1. Adversarial Item Promotion on Visually-Aware Recommender Systems by Guided Diffusion

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Information Systems
    ACM Transactions on Information Systems  Volume 42, Issue 6
    November 2024
    813 pages
    EISSN:1558-2868
    DOI:10.1145/3618085
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 August 2024
    Online AM: 28 May 2024
    Accepted: 22 May 2024
    Revised: 19 March 2024
    Received: 29 December 2023
    Published in TOIS Volume 42, Issue 6

    Check for updates

    Author Tags

    1. Visually-aware recommender system
    2. image poisoning attack
    3. diffusion model

    Qualifiers

    • Research-article

    Funding Sources

    • Australian Research Council
    • Discovery Early Career Researcher Award
    • Discovery Project
    • Industrial Transformation Training Centre

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)419
    • Downloads (Last 6 weeks)94
    Reflects downloads up to 03 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Domain-aware Multimodal Dialog Systems with Distribution-based User Characteristic ModelingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/370481121:2(1-22)Online publication date: 19-Nov-2024
    • (2024)Fairness and Diversity in Recommender Systems: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/3664928Online publication date: 21-May-2024
    • (2024)PDC-FRS: Privacy-Preserving Data Contribution for Federated Recommender SystemAdvanced Data Mining and Applications10.1007/978-981-96-0850-8_5(65-79)Online publication date: 3-Dec-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media