Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

SPERM: sequential pairwise embedding recommendation with MI-FGSM

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Visual recommendation systems have shown remarkable performance by leveraging consumer feedback and the visual attributes of products. However, recent concerns have arisen regarding the decline in recommendation quality when these systems are subjected to attacks that compromise the model parameters. While the fast gradient sign method (FGSM) and iterative FGSM (I-FGSM) are well-studied attack strategies, the momentum iterative FGSM (MI-FGSM), known for its superiority in the computer vision (CV) domain, has been overlooked. This oversight raises the possibility that visual recommender systems may be vulnerable to MI-FGSM, leading to significant vulnerabilities. Adversarial training, a regularization technique designed to withstand MI-FGSM attacks, could be a promising solution to bolster model resilience. In this research, we introduce MI-FGSM for visual recommendation. We propose the Sequential Pairwise Embedding Recommender with MI-FGSM (SPERM), a model that incorporates visual, temporal, and sequential information for visual recommendations through adversarial training. Specifically, we employ higher-order Markov chains to capture consumers’ sequential behaviors and utilize visual pairwise ranking to discern their visual preferences. To optimize the SPERM model, we employ a learning method based on AdaGrad. Moreover, we fortify the SPERM approach with adversarial training, where the primary objective is to train the model to withstand adversarial inputs introduced by MI-FGSM. Finally, we evaluate the effectiveness of our approach by conducting experiments on three Amazon datasets, comparing it with existing visual and adversarial recommendation algorithms. Our results demonstrate the efficacy of the proposed SPERM model in addressing adversarial attacks while enhancing visual recommendation performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets generated during and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Smith B, Linden G (2017) Two decades of recommender systems at Amazon. IEEE Internet Comput 21(3):12–18

    Article  Google Scholar 

  2. Toledo RY, Mota YC (2014) An e-learning collaborative filtering approach to suggest problems to solve in programming online judges. Int J Distance Educ Technol 12(2):51–65

    Article  Google Scholar 

  3. Lu J, Shambour Q, Xu Y, Lin Q, Zhang G (2013) A web-based personalized business partner recommendation system using fuzzy semantic techniques. Comput Intell 29(1):37–69

    Article  MathSciNet  Google Scholar 

  4. Ning X, Desrosiers C, Karypis G (2015) A comprehensive survey of neighborhood-based recommendation methods. Recommender systems handbook. Springer, Boston, pp 37–76

    Chapter  Google Scholar 

  5. Zhang H, Sun Y, Zhao M, Chow TWS, Wu QMJ (2020) Bridging user interest to item content for recommender systems: an optimization model. IEEE Trans Cybern 50(10):4268–4280

    Article  Google Scholar 

  6. Ricci F, Rokach L, Shapira B (2011) Introduction to recommender systems. Recommender systems handbook. Springer, Boston, pp 1–35

    Chapter  Google Scholar 

  7. Bobadilla J, Ortega F, Hernando A, Gutiérrez A (2013) Recommender systems survey. Knowl Syst 46:109–132

    Article  Google Scholar 

  8. Koren Y, Bell RM, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  9. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian personalized ranking from implicit feedback. In: UAI, pp 452–461

  10. He R, McAuley J (2016) VBPR: visual Bayesian personalized ranking from implicit feedback. In: AAAI, pp 144–150

  11. Yu W, He X, Pei J, Chen X, Xiong L, Liu J, Qin Z (2019) Visually aware recommendation with aesthetic features. VLDB J 30:495–513

    Article  Google Scholar 

  12. He R, McAuley J (2016) Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: WWW, pp 507–517

  13. He R, Fang C, Wang Z, McAuley J (2016) Vista: a visually, socially, and temporally-aware model for artistic recommendation. In: Proceedings of the tenth ACM conference on recommender systems, pp 309-316

  14. Paul A, Wu Z, Liu K, Gong S (2022) Personalized recommendation: from clothing to academic. Multimedia Tools Appl 81(10):14573–14588

    Article  Google Scholar 

  15. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow IJ, Fergus R (2014) Intriguing properties of neural networks. In: ICLR 2014

  16. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: ICLR 2015

  17. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: ICLR 2018

  18. Dong Y, Liao F, Pang T, Su H, Zhu J, Hu X, Li J (2018) Boosting adversarial attacks with momentum. In: Proceedings of IEEE CVPR’18, pp 9185–9193

  19. Carlini N, Wagner DA (2017) Towards evaluating the robustness of neural networks. In: SP, pp 39–57

  20. Shafahi A, Najibi M, Ghiasi A, Xu Z, Dickerson JP, Studer C, Davis LS, Taylor G, Goldstein T (2019) Adversarial training for free!. In: NeurIPS, pp 3353–3364

  21. Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. Comput Sci 14(7):38–39

    Google Scholar 

  22. Papernot N, Mcdaniel P, Wu X, Jha S, Swami A (2016) Distillation as a defense to adversarial perturbations against deep neural networks. In: IEEE symposium on security and privacy, pp 582–597

  23. Miyato T, Maeda S, Koyama M, Ishii S (2018) Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell 41(8):1979–1993

    Article  Google Scholar 

  24. Ren S, He K, Girshick RB, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  25. Yuan Z, Lu Y, Wang Z, Xue Y (2014) Droid-Sec: deep learning android malware detection. In: SIGCOMM, pp 371–372

  26. Hinton GE, Deng L, Yu D, Dahl GE, Mohamed A, Jaitly N, Senior AW, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012) Deep neural networks for acoustic modeling speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  27. Entezari N, Al-Sayouri SA, Darvishzadeh A, Papalexakis EE (2020) All you need is low (rank): defending against adversarial attacks on graphs. In: WSDM, pp 169–177

  28. Tang J, Du X, He X, Yuan F, Tian Q, Chua T (2019) Adversarial training towards robust multimedia recommender system. TKDE 32(5):855–867

    Google Scholar 

  29. Paul A, Wu Z, Liu K, Gong S (2021) Robust multi-objective visual Bayesian personalized ranking for multimedia recommendation. Appl Intell 52:3499–3510

    Article  Google Scholar 

  30. Paul A, Wu Z, Luo K, Ma Y, Fang L (2023) Robust multimedia recommender system based on dynamic collaborative filtering and directed adversarial learning. Int J Mach Learn Cybern 14:1–15

    Article  Google Scholar 

  31. Noia TD, Malitesta D, Merra FA (2020) TAaMR: targeted adversarial attack against multimedia recommender systems. In: DSN-DSML, pp 1–8

  32. He X, He Z, Du X, Chua T (2018) Adversarial personalized ranking for recommendation. In: SIGIR, pp 355–364

  33. Deldjoo Y, Noia TD, Merra FA (2020) Adversarial machine learning in recommender systems (AML-RecSys). In: Thirteenth ACM international conference on web search and data mining, pp 869–872

  34. Xu Y, Chen L, Xie F, Hu W, Zhu J, Chen C, Zheng Z (2020) Directional adversarial training for recommender systems. In: European conference on artificial intelligence, pp 553–560

  35. Rendle S, Freudenthaler C, Schmidt-Thieme L (2010) Factorizing personalized Markov chains for nextbasket recommendation. In: Proceedings of the 19th international conference on world wide web, pp 811–820

  36. Niu W, Caverlee J, Lu H (2018) Neural personalized ranking for image recommendation. In: WSDM, pp 423–431

  37. Kordan SB, Kotov A (2018) Deep neural architecture for multi-modal retrieval based on joint embedding space for text and images. In: WSDM, pp 28–36

  38. Wu Z, Liu Y, Zhang Q, Wu K, Zhang M, Ma S (2019) The influence of image search intents on user behavior and satisfaction. In: WSDM, pp 645–653

  39. Grauman K (2020) Computer vision for fashion: from individual recommendations to world-wide trends. In: WSDM, pp 3

  40. Kang W, Fang C, Wang Z, McAuley J (2017) Visually-aware fashion recommendation and design with generative image models. In: ICDM, pp 207–216

  41. Chu W, Tsai Y (2017) A hybrid recommendation system considering visual information for predicting favorite restaurants. In: WWW, pp 1313–1331

  42. Wang S, Wang Y, Tang J, Shu K, Ranganath S, Liu H (2017) What your images reveal: exploiting visual contents for point-of-interest recommendation. In: WWW, pp 391–400

  43. Zhang Y, Caverlee J (2019) Instagrammers, fashionistas, and me: recurrent fashion recommendation with implicit visual influence. In: CIKM, pp 1583–1592

  44. Mobasher B, Dai H, Luo T, Nakagawa M (2002) Using sequential and non-sequential patterns in predictive web usage mining tasks. In: Proceedings of the IEEE thirteenth international conference on data mining, pp 669–672

  45. Wang S, Zhou X, Wang Z, Zhang M (2012) Please spread: recommending tweets for retweeting with implicit feedback. In: Proceedings of the workshop on data-driven user behavioral modeling and mining from social media, pp 19–22

  46. Biggio B, Corona I, Maiorca D, Nelson B, Srndic N, Laskov P, Giacinto G, Roli F (2013) Evasion attacks against machine learning at test time. In: ECML-PKDD, pp 387–402

  47. Kurakin A, Goodfellow IJ, Bengio S (2017) Adversarial examples the physical world. In: ICLR 2017

  48. Carlini N, Wagner DA (2017) Adversarial examples are not easily detected: bypassing ten detection methods. In: AISec@CCS, pp 3–14

  49. Carlini N, Wagner DA (2016) Defensive distillation is not robust to adversarial examples. CoRR 2016

  50. Lam SK, Riedl J (2004) Shilling recommender systems for fun and profit. In: WWW, pp 393–402

  51. Bhaumik R, Williams C, Mobasher B, Burke R (2006) Securing collaborative filtering against malicious attacks through anomaly detection. In: ITWP 2006

  52. O’Mahony MP, Hurley NJ, Kushmerick N, Silvestre GC (2004) Collaborative recommendation: a robustness analysis. ACM Trans Internet Technol 4(4):344–377

    Article  Google Scholar 

  53. Tang X, Li Y, Sun Y, Yao H, Mitra P, Wang S (2020) Transferring robustness for graph neural network against poisoning attacks. In: WSDM, pp 600–608

  54. He R, Packer C, McAuley J (2016) Learning compatibility across categories for heterogeneous item recommendation. In: Proceedings of the sixteenth IEEE international conference on data mining, pp 937–942

  55. Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE (1991) Adaptive mixtures of local experts. Neural Comput 3(1):79–87

    Article  Google Scholar 

  56. Shani G, Brafman RI, Heckerman D (2002) An MDP-based recommender system. UAI 2002:453–460

    Google Scholar 

  57. Duchi JC, Hazan E, Singer Y (2010) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159

    MathSciNet  Google Scholar 

  58. McAuley J, Targett C, Shi J, Hengel AV (2015) Image-based recommendations on styles and substitutes. In: Proceedings of the thirty eighth international ACM SIGIR conference on research and development in information retrieval, pp 43–52

  59. Kang W, McAuley J (2018) Self-attentive sequential recommendation. In: 2018 IEEE international conference on data mining (ICDM), pp 197–206

Download references

Acknowledgements

This research was supported by Zhejiang Provincial Natural Science Foundation of China under Grant No. LZ22F010005.

Author information

Authors and Affiliations

  1. College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, People’s Republic of China

    Agyemang Paul, Yuxuan Wan, Boyu Chen & Zhefu Wu

Authors
  1. Agyemang Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxuan Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Boyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhefu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhefu Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no known conflict of interest or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Paul, A., Wan, Y., Chen, B. et al. SPERM: sequential pairwise embedding recommendation with MI-FGSM. Int. J. Mach. Learn. & Cyber. (2024). https://doi.org/10.1007/s13042-024-02288-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13042-024-02288-z

Keywords

Search

Navigation