Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Improving recommendation diversity without retraining from scratch

  • Regular Paper
  • Published:
International Journal of Data Science and Analytics Aims and scope Submit manuscript

Abstract

Diverse recommendations strongly correlate with increased sales diversity, perceived ease of use, and general user satisfaction with recommendation systems. However, many recommendation models focus only on maximizing recommendation accuracy. This can lead to a lack of diversity due to the feedback loop of recommending already popular items. Retraining the entire model to increase diversity can be expensive, time-consuming, and impractical. To address this, we propose a refinement strategy that uses reinforcement learning objectives to penalize non-diverse behavior. This allows us to improve the diversity of any pre-trained model without retraining it from scratch and needing the original training settings and labels. We evaluate our approach using three deep learning recommendation models on the Yoochoose, RetailRocket, and Movielens datasets. Our refinement scheme improves recommendation diversity by up to \(5\%\) while maintaining competitive recommendation ranking performance in metrics such as HitRate and NDCG.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Find the latest articles, discoveries, and news in related topics.

Notes

  1. Running the Wilcoxon signed-rank test [61] on the results and considering p-value \(< 0.05\) to be statistically significant.

References

  1. Linden, G., Smith, B., York, J.: Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003). https://doi.org/10.1109/MIC.2003.1167344

    Article  Google Scholar 

  2. Zheng, G., Zhang, F., Zheng, Z., Xiang, Y., Yuan, N.J., Xie, X.: Drn: a deep reinforcement learning framework for news recommendation. In: Proceedings of the 27th International World Wide Web Conference (2018)

  3. Covington, P., Adams, J., Sargin, E.: Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems (2016)

  4. Oord, A., Dieleman, S., Schrauwen, B.: Deep content-based music recommendation. In: Proceedings of the 27th Conference on Neural Information Processing Systems (2013)

  5. Himeur, Y., Alsalemi, A., Al-Kababji, A., Bensaali, F., Amira, A., Sardianos, C., Dimitrakopoulos, G., Varlamis, I.: A survey of recommender systems for energy efficiency in buildings: principles, challenges and prospects. Inform. Fusion (2021). https://doi.org/10.1016/j.inffus.2021.02.002

    Article  Google Scholar 

  6. Sayed, A., Himeur, Y., Alsalemi, A., Bensaali, F., Amira, A.: Intelligent edge-based recommender system for internet of energy applications. IEEE Syst. J. 16(3), 5001–5010 (2022). https://doi.org/10.1109/JSYST.2021.3124793

    Article  ADS  Google Scholar 

  7. Fleder, D., Hosanagar, K.: Recommender systems and their impact on sales diversity. In: Proceedings of the 8th ACM Conference on Electronic Commerce (2007)

  8. Hu, R., Pu, P.: Helping users perceive recommendation diversity. In: Proceedings of the 5th ACM Conference on Recommender Systems (2011)

  9. Shipard, J., Wiliem, A., Thanh, K.N., Xiang, W., Fookes, C.: Diversity is definitely needed: Improving model-agnostic zero-shot classification via stable diffusion. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 769–778. IEEE Computer Society, Los Alamitos, CA, USA (2023). https://doi.org/10.1109/CVPRW59228.2023.00084 . https://doi.ieeecomputersociety.org/10.1109/CVPRW59228.2023.00084

  10. Abdollahpouri, H., Burke, R., Mobasher, B.: Managing popularity bias in recommender systems with personalized re-ranking. In: Proceedings of the 33rd AAAI Florida Artificial Intelligence Research Society Conference (2019)

  11. Castells, P., Wang, J., Lara, R., Zhang, D.: Workshop on novelty and diversity in recommender systems. In: Proceedings of the 5th ACM Conference on Recommender Systems (2011)

  12. Adomavicius, G., Kwon, Y.: Maximizing aggregate recommendation diversity: a graph-theoretic approach. In: Proceedings of the 5th ACM Conference on Recommender Systems (2011)

  13. Mourão, F., Fonseca, C., Araújo, C., Jr., W.M.: The oblivion problem: Exploiting forgotten items to improve recommendation diversity. In: Proceedings of the 5th ACM Conference on Recommender Systems (2011)

  14. Stamenkovic, D., Karatzoglou, A., Arapakis, I., Xin, X., , Katevas, K.: Choosing the best of both worlds: Diverse and novel recommendations through multi-objective reinforcement learning. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining (2021)

  15. Lops, P., Jannach, D., Musto, C., Bogers, T., Koolen, M.: Trends in content-based recommendation. User Model. User-Adap. Inter. 29(2), 239–249 (2019)

    Article  Google Scholar 

  16. Zhang, R., Liu, Q.-D., Chun-Gui, Wei, J.-X., Jia-Xuan, Huiyi-Ma: Collaborative filtering for recommender systems. In: Proceedings of the 2nd International Conference on Advanced Cloud and Big Data (2014)

  17. Wang, S., Hu, L., Wang, Y., Cao, L., Sheng, Q.Z., Orgun1, M.: Sequential recommender systems: Challenges, progress and prospects. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence (2019)

  18. Sedhain, S., Menon, A.K., Sanner, S., Xie, L.: Autorec: Autoencoders meet collaborative filtering. In: Proceedings of the 24th International Conference on World Wide Web (2015)

  19. Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management (2018)

  20. Zhang, R., Liu, Q.-d., Chun-Gui, Wei, J.-X., Jia-Xuan, Huiyi-Ma: Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (2019)

  21. Wang-Cheng Kang, J.M.: Self-attentive sequential recommendation. In: Proceedings of the 18th IEEE International Conference on Data Mining (2018)

  22. Xin, X., Karatzoglou, A., Arapakis, I., Jose, J.: Self-supervised reinforcement learning for recommender systems. In: Proceedings of the 43th International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)

  23. Zou, L., Xia, L., Ding, Z., Song, J., Liu, W., Yin, D.: Reinforcement learning to optimize long-term user engagement in recommender systems. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2019)

  24. Xin, X., Pimentel, T., Karatzoglou, A., Ren, P., Christakopoulou, K., Ren., Z.: Rethinking reinforcement learning for recommendation: A prompt perspective. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)

  25. Xin, X., Karatzoglou, A., Arapakis, I., Jose., J.M.: Supervised advantage actor-critic for recommender systems. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining (2021)

  26. Ji, L., Qi, Q., Han, B., Yang, H.: Reinforcement learning to optimize lifetime value in cold-start recommendation. In: Proceedings of the 30th ACM International Conference on Information and Knowledge Management (2021)

  27. Shani, G., Heckerman, D., Brafman, R.I.: An mdp-based recommender system. J. Mach. Learn. Res. (2005)

  28. Walke, H.R., Yang, J.H., Yu, A., Kumar, A., Orbik, J., Singh, A., Levine, S.: Don’t start from scratch: leveraging prior data to automate robotic reinforcement learning. In: Proceedings of the 6th Conference on Robot Learning (2022)

  29. Sardianos, C., Varlamis, I., Chronis, C., Dimitrakopoulos, G., Alsalemi, A., Himeur, Y., Bensaali, F., Amira, A.: The emergence of explainability of intelligent systems: delivering explainable and personalized recommendations for energy efficiency. Int. J. Int. Syst. (2020). https://doi.org/10.1002/int.22314

    Article  Google Scholar 

  30. Himeur, Y., Sohail, S.S., Bensaali, F., Amira, A., Alazab, M.: Latest trends of security and privacy in recommender systems: a comprehensive review and future perspectives. Comput. Secur. 118, 102746 (2022). https://doi.org/10.1016/j.cose.2022.102746

    Article  Google Scholar 

  31. Himeur, Y., Sayed, A., Alsalemi, A., Bensaali, F., Amira, A., Varlamis, I., Eirinaki, M., Sardianos, C., Dimitrakopoulos, G.: Blockchain-based recommender systems: applications, challenges and future opportunities. Comput. Sci. Rev. (2022). https://doi.org/10.1016/j.cosrev.2021.100439

    Article  MathSciNet  Google Scholar 

  32. Carbonell, J., Goldstein, J.: The use of mmr, diversity-based reranking for reordering documents and producing summaries. In: Proceesdings of the He 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (1998)

  33. Noia, T.D., Ostuni, V.C., Rosati, J., Tomeo, P., Sciascio, E.D.: An analysis of users’ propensity toward diversity in recommendations. In: Proceedings of the 8th ACM Conference on Recommender Systems (2014)

  34. Chen, L., Zhang, G., Zhou, H.: Improving the diversity of top-n recommendation via determinantal point process. In: Large Scale Recommendation Systems Workshop at the 11th ACM Conference on Recommender Systems (2017)

  35. Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural networks for facial expression recognition. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (2015)

  36. Yin, X., Chen, W., Wu, X., Yue, H.: Fine-tuning and visualization of convolutional neural networks. In: Proceedings of the 12th IEEE Conference on Industrial Electronics and Applications (2017)

  37. Kojima, T., Matsuo, Y., Iwasawa, Y.: Robustifying vision transformer without retraining from scratch by test-time class-conditional feature alignment. In: Raedt, L.D. (ed.) Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, pp. 1009–1016. International Joint Conferences on Artificial Intelligence Organization, (2022). https://doi.org/10.24963/ijcai.2022/141 . Main Track

  38. Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (2018)

  39. Jaques, N., Gu, S., Turner, R.E., Eck, D.: Generating music by fine-tuning recurrent neural networks with reinforcement learning. In: Proceedings of the 30th Conference on Neural Information Processing Systems (2016)

  40. Jaques, N., Gu, S., Bahdanau, D., Hernandez, J.M., Turner, L.R.E., Eck, D.: Tuning recurrent neural networks with reinforcement learning. In: Proceedings of the 5th International Conference on Learning Representations (2017)

  41. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Gray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., Lowe, R.: Training language models to follow instructions with human feedback. In: Proceedings of the 36th Conference on Neural Information Processing Systems (2022)

  42. Wei, T., Feng, F., Chen, J., Wu, Z., Yi, J., He, X.: Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2021)

  43. Liu, S., Zheng, Y.: Long-tail session-based recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems (2020)

  44. Yoon-Joo Park, A.T.: The long tail of recommender systems and how to leverage it. In: Proceedings of the 2nd ACM Conference on Recommender Systems (2008)

  45. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, USA (2018). http://incompleteideas.net/book/the-book-2nd.html

  46. Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: Proceedings of the 20th International Conference on Neural Information Processing Systems (2007)

  47. Bellman, R.: Dynamic Programming. Dover Publications, USA (1957). https://doi.org/10.1515/9781400835386

  48. Ben-Shimon, D., Tsikinovsky, A., Friedmann, M., Shapira, B., Rokach, L., Hoerle, J.: Recsys challenge 2015 and the yoochoose dataset. In: Proceedings of the 9th ACM Conference on Recommender Systems (2015)

  49. Zykov, R., Artem, N., Alexander, A.: RetailRocket Recommender System Dataset. Kaggle (2022). https://doi.org/10.34740/KAGGLE/DSV/4471234 . https://www.kaggle.com/dsv/4471234

  50. Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Transact. Interact. Intelli. Syst. 5(4), 1–19 (2015). https://doi.org/10.1145/2827872

    Article  Google Scholar 

  51. Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (2019)

  52. Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., Ma, J.: Neural attentive session-based recommendation. In: Proceedings of the 26th ACM Conference on Information and Knowledge Management (2017)

  53. Liu, Q., Zeng, Y., Mokhosi, R., Zhang, H.: Stamp: Short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24TH ACM Conference on Knowledge Discovery and Data Mining (2018)

  54. Yu, F., Zhu, Y., Liu, Q., Wu, S., Wang, L., Tan, T.: Tagnn: Target attentive graph neural networks for session-based recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)

  55. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (2015)

  56. Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D.: Session-based recommendations with recurrent neural networks. In: Proceedings of the 4th International Conference on Learning Representations (2016)

  57. Song, K., Ji, M., Park, S., Moon, I.-C.: Hierarchical context enabled recurrent neural network for recommendation. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence (2019)

  58. Qu, S., Yuan, F., Guo, G., Zhang, L., Wei, W.: Cmnrec: sequential recommendations with chunk-accelerated memory network. IEEE Trans. Knowl. Data Eng. 35(4), 3540–3550 (2023). https://doi.org/10.1109/TKDE.2022.3141102

    Article  Google Scholar 

  59. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature (2015)

  60. Gini, C.: On the Measure of Concentration with Special Reference to Income and Statistics. Colorado College Publication, USA (1936)

    Google Scholar 

  61. Wilcoxon, F.: In: Kotz, S., Johnson, N.L. (eds.) Individual Comparisons by Ranking Methods, pp. 196–202. Springer, New York, NY (1992). https://doi.org/10.1007/978-1-4612-4380-9_16

Download references

Author information

Authors and Affiliations

  1. Aboitiz School of Innovation, Technology and Entrepreneurship, Asian Institute of Management, 123 Paseo de Roxas, Makati, 1229, Metro Manila, Philippines

    Joseph Noel, Christopher Monterola & Daniel Stanley Tan

  2. Faculty of Science, Open Universiteit Nederland, Valkenburgerweg 177, Heerlen, 6401 DL, Limburg, The Netherlands

    Daniel Stanley Tan

Authors
  1. Joseph Noel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Monterola
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Stanley Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors did not receive support from any organization for the submitted work.

Corresponding author

Correspondence to Joseph Noel.

Ethics declarations

Conflict of interest

The authors also have no Conflict of interest to declare relevant to this article’s content.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Noel, J., Monterola, C. & Tan, D.S. Improving recommendation diversity without retraining from scratch. Int J Data Sci Anal (2024). https://doi.org/10.1007/s41060-024-00518-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41060-024-00518-9

Keywords

Search

Navigation