research-article

An attention-based deep learning method for solving the cold-start and sparsity issues of recommender systems

Authors:

Narges Heidari,

Abbas KoochariAuthors Info & Claims

Volume 256, Issue C

https://doi.org/10.1016/j.knosys.2022.109835

Published: 28 November 2022 Publication History

Abstract

Matrix Factorization is a successful approach for generating an effective recommender system. However, most existing matrix factorization methods suffer from the sparsity and cold-start issues of the user-item matrix as the primary information source of recommender systems. Besides, they are not much scalable to apply to large real-world applications. A main idea to overcome the cold-start and sparsity issues is to use additional information sources such as user/item profiles or user reviews on items. In this paper, a novel Attention-based Deep Learning Recommender System, so-called ADLRS, is proposed to employ the information sources in the matrix factorization method framework. The proposed method uses a language model to represent contextual information such that important features are effectively embedded. Moreover, a deep autoencoder reduces the dimensionality of item vectors embedded by the language model. Then, these vectors are used as regularization terms in the matrix factorization framework to form an objective function. Then, an iterative algorithm is designed to solve the objective function and provide a method prediction of unknown rating values. Experimental results show that the proposed method achieves superior performance compared to other state-of-the-art ones in most cases. Moreover, the improvement rate for sparse datasets and cold items proves that the proposed method effectively deals with sparsity, cold start and scalability problems.

Highlights

•

An attention-based deep learning recommender system called ADLRS is proposed.

•

ADLRS incorporates profiles of items with the framework of matrix factorization.

•

The BERT language model is used to represents item profiles in the form of vectors.

•

A deep autoencoder is used to extract effective features and reduce the dimensionality of vectors.

•

An iterative method is used to solve the objective function and predict unknown ratings.

References

[1]

Yin R., et al., A deeper graph neural network for recommender systems, Knowl.-Based Syst. 185 (2019).

Digital Library

[2]

Mao M., et al., Multirelational social recommendations via multigraph ranking, IEEE Trans. Cybern. 47 (12) (2016) 4049–4061.

[3]

Ayata D., Yaslan Y., Kamasak M.E., Emotion based music recommendation system using wearable physiological sensors, IEEE Trans. Consum. Electron. 64 (2) (2018) 196–203.

Digital Library

[4]

Alharthi H., Inkpen D., Szpakowicz S., A survey of book recommender systems, J. Intell. Inf. Syst. 51 (1) (2018) 139–160.

[5]

Bobadilla J., et al., Recommender systems survey, Knowl.-Based Syst. 46 (2013) 109–132.

Digital Library

[6]

Stitini O., Kaloun S., Bencharef O., An improved recommender system solution to mitigate the over-specialization problem using genetic algorithms, Electronics 11 (2) (2022) 242.

[7]

Ekstrand M.D., Riedl J.T., Konstan J.A., Collaborative Filtering Recommender Systems, Now Publishers Inc, 2011.

[8]

Parvin H., et al., A scalable and robust trust-based nonnegative matrix factorization recommender using the alternating direction method, Knowl.-Based Syst. 166 (2019) 92–107.

[9]

Aljunid M.F., Huchaiah M.D., An efficient hybrid recommendation model based on collaborative filtering recommender systems, CAAI Trans. Intell. Technol. 6 (4) (2021) 480–492.

Digital Library

[10]

Aditya P., Budi I., Munajat Q., A comparative analysis of memory-based and model-based collaborative filtering on the implementation of recommender system for E-commerce in Indonesia: A case study PT X, in: 2016 International Conference on Advanced Computer Science and Information Systems, ICACSIS, IEEE, 2016.

[11]

Moradi P., et al., A trust-aware recommender algorithm based on users overlapping community structure, in: 2016 Sixteenth International Conference on Advances in ICT for Emerging Regions (ICTer), 2016.

[12]

Ahmadian S., et al., A social recommender system based on reliable implicit relationships, Knowl.-Based Syst. 192 (2020).

[13]

Qiu L., et al., Aspect-based latent factor model by integrating ratings and reviews for recommender system, Knowl.-Based Syst. 110 (2016) 233–243.

[14]

Symeonidis P., Malakoudis D., Multi-modal matrix factorization with side information for recommending massive open online courses, Expert Syst. Appl. 118 (2019) 261–271.

[15]

Basiri M.E., et al., ABCDM: An attention-based bidirectional CNN-rnn deep model for sentiment analysis, Future Gener. Comput. Syst. 115 (2021) 279–294.

Digital Library

[16]

Wang S., et al., An automated CNN recommendation system for image classification tasks, in: 2017 IEEE International Conference on Multimedia and Expo, ICME, IEEE, 2017.

[17]

Wu C., et al., Recurrent neural network based recommendation for time heterogeneous feedback, Knowl.-Based Syst. 109 (2016) 90–103.

[18]

Yi B., et al., Deep matrix factorization with implicit feedback embedding for recommendation system, IEEE Trans. Ind. Inf. 15 (8) (2019) 4591–4601.

[19]

Chen Z., et al., Conditional restricted Boltzmann machine for item recommendation, Neurocomputing 385 (2020) 269–277.

[20]

Vaswani A., et al., Attention is all you need, in: Advances in Neural Information Processing Systems, 2017.

[21]

Zhang S., et al., Next item recommendation with self-attention, 2018, arXiv preprint arXiv:1808.06414.

[22]

Pradhan T., Gupta A., Pal S., Hasvrec: A modularized hierarchical attention-based scholarly venue recommender system, Knowl.-Based Syst. 204 (2020).

[23]

Pang G., et al., ACNN-FM: A novel recommender with attention-based convolutional neural network and factorization machines, Knowl.-Based Syst. 181 (2019).

[24]

Devlin J., et al., Bert: Pre-training of deep bidirectional transformers for language understanding, 2018, arXiv preprint arXiv:1810.04805.

[25]

F. Sun, et al. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019.

[26]

Malkiel I., et al., RecoBERT: A catalog language model for text-based recommendations, 2020, arXiv preprint arXiv:2009.13292.

[27]

Fu Y., Wang T., Item-based collaborative filtering with BERT, in: ECNLP 3, 2020, p. 54.

[28]

Ahmadian S., Moradi P., Akhlaghian F., An improved model of trust-aware recommender systems using reliability measurements, in: 2014 6th Conference on Information and Knowledge Technology, IKT, IEEE, 2014.

[29]

Zhao J., et al., TrustTF: A tensor factorization model using user trust and implicit feedback for context-aware recommender systems, Knowl.-Based Syst. 209 (2020).

[30]

Kiran R., Kumar P., Bhasker B., DNNRec: A novel deep learning based hybrid recommender system, Expert Syst. Appl. 144 (2020).

[31]

Ben-Lhachemi N., Boumhidi J., Hashtag recommender system based on LSTM neural reccurent network, in: 2019 Third International Conference on Intelligent Computing in Data Sciences, ICDS, IEEE, 2019.

[32]

Liu Y., et al., A novel deep hybrid recommender system based on auto-encoder with neural collaborative filtering, Big Data Min. Anal. 1 (3) (2018) 211–221.

[33]

Bathla G., Aggarwal H., Rani R., AutoTrustRec: Recommender system with social trust and deep learning using AutoEncoder, Multimedia Tools Appl. (2020) 1–16.

[34]

H. Parvin, P. Moradi, S. Esmaeili, TCFACO: Trust-aware collaborative filtering method based on ant colony optimization, 118 (2019) 152–168.

[35]

Tahmasbi H., Jalali M., Shakeri H., TSCMF: Temporal and social collective matrix factorization model for recommender systems, J. Intell. Inf. Syst. 56 (1) (2021) 169–187.

[36]

Srifi M., et al., Recommender systems based on collaborative filtering using review texts—a survey, Information 11 (6) (2020) 317.

[37]

Zhang Y., et al., idoctor: Personalized and professionalized medical recommendations based on hybrid matrix factorization, Future Gener. Comput. Syst. 66 (2017) 30–35.

[38]

Sarwar B., et al., Application of Dimensionality Reduction in Recommender System-a Case Study, Minnesota Univ Minneapolis Dept of Computer Science, 2000.

[39]

Koren Y., Bell R., Volinsky C., Matrix factorization techniques for recommender systems, Computer 42 (8) (2009) 30–37.

Digital Library

[40]

Hernando A., Bobadilla J., Ortega F., A non negative matrix factorization for collaborative filtering recommender systems based on a Bayesian probabilistic model, Knowl.-Based Syst. 97 (2016) 188–202.

Digital Library

[41]

Shulong Chen Y.P., Matrix factorization for recommendation with explicit and implicit feedback, Knowl.-Based Syst. 158 (0950–7051) (2018) 109–117.

[42]

Ranjbar M., et al., An imputation-based matrix factorization method for improving accuracy of collaborative filtering systems, Eng. Appl. Artif. Intell. 46 (2015) 58–66.

[43]

Xue G., et al., Dynamic network embedding survey, Neurocomputing 472 (2022) 212–223.

Digital Library

[44]

M. Jamali, M. Ester, A matrix factorization technique with trust propagation for recommendation in social networks, in: Proceedings of the fourth ACM Conference on Recommender Systems, 2010.

[45]

Hwang W.-S., et al., Data imputation using a trust network for recommendation via matrix factorization, Comput. Sci. Inf. Syst. 15 (2) (2018) 347–368.

[46]

Liu D., Ye X., A matrix factorization based dynamic granularity recommendation with three-way decisions, Knowl.-Based Syst. 191 (2020).

Digital Library

[47]

S. Sedhain, et al. Autorec: Autoencoders meet collaborative filtering, in: Proceedings of the 24th International Conference on World Wide Web, 2015.

[48]

F. Strub, R. Gaudel, J. Mary, Hybrid recommender system based on autoencoders, in: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, 2016.

[49]

L. Zheng, V. Noroozi, P.S. Yu, Joint deep modeling of users and items using reviews for recommendation, in: Proceedings of the tenth ACM International Conference on Web Search and Data Mining, 2017.

[50]

Aljunid M.F., Doddaghatta Huchaiah M., Multi-model deep learning approach for collaborative filtering recommendation system, CAAI Trans. Intell. Technol. 5 (4) (2020) 268–275.

[51]

Wu H., et al., Dual-regularized matrix factorization with deep neural networks for recommender systems, Knowl.-Based Syst. 145 (2018) 46–58.

Digital Library

[52]

Logeswaran L., Lee H., An efficient framework for learning sentence representations, 2018, arXiv preprint arXiv:1803.02893.

[53]

Song X., et al., Bi-CLKT: Bi-graph contrastive learning based knowledge tracing, 2018, arXiv preprint arXiv:2201.09020.

[54]

Liu J., et al., A survey on heterogeneous information network based recommender systems: Concepts, methods, applications and resources, AI Open (2022).

[55]

Shi C., et al., A survey of heterogeneous information network analysis, IEEE Trans. Knowl. Data Eng. 29 (1) (2016) 17–37.

Digital Library

[56]

Yang Y., et al., Interpretable and efficient heterogeneous graph convolutional network, IEEE Trans. Knowl. Data Eng. (2021).

[57]

Yang S., et al., Variational co-embedding learning for attributed network clustering, 2021, arXiv preprint arXiv:2104.07295.

[58]

Yin H., et al., Deep fusion of multimodal features for social media retweet time prediction, World Wide Web 24 (4) (2021) 1027–1044.

Digital Library

[59]

J. Tang, K. Wang, Personalized top-n sequential recommendation via convolutional sequence embedding, in: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, 2018.

[60]

B. Hidasi, et al. Parallel recurrent neural network architectures for feature-rich session-based recommendations, in: Proceedings of the 10th ACM Conference on Recommender Systems, 2016.

[61]

Hidasi B., et al., Session-based recommendations with recurrent neural networks, 2015, arXiv preprint arXiv:1511.06939.

[62]

Darban Z.Z., Valipour M.H., GHRS: Graph-based hybrid recommendation system with application to movie recommendation, Expert Syst. Appl. (2022).

[63]

Xu C., et al., Long-and short-term self-attention network for sequential recommendation, Neurocomputing 423 (2021) 580–589.

[64]

Z. Yang, et al. Hierarchical attention networks for document classification, in: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016.

[65]

Xia H., Luo Y., Liu Y., Attention neural collaboration filtering based on GRU for recommender systems, Complex Intell. Syst. 7 (3) (2021) 1367–1379.

[66]

Chou Y.-C., et al., Rating prediction based on merge-CNN and concise attention review mining, IEEE Access 8 (2020).

[67]

Y. Zhu, et al. Query-based interactive recommendation by meta-path and adapted attention-GRU, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019.

[68]

Kang W.-C., McAuley J., Self-attentive sequential recommendation, in: 2018 IEEE International Conference on Data Mining, ICDM, IEEE, 2018.

[69]

Feng F., et al., Language-agnostic bert sentence embedding, 2020, arXiv preprint arXiv:2007.01852.

[70]

Kaviani M., Rahmani H., Emhash: Hashtag recommendation using neural network based on bert embedding, in: 2020 6th International Conference on Web Research, ICWR, IEEE, 2020.

[71]

Zhuang Y., Kim J., A BERT-based multi-criteria recommender system for hotel promotion management, Sustainability 13 (14) (2021) 8039.

[72]

Aizawa A., An information-theoretic perspective of tf–idf measures, Inf. Process. Manage. 39 (1) (2003) 45–65.

Digital Library

[73]

Goldberg Y., Levy O., word2vec explained: deriving Mikolov, et al.’s negative-sampling word-embedding method, 2014, arXiv preprint arXiv:1402.3722.

[74]

Vaswani A., et al., Attention is all you need, Adv. Neural Inf. Process. Syst. (2017) 30.

[75]

Wang W., Zhao M., Wang J., Effective android malware detection with a hybrid model based on deep autoencoder and convolutional neural network, J. Ambient Intell. Humaniz. Comput. 10 (8) (2019) 3035–3043.

[76]

Willmott C.J., Matsuura K., Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance, Clim. Res. 30 (1) (2005) 79–82.

[77]

Aljunid M.F., Huchaiah M.D., IntegrateCF: Integrating explicit and implicit feedback based on deep learning collaborative filtering algorithm, Expert Syst. Appl. 207 (2022).

[78]

Jannach D., et al., What recommenders recommend: an analysis of recommendation biases and possible countermeasures, User Model. User-Adapt. Interact. 25 (5) (2015) 427–491.

Cited By

Devika PMilton A(2025)Book recommendation using sentiment analysis and ensembling hybrid deep learning modelsKnowledge and Information Systems10.1007/s10115-024-02250-z67:2(1131-1168)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s10115-024-02250-z
Karabila IDarraz NEl-Ansari AAlami NEl mallahi M(2024)Enhanced E-commerce Recommender System Based on Deep Learning and Ensemble ApproachesProceedings of the 7th International Conference on Networking, Intelligent Systems and Security10.1145/3659677.3659747(1-8)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3659677.3659747
Yu DZhou XNoorian AHazratifard M(2024)An AI-driven social media recommender system leveraging smartphone and IoT dataThe Journal of Supercomputing10.1007/s11227-024-06722-581:1Online publication date: 10-Dec-2024
https://dl.acm.org/doi/10.1007/s11227-024-06722-5
Show More Cited By

Index Terms

An attention-based deep learning method for solving the cold-start and sparsity issues of recommender systems
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Item cold-start recommendations: learning local collective embeddings
RecSys '14: Proceedings of the 8th ACM Conference on Recommender systems

Recommender systems suggest to users items that they might like (e.g., news articles, songs, movies) and, in doing so, they help users deal with information overload and enjoy a personalized experience. One of the main problems of these systems is the ...
Merging trust in collaborative filtering to alleviate data sparsity and cold start

Providing high quality recommendations is important for e-commerce systems to assist users in making effective selection decisions from a plethora of choices. Collaborative filtering is a widely accepted technique to generate recommendations based on ...
Resolving data sparsity and cold start in recommender systems
UMAP'12: Proceedings of the 20th international conference on User Modeling, Adaptation, and Personalization

Recommender systems (RSs) are heavily used in e-commerce to provide users with high quality, personalized recommendations from a large number of choices. Collaborative filtering (CF) is a widely used technique to generate recommendations [1]. The main ...

Comments

Information & Contributors

Information

Published In

cover image Knowledge-Based Systems

Knowledge-Based Systems Volume 256, Issue C

Nov 2022

755 pages

ISSN:0950-7051

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 28 November 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Devika PMilton A(2025)Book recommendation using sentiment analysis and ensembling hybrid deep learning modelsKnowledge and Information Systems10.1007/s10115-024-02250-z67:2(1131-1168)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s10115-024-02250-z
Karabila IDarraz NEl-Ansari AAlami NEl mallahi M(2024)Enhanced E-commerce Recommender System Based on Deep Learning and Ensemble ApproachesProceedings of the 7th International Conference on Networking, Intelligent Systems and Security10.1145/3659677.3659747(1-8)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3659677.3659747
Yu DZhou XNoorian AHazratifard M(2024)An AI-driven social media recommender system leveraging smartphone and IoT dataThe Journal of Supercomputing10.1007/s11227-024-06722-581:1Online publication date: 10-Dec-2024
https://dl.acm.org/doi/10.1007/s11227-024-06722-5
Devika PMilton A(2024)Book recommendation system: reviewing different techniques and approachesInternational Journal on Digital Libraries10.1007/s00799-024-00403-725:4(803-824)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s00799-024-00403-7
Cai RLu RChen WHao Z(2024)Counterfactual contextual bandit for recommendation under delayed feedbackNeural Computing and Applications10.1007/s00521-024-09800-036:23(14599-14613)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00521-024-09800-0

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents