I.J. Intelligent Systems and Applications, 2019, 5, 46-54 Published Online May 2019 in MECS (http://www.mecs-press.org/) DOI: 10.5815/ijisa.2019.05.06 An Application-oriented Review of Deep Learning in Recommender Systems Jyoti Shokeen Research Scholar, Department of CSE, UIET, M.D. University, Rohtak, 124001, India E-mail: jyotishokeen12@gmail.com Dr. Chhavi Rana Assistant Professor, Department of CSE, UIET, M.D. University, Rohtak, 124001, India E-mail: chhavi1jan@yahoo.com Received: 02 May 2018; Accepted: 21 June 2018; Published: 08 May 2019 Abstract—The development in technology has gifted huge set of alternatives. In the modern era, it is difficult to select relevant items and information from the large amount of available data. Recommender systems have been proved helpful in choosing relevant items. Several algorithms for recommender systems have been proposed in previous years. But recommender systems implementing these algorithms suffer from various challenges. Deep learning is proved successful in speech recognition, image processing and object detection. In recent years, deep learning has been also proved effective in handling information overload and recommending items. This paper gives a brief overview of various deep learning techniques and their implementation in recommender systems for various applications. The increasing research in recommender systems using deep learning proves the success of deep learning techniques over traditional methods of recommender systems. Index Terms—Recommender system, Deep learning, Collaborative filtering, Deep neural network, Social recommender system. learning methods have been extensively used in various applications like object detection [3], playing games [4], automatic text generation for images [5], image processing [6], inclusion of sound in silent movies [7], music recommendation [8,9], etc. An increasing use of deep learning techniques in generating meaningful recommendations can be seen in recent years. A workshop on "Deep Learning for Recommender Systems" in conjunction with an international conference "RecSys" from year 2016 reveals the significance and proliferation of deep learning in RS. We can realize its significance by the huge number of publications in this area. The main motivation behind this research is to reveal the advancement in the field of RS. We present an outline of the state-of-the art research in this area. The rest of this paper is structured in the following style. Section II gives the overview of RS and the traditional technique for RSs. Section III discusses the concept of deep learning and its techniques. Section IV surveys the related work of deep learning in RSs in different domains. Section V gives a few works done in social recommender system employing deep learning. Lastly, Section VI concludes the paper. I. INTRODUCTION In the modern age where vast amount of data is generated every second, it becomes difficult to select relevant items from the overwhelming set of choices. Many of the times it is difficult to reach a decision without having prior knowledge about the items. The result is that people rely on the advices or recommendations of their friends or some expert. Recommender systems (RSs) have been proved helpful in handling information overload [1]. RSs are important information filtering tools in recommending relevant, interested and striking items to users. Typically, a RS compares the user profile with profile of similar users or it use the past history or behavior of users to recommend items [2]. The rating matrix is used to determine the preferences of user for an item. A number of techniques have been proposed for RSs but deep learning is a new research area in recommendation. In previous years, deep Copyright © 2019 MECS II. RECOMMENDER SYSTEMS Recommender systems are the information retrieval systems which recommend only selective items from the enormous collection of items. Thus, RSs are essential in reducing the problem of information overload. RSs are vital tools in promoting sales in e-commerce websites [10]. They aim to recommend items such as movies, music, books, events, products, etc. According to a survey report, about 60% of videos are watched using the recommendations by YouTube and 80% of movies watched is recommended by Netflix. Reference [2] presents the evolution of RS into three generations, namely web 1.0, web 2.0 and web 3.0. Traditional algorithms for designing RSs are mainly of three types: collaborative filtering, content-based filtering and hybrid algorithms. In collaborative filtering based algorithms [11], the systems find the similar users and use their I.J. Intelligent Systems and Applications, 2019, 5, 46-54 An Application-oriented Review of Deep Learning in Recommender Systems tastes and preferences to recommend items to the customers. Collaborative filtering based algorithms have been extensively used in traditional RSs. Collaborative filtering approaches can be further of two types: userbased and item-based approaches [12]. In user-based collaborative filtering approach, the users with same preferences are grouped and recommendation to any item is given by evaluating the preferences of that group. On the other hand, item-based approach assumes that the preference of people remain stable or drift very slightly. In content-based filtering algorithms, the system searches for the similar type of contents or items by using the past behavior of user and then recommend the similar contents to the user. However, the hybrid algorithms combine the features of collaborative filtering and content-based algorithms to give recommendations to the user. The hybrid techniques may use some features of collaborative filtering approach into content-based filtering approach or it may use the features of content-based filtering approach into collaborative filtering approach. But these approaches face various issues such as cold-start, data sparsity, privacy [13], scalability and trust [14]. A good RS takes into account the changing system’s contents and the dynamic users’ preferences. Rana and Jain [76] explore various parameters which are important for the dynamism in RS. These parameters are: serendipity, novelty, temporal characteristic, context, dynamic environment and diversity. On the other hand, Shokeen and Rana [77, 79] explore the dynamics involved for the success of RSs that are based on social networking data. Such RSs are termed as social recommender systems. They also give a review on different aspects of these systems. These aspects are: tagging, communities, trusted relations, communities and cross-domain knowledge. III. DEEP LEARNING Deep learning is a subfield of machine learning. It is a technique that learns features directly from data. The computation models in deep learning consist of several processing layers to learn data representation [15]. The deep learning models learn high level features from lowlevel features where data can take any form from text to audio and images. Unlike other neural network methods, deep learning networks consist of two or more than two layers in which features from previous layers are aggregated to build more complex features in the next layers. The deep learning based methods can be trained using either supervised or unsupervised learning approaches. The main motivation behind the proliferation of deep learning models is their accuracy, easy adaption to new problems, enormous amount of available data and less time to train GPUs. Fig. 1 demonstrates deep neural network consisting of an input layer, hidden layers and an output layer. As the number of layers increase in these networks, the data representation becomes more complex. Copyright © 2019 MECS 47 Fig.1. Deep Neural Network When large amount of data is available for training, deep learning models give best results as compared to non-deep learning methods. Fig. 2 shows a diagram of any active node to be used in hidden layers and output layer of deep learning models. In this figure, the set of inputs to a neuron or node j is denoted by {x1, x2, x3,….,xn} and weights of the inputs for neuron j is represented by {w1j, w2j,….,wnj}. ∑ denotes the transfer function that multiplies each input with its weight and then sums it. We denote this sum by netj for the neuron j. The activation function f(S) performs computation of netj and if the value is greater than threshold θ, it gives output oj for neuron j. In literature, a diverse number of deep learning techniques have been used for recommendation. These techniques include convolutional neural networks [8] [16], deep neural networks [17,18], deep belief networks, deep autoencoders [19-21], multiperceptron layer [22], restricted boltzmann machine [23,24] and recurrent neural networks [25,26]. Some of the authors have used a single deep learning technique to build the recommendation model while others have integrated two or more than two techniques for generating recommendations. Also, some RSs are based on the hybrid models which are based on the integration of deep learning methods with traditional methods of RSs [22] [27-29]. An advantage of using multiple deep learning methods for model construction is that one technique can counteract the effects of other technique. This hybridization can generate an efficient model. Fig.2. Architecture of an Active Node I.J. Intelligent Systems and Applications, 2019, 5, 46-54 48 An Application-oriented Review of Deep Learning in Recommender Systems A. Deep Learning Techniques In this subsection, we also give a brief explanation of some of the deep learning techniques. These techniques are as follows: Convolutional neural network: A network with convolutional layers and pooling operations is termed as convolutional neural network. This technique increases the efficiency of network by capturing both local and global features. This network performs better in grid-like networks. Deep neural network: A deep neural network is similar to an artificial neural network in terms of structure but differs in terms of layers. Deep neural networks encompass multiple hidden layers that can model complex relationships. A deep neural network is multi-layer perceptron that uses backpropagation to learn the network. Stacked denoising autoencoders is one of the approaches used in building deep neural networks [30]. Wang et al. [19] proposed relational stacked deep autoencoder (RSDAE), which is a probabilistic model, to integrate deep representation learning and relational learning. It has been shown through real-world datasets that RSDAE model outperforms the state-of-the-art. Deep belief network: Deep belief networks (DBN) are generative models consisting of complex layers of latent and stochastic variables. The latent variables are generally termed as feature detectors. There are symmetric and undirected connections between the upper two layers whereas directed connections exist at the lower layers [31]. The learning in deep belief nets occur layer-by-layer where the data values from one layer are used to train the next layer and so on. Restricted Boltzmann machine: A restricted Boltzmann machine (RBM) is an artificial neural network that learns the probability distribution through inputs. On the basis of the nature of task, they can be supervised or unsupervised. RBMs are the modified form of Boltzmann machine, with the constraint that there exists bipartite graph between neurons. RBMs plays an important role in dimensionality reduction [32], collaborative filtering [23] and classification [33]. On stacking RBMs, deep belief networks are formed. Recently, authors in [75] have attempted to minimize the long training time of deep neural networks by modifying RBM. Deep Autoencoders: Deep autoencoders is a special variety of deep neural networks and follows unsupervised learning approach. A deep autoencoder comprises two consistent DBN containing usually five layers. The first half refers to the encoding part of DBN and the second half is the decoding DBN. Deep autoencoders are non-linear autoencoders as Copyright © 2019 MECS the layers in deep autoencoders are restricted Boltzmann machines [34]. In initialization stage, the data is processed through multiple layers of restricted Boltzmann machines. The processing allows deep auto encoders to abstract high-dimensional data from latent features [35]. An extended form of stacked autoencoders is stacked denoising autoencoders. Recurrent neural networks: Recurrent neural networks are dynamical, feedbackward systems that consist of memory to retain previous computations. These networks are useful in modeling sequential data. IV. DEEP LEARNING IN RECOMMENDER SYSTEMS In literature, a number of algorithms and techniques have been proposed to build RSs. It is interesting to note that collaborative filtering is the widely used technique in RSs. But most of these algorithms face data sparsity and cold-start problem issues. Recently, deep learning based techniques implementation RSs have proved helpful in alleviating these issues. It becomes easy to predict null rating values with deep learning [36]. Some deep learning-based RSs have also incorporated information from social networks to improve the recommendations. With time, some methods have been modified and revised. A comprehensive survey of RSs using deep learning has been made by many researchers. It is essential to say that collaborative filtering is the winning approach employed by many RSs. But in real-world datasets, the sparsity in data leads to decreasing the performance of the collaborative filtering approach. Many researchers have combined collaborative filtering approach with deep learning methods to resolve the issues related to data sparseness and cold-start problem [9, 19, 27, 28]. Collaborative deep learning has given more successful results when compared to traditional collaborative filtering techniques [37, 38]. With deep learning, it is easy to predict the null rating values [36]. Deep collaborative filtering has also been tested for movie recommendation [37]. [29] have proposed a collaborative deep learning model that uses deep learning to retrieve the textual information and collaborative filtering to obtain the feedback from the ratings matrix. [27, 39] employ deep neural networks to explore item content characteristics and then apply these characteristics into the timeSVD++ collaborative filtering model. Cheng et al. [40] have proposed a wide and deep learning framework to integrate the features of deep neural networks and linear models. In this framework, wide linear models are used to learn sparse features connections whereas deep neural networks are used to generalize earlier hidden features. The framework successfully utilizes the capabilities of both models to train the network. Many attempts have been made for introducing tags related information to RS to enhance the performance of conventional RS. But user-defined tags undergo various challenges such as ambiguity, redundancy and sparsity. I.J. Intelligent Systems and Applications, 2019, 5, 46-54 An Application-oriented Review of Deep Learning in Recommender Systems To deal with these challenges, deep neural networks extract the information from tags and process the information through multiple layers to retrieve more advanced and abstract data [41]. Authors in [42] have provided a very short survey of deep learning methods used in RSs. Zhang et al. [43] present a comprehensive review of deep learning techniques based RSs. A few recent works in this area which have been included in this section have not been covered in the past survey. A. Entertainment For entertainment, researchers have deployed deep learning techniques for music recommendation. Wang and Wang [9] have worked in this area and proposed a content based recommendation model that combines deep belief network and probabilistic graph models. The model is used in training network to learn features. This model works better than the traditional hybrid models. Oord et al. [8] have used deep convolutional neural networks to predict the hidden features in musical audio when they cannot be obtained directly from usage data. Liang et al. [44] proposed a recommendation system based on a content model and a collaborative model. The system pretrains a multi-layer neural network on semantic tagging data and treats it to extract the content features. The highlevel representation generated by the last hidden layer of network is used as a prior in the collaborative model for the latent representation of songs. Recently, researchers use heterogeneous information to improve the quality of RSs. Zhang et al. [45] have investigated in this direction and proposed a collaborative knowledge base embedding framework to learn hidden and semantic representation simultaneously. They have used denoising auto-encoders and convolutional auto-encoders to mine semantic data from multiple forms of data. They have designed components to mine semantic data from multiple forms of data. Denoising auto-encoders extract textual data representations and then convolutional auto-encoders are used to mine visual representations. Recently, deep neural architecture is employed by Oramas et al. [18] to propose a multimodal approach for song recommendation. The approach combines audio and text information with user feedback information using deep neural networks. B. Cross-domain Nowadays, users have multiple accounts on social networks. The data from multiple sites assist RS in reducing the cold-start problem [46–51]. Authors in [46] have worked in this direction to integrate cross-domain data. They have used multi-view deep neural networks to integrate the data from multiple social media sites to enhance the quality of RS. They have also suggested various dimensionality reduction methods to scale their framework to large datasets. The dimensionality reduction methods are: top features, k-means, local sensitive hashing and reduction in number of training examples. Huang et al. [52] focus on retrieving images by extracting the clothing attributes from cross-domains. They propose a Dual Attribute-aware Ranking Network (DARN) to learn the retrieved features. With DARN, it is Copyright © 2019 MECS 49 easy to integrate semantic attributes with visual similarity attributes into the feature learning phase. The system retrieves the clothing images similar to the given offline clothing images. In another work, [22] use multi-view neural networks and propose a cross-domain RS to overcome the data sparsity problem. Recently, Khan et al. [51] give a broad survey of cross-domain RSs. C. Medicine Recently, researchers have also started applying deep learning in health care domain. The processing of medical data with deep learning methods is useful in boosting the prediction power of health care systems. The use of deep learning methods in clinical datasets can help in treatment recommendation, personalized prescription, disease risk prediction and clinical dataset analysis. [53] have presented a review of deep learning techniques in clinical imaging, genomics, electronic heart records and wearable device data. Manogaran et al. [54] propose a deep learning method using Adaptive Neuro-Fuzzy Inference System (ANFIS) and Multi Kernel Learning. They have used Multi Kernel Learning to segregate features of heart disease patients and healthy people. Their approach aims to solve unsupervised learning problems. They have incorporated ANFIS method that uses adaptive and nonadaptive nodes. Yuan et al. [55] give a deep learningbased socialized RS to recommends healthcare services to users based on the trust and distrust relationships with the target user. The system also considers the structure information and the node information of users in the network. The entire information is then fused into a deep learning-based model. This model is a multilayer perceptron and measures the trust strength by assessing the weights of features. The model gives the reliable healthcare recommendations even for the cold-start users. Katzman et al. [56] propose a deep neural network-based RS called DeepSurv for personalized treatment recommendation. D. Industrial With the excessive sparsity in features of real-world datasets, it is challenging to scale the features to meet the industrial requirements. One way to handle the problem of data sparsity in industry is to implement deep learning. Covington et al. [57] revealed the huge impact of deep learning on YouTube video recommendations. They propose a deep neural architecture based RS that divides the task of recommendation into two phases: candidate generation and ranking. The candidate generation model is used to select a subset of videos from the video corpus. Then, the ranking model uses the nearest neighbor score of the selected candidates to build a set of top-n videos. Chen et al. [58] have proposed a locally connected deep learning framework to deal with huge industrial datasets. Their framework condenses the model size by transforming sparse data into dense data. The reduction in model size ultimately produces effective results in less time. I.J. Intelligent Systems and Applications, 2019, 5, 46-54 50 An Application-oriented Review of Deep Learning in Recommender Systems E. Images Deep learning is the widely used technique in image processing. Yang et al. [59] have used the deep learning approach for personalized image recommendations in which visual data and user's preferences over images are used to learn hybrid representation. This is achieved by designing a dual-net deep network where input images and user's preferences are first mapped into a same latent semantic space and then the decision is made by calculating the distance between images and users. They have also proposed a comparative deep learning (CDL) technique to train the deep network. CDL is based on the idea that the distance between a user and a positive image is less than the distance between the user and a negative image. The results demonstrate that this technique is better than other approaches in image recommendations. A more recent work shows that images related with an individual attempts to generate visual user interest profile which acts as a foundation for both recommendation and optimization. Zhou et al. [60] employ deep learning to extract semantic information from the visual content. F. Miscellaneous Various application areas are associated with textual recommendations such as scientific paper recommendations, blog posts and news articles which require text to be recommended to users. For this, Bansal et al. [61] propose a method leveraging deep neural networks to give vector demonstration for the textual content. Gradient descent is directly used to train the text- to-vector mapping, thereby offering chance to execute multi-task learning. Authors in [62] have recently developed personalized scientific research paper RS using recurrent neural networks. This technique effectively searches hidden semantic features of research papers as compared to the conventional bag-of-word techniques. Recently, Xu et al. [63] address the uncontrolled vocabulary issue of social tags through deep neural network approach. In this approach, the tag-based user and item profiles are mapped to more abstract deep features. They propose a deep-semantic similarity-based personalized model that consists of many hidden layers due to which it becomes more time-consuming and expensive to train large online recommendation system. To alleviate this problem, they propose a hybrid deep learning model that integrates autoencoders with the above model. The model outperforms the existing techniques in personalized recommendations. Currently, deep learning is also being used in favorite restaurant prediction [64], online news RS [20], generation of user interest profile [60], session-based recommendation [26, 78]. Recurrent neural networks are recently used in session recommenders. Recurrent neural networks in [26] utilizes the past session data of users to improve the recommendations before the session starts. It shows promising results in dealing with the cold-start problem. In this paper, we have taken a dataset of 45 papers specifically concerned with deep learning in RSs. Table 1 classifies these papers on the basis of different applications. We also list the type of deep learning method used in designing RS for these applications. Table 1. Classification of papers on the basis of applications Applications Entertainment Cross-domain Medicine Industrial Images Miscellaneous Copyright © 2019 MECS References Wang and Wang [9] Van de Oord et al. [8] Liang et al. [44] Zhang et al. [45] Oramas et al. [18] Elkahky et al. [46] Huang et al. [52] Lian et al. [22] Manogaran et al. [54] Yuan et al. [55] Katzman et al. [56] Covington et al. [57] Chen et al. [58] Lei et al. [65] Zhou et al. [60] Bansal et al. [61] Hassan [62] Zhenghua Xu [63] Jia et al. [24] Chu and Tsai [64] Cao et al. [20] Ruocco et al. [26] Quadrana et al. [78] Type of Model Deep belief network Deep convolutional network Multilayer perceptron Denoising autoencoder, Convolutional autoencoder Deep neural network Deep neural network Deep convolutional network Multilayer perceptron Deep neural network Multilayer perceptron Deep neural network Deep neural network Multilayer perceptron Convolutional neural network and Multilayer perceptron Convolutional neural network Deep neural network and Multilayer perceptron Recurrent neural network Autoencoder Restricted Boltzmann Machine Convolutional neural network Autoencoder Recurrent neural network Recurrent neural network I.J. Intelligent Systems and Applications, 2019, 5, 46-54 An Application-oriented Review of Deep Learning in Recommender Systems 51 VI. CONCLUSION V. DEEP LEARNING IN SOCIAL RECOMMENDER SYSTEMS The combination of social networks with RS is termed as social recommender system [79]. Social recommender systems employ the data from social networking sites and thus improve the recommendations [80]. These recommendation systems are more capable than traditional RSs. The user trust information further helps in improving the recommendations. Jia et al. [24] combined knowledge from different social networks to ascertain the correlation between the online information and event participation. They propose a model called CLER (Collaborative Learning Approach for Event Recommendation) that considers both the similarity between events and users and the feature descriptions. Deng et al. [35] propose DLMF (Deep learning based matrix factorization) approach for social network trust based recommendations. They use deep encoders to train the initial values and hidden features of users and items. Final latent features of items and users are learned by minimizing the objective function. Also, privacy is one of the issues in social recommender systems as these systems rely on user personal details. To address this problem, Dang et al. [66] propose a rating prediction approach called "dTrust" that exploits the topology of trust-user-item network. This approach uses deep feedforward neural network to combine user relations and user-item ratings to predict ratings. To learn user features from large, sparse and diverse social networks, [67] propose DUIF (Deep User-Image Feature) as a deep learning framework to give useful recommendations. This framework employs user and images features to evaluate similarities between them. More recently, Wang et al. [68] use deep neural networks to propose neural collaborative social ranking (NCSR) method to integrate user-item interactions from information domains and user relations from social domain for cross-domain social recommendation. They consider the users with multiple accounts on social networks as silk routes to recommend items from information domain to social users. Table 2 shows the period wise statistics of these papers from year 2013 to 2017. It is clear from the table that deep learning is extensively used in recent years to enhance the performance of RSs. Moreover, deep learning has also opened the doors for improving the accuracy of social recommender systems. Table 2. Period wise statistics of papers on deep learning based RS Period 2013 2014 2015 #Papers 1 2 5 2016 17 2017 20 Copyright © 2019 MECS #Paper References [8] [9], [29] [19], [37], [44], [46], [52] [16], [21], [24], [25], [28], [36], [40], [41], [45], [47], [57], [60], [61], [69], [70], [65], [71] [17], [18], [20], [22], [26], [27], [38], [42], [43], [51], [53], [62]– [64], [66], [68], [72]–[74], [78] A voluminous research has been done and is also proceeding in recommender systems using deep learning. It is an active research area where profound work is still required to advance the process of learning. We have used a dataset of 45 papers specifically concerned with deep learning in recommender systems that validates its exploration in recent years. Studies on dimensionality reduction could be a remarkable future work in enhancing the proficiency of RSs. This can be achieved by importing rich ideas from other spheres of machine learning. Also, the recent growth in social networks is beneficial in improving the capabilities of recommender systems. In the future work, we would like to compare the performance of some deep learning-based RSs with non-deep learning based RSs. REFERENCES J. A. Konstan and J. Riedl, “Recommender systems : from algorithms to user experience,” User Modeling and UserAdaped Interaction, vol. 22, no. 1-2, pp. 101–123, 2012. DOI: 10.1007/s11257-011-9112-x. [2] R. Singh, K. Chuchra, and A. Rani, “A Survey on the Generation of Recommender Systems,” International Journal of Information Engineering and Electronic Business, vol. 9, no. 3, pp. 26–35, 2017. DOI: 10.5815/ijieeb.2017.03.04. [3] C. Szegedy, A. Toshev, and D. Erhan, “Deep Neural Networks for Object Detection,” in Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013, pp. 2553–2561. [4] V. Mnih et al., “Playing Atari with Deep Reinforcement Learning,” in arXiv:1312.5602, 2013, pp. 1–9. [5] I. Sutskever, J. Martens, and G. E. Hinton, “Generating Text with Recurrent Neural Networks,” in Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, pp. 1017–1024. [6] A. Karpathy and L. Fei-Fei, “Deep Visual-Semantic Alignments for Generating Image Descriptions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 664–676, 2017. DOI: 10.1109/TPAMI.2016.2598339. [7] A. Owens, A. Torralba, P. Isola, E. H. Adelson, J. Mcdermott, and W. T. Freeman, “Visually Indicated Sounds,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2405–2413. [8] A. van den Oord, S. Dieleman, and B. Schrauwen, “Deep content-based music recommendation,” in Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013, pp. 2643–2651. [9] X. Wang and Y. Wang, “Improving Content-based and Hybrid Music Recommendation using Deep Learning,” Proceedings of ACM International Conference on Multimedia, pp. 627–636, 2014. DOI: 10.1145/2647868.2654940. [10] B. Kumar and N. Sharma, “Approaches, Issues and Challenges in Recommender Systems: A Systematic Review,” Indian Journal of Science and Technology, vol. 9, no. 47, 2016. DOI: 10.17485/ijst/2016/v9i47/94892. [11] J. Ben Schafer, D. Frankowski, J. Herlocker, and S. Sen, “Collaborative filtering recommender systems,” in The Adaptive Web, P. Brusilovsky, A. Kobsa, and W. Nejdl, Eds. Springer-Verlag, 2007, pp. 291–324. [1] I.J. Intelligent Systems and Applications, 2019, 5, 46-54 52 An Application-oriented Review of Deep Learning in Recommender Systems [12] J. Shokeen, “A Comparison of Collaborative Filteringbased Recommender Systems,” Journal of Emerging Technologies and Innovative Research, vol. 5, no. 4, pp. 868–871, 2018. [13] A. Friedman, B. P. Knijnenburg, K. Vanhecke, L. Martens, and S. Berkovsky, “Privacy Aspects of Recommender Systems,” in Recommender Systems Handbook, Springer, 2015, pp. 649–688. [14] J. Shokeen and C. Rana, “A study on Trust-aware Social Recommender Systems,” in Proceedings of the 12th INDIACom and 5th International Conference on Computing for Sustainable Global Development, 2018, pp. 4268–4272. [15] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015. DOI:10.1038/nature14539. [16] Y. S. Rawat and M. S. Kankanhalli, “ConTagNet : Exploiting User Context for Image Tag Recommendation,” in Proceedings of the 2016 ACM on Multimedia Conference, pp. 1102–1106, 2016. DOI: 10.1145/2964284.2984068. [17] V. Kumar, D. Khattar, S. Gupta, and M. Gupta, “Deep Neural Architecture for News Recommendation,” in Working Notes of the 8th International Conference of the CLEF Initiative, Dublin, Ireland. CEUR Workshop Proceedings, 2017. [18] S. Oramas, O. Nieto, M. Sordo, and X. Serra, “A Deep Multimodal Approach for Cold-start Music Recommendation,” in Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems, 2017. DOI: 10.1145/3125486.3125492. [19] H. Wang, X. Shi, and D.-Y. Yeung, “Relational Stacked Denoising Autoencoder for Tag Recommendation,” in Proceedings of the 29th AAAI Conference on Artificial Intelligence, 2015, pp. 3052–3058. [20] S. Cao, N. Yang, and Z. Liu, “Online news recommender based on stacked auto-encoder,” in 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), 2017, pp. 721–726. DOI: 10.1109/ICIS.2017.7960088. [21] Y. Wu, C. Dubois, A. X. Zheng, and M. Ester, “Collaborative Denoising Auto-Encoders for Top-N Recommender Systems,” in Proceedings of the Ninth ACM International Conference on Web Search and Data Mining, 2016, pp. 153–162. DOI: 10.1145/2835776.2835837. [22] J. Lian, F. Zhang, X. Xie, and G. Sun, “CCCFNet: A Content-Boosted Collaborative Filtering Neural Network for Cross Domain Recommender Systems,” in Proceedings of the 26th International Conference on World Wide Web Companion, 2017, pp. 817–818. DOI: 10.1145/3041021.3054207. [23] R. Salakhutdinov, A. Mnih, and G. Hinton, “Restricted Boltzmann machines for collaborative filtering,” in Proceedings of the 24th International Conference on Machine Learning, 2007, pp. 791–798. DOI:10.1145/1273496.1273596. [24] X. Jia, X. Li, K. Li, V. Gopalakrishnan, G. Xun, and A. Zhang, “Collaborative restricted Boltzmann machine for social event recommendation,” in 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2016, pp. 402–405. DOI: 10.1109/ASONAM.2016.7752265. [25] B. Hidasi, M. Quadrana, A. Karatzoglou, and D. Tikk, “Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations,” in Proceedings of the 10th ACM Conference on Copyright © 2019 MECS [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] Recommender Systems, 2016, pp. 241–248. DOI: 10.1145/2959100.2959167. M. Ruocco, O. S. L. Skrede, and H. Langseth, “InterSession Modeling for Session-Based Recommendation,” in Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems, 2017, pp. 24–31. DOI: 10.1145/3125486.3125491. J. Wei, J. He, K. Chen, Y. Zhou, and Z. Tang, “Collaborative filtering and deep learning based recommendation system for cold start items,” Expert Systems and Applications, vol. 69, pp. 29–39, 2017. DOI: 10.1016/j.eswa.2016.09.040. H. Ying, L. Chen, Y. Xiong, and J. Wu, “Collaborative Deep Ranking: A Hybrid Pair-Wise Recommendation Algorithm with Implicit Feedback,” in Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2016, vol. 9652, pp. 555–567, https://doi.org/10.1007/978-3-319-31750-2_44 H. Wang, N. Wang, and D.-Y. Yeung, “Collaborative Deep Learning for Recommender Systems,” in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 1235–1244. DOI: 10.1145/2783258.2783273. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, “Stacked Denoising Autoencoders : Learning Useful Representations in a Deep Network with a Local Denoising Criterion,” The Journal of Machine Learning Research, vol. 11, pp. 3371–3408, 2010. G. E. Hinton, S. Osindero, and Y.-W. Teh, “A Fast Learning Algorithm for Deep Belief Nets,” Neural Computation, vol. 18, no. 7, pp. 1527–1554, 2006. DOI: 10.1162/neco.2006.18.7.1527 G. E. Hinton and R. R. Salakhutdinov, “Reducing the Dimensionality of Data with Neural Networks,” Science, vol. 313, no. 5786, pp. 504–507, 2006. DOI: 10.1126/science.1127647 H. Larochelle and Y. Bengio, “Classification using discriminative restricted Boltzmann machines,” in Proceedings of the 25th International Conference on Machine Learning, 2008, pp. 536–543. DOI:10.1145/1390156.1390224 P. Baldi, “Autoencoders, Unsupervised Learning , and Deep Architectures,” in Proceedings of ICML Workshop on Unsupervised and Transfer Learning, vol. 27, I. Guyon, G. Dror, V. Lemaire, G. Taylor, and D. Silver, Eds. Bellevue, Washington, USA: PMLR, 2012, pp. 37–50. S. Deng, L. Huang, G. Xu, X. Wu, and Z. Wu, “On Deep Learning for Trust-Aware Recommendations in Social Networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 5, pp. 1164–1177, 2016. DOI: 10.1109/TNNLS.2016.2514368 T. Gao, X. Li, Y. Chai, and Y. Tang, “Deep learning with consumer preferences for recommender system,” in 2016 IEEE International Conference on Information and Automation (ICIA), 2016, pp. 1556–1561. DOI: 10.1109/ICInfA.2016.7832066 S. Li, J. Kawale, and Y. Fu, “Deep Collaborative Filtering via Marginalized Denoising Auto-encoder,” in Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, 2015, pp. 811–820. DOI: 10.1145/2806416.2806527 M. Oggretir and A. T. Cemgil, “Comparison of collaborative deep learning and nonnegative matrix factorization for recommender systems,” in 2017 25th Signal Processing and Communications Applications Conference (SIU), 2017. DOI: 10.1109/SIU.2017.7960695 I.J. Intelligent Systems and Applications, 2019, 5, 46-54 An Application-oriented Review of Deep Learning in Recommender Systems [39] J. Wei, J. He, K. Chen, Y. Zhou, and Z. Tang, “Collaborative Filtering and Deep Learning Based Hybrid Recommendation for Cold Start Problem,” 2016 IEEE 14th Intl Conf Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence Computing and Cyber Science and Technology Congress, pp. 874–877, 2016. [40] H.-T. Cheng et al., “Wide & Deep Learning for Recommender Systems,” in Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, 2016, pp. 7–10. DOI: 10.1145/2988450.2988454 [41] Y. Zuo, J. Zeng, M. Gong, and L. Jiao, “Tag-aware recommender systems based on deep neural networks,” Neurocomputing, vol. 204, no. C, pp. 51–60, 2016. DOI: 10.1016/j.neucom.2015.10.134 [42] B. T. Betru, C. A. Onana, and B. Batchakui, “Deep Learning Methods on Recommender System : A Survey of State-of-the-art,” International Journal of Computer Applications, vol. 162, no. 10, pp. 17–22, 2017. DOI: 10.5120/ijca2017913361 [43] S. Zhang, L. Yao, and A. Sun, “Deep Learning based Recommender System: A Survey and New Perspectives,” arXiv:1707.07435, pp. 1–35, 2017. [44] D. Liang, M. Zhan, and D. P. W. Ellis, “Content-Aware Collaborative Music Recommendation Using Pre-Trained Neural Networks,” in ISMIR, 2015, pp. 295–301. [45] F. Zhang, N. J. Yuan, D. Lian, X. Xie, and W.-Y. Ma, “Collaborative Knowledge Base Embedding for Recommender Systems,” in Proceeding of the 22nd ACM SIGKDD International Conference on Knowledge Discovery And Data Mining, 2016, pp. 353–362. DOI: 10.1145/2939672.2939673 [46] A. M. Elkahky, Y. Song, and X. He, “A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems,” in Proceedings of the 24th International Conference on World Wide Web, 2015, pp. 278–288. DOI:10.1145/2736277.2741667 [47] V. Kumar, K. M. P. D. Shrivastva, and S. Singh, “Cross Domain Recommendation Using Semantic Similarity and Tensor Decomposition,” Procedia Computer Science, vol. 85, pp. 317–324, 2016. https://doi.org/10.1016/j.procs.201605.239 [48] R. Zafarani and H. Liu, “Connecting Users across Social Media Sites : A Behavioral-Modeling Approach,” in Proceedings of the 19th ACM SIGKDD International Conference on Knowledge discovery and data mining, 2013, pp. 41–49. DOI: 10.1145/2487575.2487648 [49] J. Zhang, Z. Yuan, and K. Yu, “Cross Media Recommendation in Digital Library,” in International Conference on Asian Digital Libraries, 2014, pp. 208–217. https://doi.org/10.1007/978-3-319-12823-8_21 [50] X. Liu, T. Xia, Y. Yu, C. Guo, and Y. Sun, “Cross Social Media Recommendation,” in Proceedings of the Tenth International AAAI Conference on Web and Social Media (ICWSM 2016), 2016, pp. 221–230. [51] M. M. Khan, R. Ibrahim, and I. Ghani, “Cross Domain Recommender Systems: A Systematic Literature Review,” ACM Computing Surveys, vol. 50, no. 3, pp. 1–34, 2017. DOI: 10.1145/3073565 [52] J. Huang, R. Feris, Q. Chen, and S. Yan, “Cross-domain image retrieval with a dual attribute-aware ranking network,” in Proceedings of the 2015 IEEE International Conference on Computer Vision, 2015, pp. 1062–1070. DOI: 10.1109/ICCV.2015.127 [53] R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, “Deep learning for healthcare: review , opportunities and Copyright © 2019 MECS [54] [55] [56] [57] [58] [59] [60] [61] [62] [63] [64] [65] [66] [67] 53 challenges,” Briefings in Bioinformatics, pp. 1–11, 2017. DOI: 10.1093/bib/bbx044 G. Manogaran, R. Varatharajan, and M. K. Priyan, “Hybrid Recommendation System for Heart Disease Diagnosis based on Multiple Kernel Learning with Adaptive Neuro-Fuzzy Inference System,” Multimedia Tools and Applications, vol. 77, no. 4, pp. 4379–4399, 2018. https://doi.org/10.1007/s11042-017-5515-y W. Yuan, C. Li, D. Guan, and G. Han, “Socialized healthcare service recommendation using deep learning,” Neural Computing and Applications, vol. 4, pp.1-12, 2018. https://doi.org/10.1007/s00521-018-3394-4 J. Katzman, U. Shaham, J. Bates, A. Cloninger, T. Jiang, and Y. Kluger, “DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network,” BMC Medical Research Methodology, vol. 18, no.1, pp. 1–12, 2018. DOI: 10.1186/s12874-018-0482-1 P. Covington, J. Adams, and E. Sargin, “Deep neural networks for youtube recommendations,” in Proceedings of the 10th ACM Conference on Recommender Systems, 2016, pp. 191-198. DOI: 10.1145/2959100.2959190 C. Chen, P. Zhao, L. Li, J. Zhou, X. Li, and M. Qiu, “Locally Connected Deep Learning Framework for Industrial-scale Recommender Systems,” in Proceedings of the 26th International Conference on World Wide Web Companion, 2017, pp. 769–770, DOI: 10.1145/3041021.3054227 B. Yang, Y. Lei, J. Liu, and W. Li, “Social Collaborative Filtering by Trust,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, vol. 39, no. 8, pp. 1633–1647. DOI: 10.1109/TPAMI.2016.2605085 J. Zhou, R. Albatal, and C. Gurrin, “Applying visual user interest profiles for recommendation and personalisation,” in International Conference on Multimedia Modeling, 2016, pp. 361–366. https://doi.org/10.1007/978-3-31927674-8_34 T. Bansal, D. Belanger, and A. McCallum, “Ask the GRU: Multi-task Learning for Deep Text Recommendations,” in Proceedings of the 10th ACM Conference on Recommender Systems, 2016, pp. 107–114. http://dx.doi.org/10.1145/2959100.2959180 H. A. M. Hassan, “Personalized Research Paper Recommendation using Deep Learning,” in Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP '17), 2017, pp. 327–330. DOI: 10.1145/3079628.3079708 Z. Xu, T. Lukasiewicz, C. Chen, Y. Miao, and X. Meng, “Tag-Aware Personalized Recommendation Using a Hybrid Deep Model,” in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, 2017, pp. 3196–3202. DOI: 10.24963/ijcai.2017/446 W.-T. Chu and Y.-L. Tsai, “A hybrid recommendation system considering visual information for predicting favorite restaurants,” World Wide Web, vol. 20, no. 6, pp.1313-1331, 2017. https://doi.org/10.1007/s11280-0170437-1 C. Lei, D. Liu, W. Li, Z. Zha, and H. Li, “Comparative Deep Learning of Hybrid Representations for Image Recommendations,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2545–2553. DOI: 10.1109/CVPR.2016.279 Q. Dang, C. Ignat, and Q. Dang, “dTrust : a simple deep learning approach for social recommendation,” in 3rd IEEE International Conference on Collaboration and Internet Computing, 2017. DOI: 10.1109/CIC.2017.00036 X. Geng, H. Zhang, J. Bian, and T. S. Chua, “Learning I.J. Intelligent Systems and Applications, 2019, 5, 46-54 54 [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] An Application-oriented Review of Deep Learning in Recommender Systems image and user features for recommendation in social networks,” in 2015 IEEE International Conference on Computer Vision, pp. 4274–4282, 2015. DOI: 10.1109/ICCV.2015.486 X. Wang, X. He, L. Nie, and T.-S. Chua, “Item Silk Road: Recommending Items from Information Domains to Social Users,” in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017, pp. 185-194. J. Wei, J. He, K. Chen, Y. Zhou, and Z. Tang, “Collaborative Filtering and Deep Learning Based Hybrid Recommendation for Cold Start Problem,” in 14th International Conference on Dependable, Autonomic and Secure Computing, 14th International Conference on Pervasive Intelligence and Computing, 2nd International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congre, 2016, pp. 874– 877. Z. Xu, C. Chen, and T. Lukasiewicz, “Tag-Aware Personalized Recommendation Using a Deep-Semantic Similarity Model with Negative Sampling,” in Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 2016, pp. 1921–1924. DOI: 10.1145/2983323.2983874 F. Strub, R. Gaudel, and J. Mary, “Hybrid Recommender System based on Autoencoders,” in Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, 2016, pp. 11–16. DOI: 10.1145/2988450.2988456 G. Trigeorgis, K. Bousmalis, S. Zafeiriou, and B. W. Schuller, “A deep matrix factorization method for learning attribute representations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 3, pp. 417–429, 2017. DOI: 10.1109/TPAMI.2016.2554555 X. Dong, L. Yu, Z. Wu, Y. Sun, L. Yuan, and F. Zhang, “A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems,” in Thirty-First AAAI Conference on Artificial Intelligence, 2017, pp. 1309–1315. H. Soh, S. Sanner, M. White, and G. Jamieson, “Deep Sequential Recommendation for Personalized Adaptive User Interfaces,” in Proceedings of the 22nd International Conference on Intelligent User Interfaces, 2017, pp. 589– 593. DOI: 10.1145/3025171.3025207 D.T.V.D. Rao and K.V. Ramana, “Winograd's Inequality: Effectiveness of Efficient Training of Deep Neural Networks,” International Journal of Intelligent Systems and Applications, vol. 6, pp.49--58, 2018. DOI: 10.5815/ijisa.2018.06.06 C. Rana and S.K. Jain, “A study of the dynamic features of recommender systems,” Artificial Intelligence Review, vol. 43, pp.141--153, 2015. DOI: 10.1007/s10462-0129359-6 J. Shokeen and C. Rana, “A review on the dynamics of social recommender systems,” International Journal of Web Engineering and Technology, vol. 13, no.3, pp.255276, 2015. DOI: 10.1504/IJWET.2018.10016164 M. Quadrana, A. Karatzoglou, B. Hidasi, and P. Cremonesi, “Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks,” in Proceedings of the 11th Conference on Recommender Systems, 2017. J. Shokeen and C. Rana, “A study on features of social recommender systems,” Artificial Intelligence Review, 2019. DOI: 10.1007/s10762-019-09684-w J. Shokeen, “On measuring the role of social networks in project recommendation,” International Journal of Computer Sciences and Engineering, vol. 6, no. 4, 2019. Copyright © 2019 MECS DOI: 10.26438/ijcse/v6i4.215219 Authors’ Profiles Jyoti Shokeen belongs to Delhi. She is a Research Scholar in Department of Computer Science and Engineering, University Institute of Engineering and Technology, Maharshi Dayanand University, Rohtak, Haryana. She works under the guidance of Dr. Chhavi Rana. She received her M.Tech degree in CSE in 2014. Her research areas include Social networks, Information Retrieval and Machine learning. She has published a number of research papers in International Journals. She has also published several papers in IEEE and Springer conferences. Dr. Chhavi Rana belongs to Haryana. She is an Assistant Professor in Department of Computer Science and Engineering, University Institute of Engineering and Technology, Maharshi Dayanand University, Rohtak, Haryana. She received her Ph.D degree in Web Mining from NIT, Kurukshetra University, Haryana in 2014. Her research areas include information management, information retrieval, ICT and data mining. She has published more than 50 research papers in reputed International Journals and Conferences. She has an experience of more than 10 years in teaching data mining. She has also been reviewer on IEEE Transaction’s on Systems, Man and Cybernetics: Systems, Artificial Intelligence Review, Springer as well as Inderscience Publishers. She has also published 4 books on her research work. How to cite this paper: Jyoti Shokeen, Chhavi Rana, "An Application-oriented Review of Deep Learning in Recommender Systems", International Journal of Intelligent Systems and Applications(IJISA), Vol.11, No.5, pp.46-54, 2019. DOI: 10.5815/ijisa.2019.05.06 I.J. Intelligent Systems and Applications, 2019, 5, 46-54