research-article

Deep Feature Fusion over Multi-field Categorical Data for Rating Prediction

Authors:

Dong WangAuthors Info & Claims

AICCC '18: Proceedings of the 2018 Artificial Intelligence and Cloud Computing Conference

Pages 16 - 22

https://doi.org/10.1145/3299819.3299827

Published: 21 December 2018 Publication History

Abstract

Many predictive tasks in recommender systems model from categorical variables. Different from continuous features extracted from images and videos, categorical data is discrete and of multi-field while their dependencies are little known, which brings the problem of heavy computation on a large-scale sparse feature space. Deep learning methods have strong feature extraction capabilities and now have been more and more widely applied to recommender systems, but they do not perform well on discrete data. To tackle these two problems, in this paper we propose Deep Feature Fusion Model(DFFM) over sparse multi-field categorical data. DFFM utilizes categorical features as inputs and applies the Stacked Denoising AutoEncoder to obtain a dense representation. We construct a full feature connection layer and adopt a multi-layer convolution neural network to further extract deeper features and convert rating prediction to a classification problem. The extensive experiments on real world datasets show that our proposed method outperforms other state-of-the-art approaches.

References

[1]

Bauman, K., Liu, B., and Tuzhilin, A. Recommending items with conditions enhancing user experiences based on sentiment analysis of reviews. In CBRecSys@ RecSys (2016), pp. 19--22.

[2]

Blei, D. M., Ng, A. Y., and Jordan, M. I. Latent dirichlet allocation. Journal of Machine Learning Research 3 (2003), 993--1022.

Digital Library

[3]

Cheng, C., Xia, F., Zhang, T., King, I., and Lyu, M. Gradient boosting factorization machines. In Proceedings of the 8th ACM Conference on Recommender systems (RecSys) (2014), pp. 265--272.

Digital Library

[4]

Cheng, H.-T., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., and Ispir, M. Wide & deep learning for recommender systems. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (2016), ACM, pp. 7--10.

Digital Library

[5]

Covington, P., Adams, J., and Sargin, E. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys) (2016), ACM, pp. 191--198.

Digital Library

[6]

Debasis, G., Dwaipayan, R., Mandar, M., and Gareth, J. Word embedding based generalized language model for information retrieval. In Proceedings of the 38th International ACM SIGIR Conference (2015), ACM, pp. 795--798.

Digital Library

[7]

Dong, X., Yu, L., Wu, Z., Sun, Y., Yuan, L., and Zhang, F. A hybrid collaborative filtering model with deep structure for recommendersystems. In Proceedings of the 31th AAAI Conference (2017), pp. 1309--1315.

Digital Library

[8]

Freudenthaler, C., Schmidt-Thieme, L., and Rendle, S. Bayesian factorization machines. In Workshop on Sparse Representation and Low-rank Approximation, Neural Information Processing Systems (NIPS-WS) (2011).

[9]

Guo, G., Zhang, J., and Neil, Y.-S. Trustsvd: Collaborative filtering with both the explicit and implicit influence of user trust and of item ratings. In Proceedings of the 29th AAAI Conference (2015).

Digital Library

[10]

Guo, W., Wu, S., Wang, L., and Tan, T. Personalized ranking with pairwise factorization machines. Neurocomputing (2016).

Digital Library

[11]

Hao, M., Haixuan, Y., Lyu, M. R., and Irwin, K. Sorec: social recommendation using probabilistic matrix factorization. In 17th ACM conference on Information and knowledge management (CIKM) (2008), pp. 931--940.

Digital Library

[12]

Jiang, M., Cui, P., Liu, R., Yang, Q., Wang, F., Zhu, W., and Yang, S. Social contextual recommendation. In Proceedings of the 21st ACM international conference on Information and knowledge management (CIKM) (2012), pp. 45--54.

Digital Library

[13]

Karmaker, S., Shubhra, K., Sondhi, P., and Zhai, C. Generative feature language models for mining implicit features from customer reviews. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management (CIKM) (2016), ACM, pp. 929--938.

Digital Library

[14]

Kim, D.H., Park, C., Oh, J., Lee, S., and Yu, H. Convolutional matrix factorization for document context-aware recommendation. In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys) (2016), pp. 233--240.

Digital Library

[15]

Kim, Y. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in NaturalLanguage Processing (2014), pp. 1746--1751.

[16]

Koren, Y., Bell, R. M., and Volinsky, C. Matrix factorization techniques for recommender systems. IEEE Computer 42, 8 (2009), 30--37.

Digital Library

[17]

Krizhevsky, A., Ilya, S., and Geoffrey, H. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS) (2012), pp. 1097--1105.

Digital Library

[18]

Li, M., Liu, Z., Smola, A. J., and Wang, Y.-X. Difacto: Distributed factorization machines. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (WSDM) (2016), pp. 377--386.

Digital Library

[19]

Li, S., Kawale, J., and Fu, Y. Deep collaborative filtering via marginalized denoising auto-encoder. In Proceedings of the 24th ACM International Conference on Informationand Knowledge Management (CIKM) (2015), pp. 811--820.

Digital Library

[20]

Ma, H., King, I., and Lyu, M. R. Learning to recommend with social trust ensemble. In 32nd international ACM conference on Research and development in information retrieval (SIGIR) (2009), pp. 203--210.

Digital Library

[21]

Ma, H., Zhou, D., Liu, C., Lyu, M. R., and King, I. Recommender systems with social regularization. In the fourth ACM international conference on Web search and data mining (2011), pp. 287--296.

Digital Library

[22]

Mnih, A., and Salakhutdinov, R. Probabilistic matrix factorization. In Advances in neural information processing systems (NIPS) (2007), pp. 1257--1264.

Digital Library

[23]

Mohsen, J., and Martin, E. A matrix factorization technique with trust propagation for recommendation in social networks. In Proceedings of the fourth ACM conference on Recommender systems (RecSys) (2010), pp. 135--142.

Digital Library

[24]

Nguyen, T. V., Karatzoglou, A., and Baltrunas, L. Gaussian process factorization machines for context-aware recommendations. In the 37th international ACM conference on Research & development in information retrieval (SIGIR) (2014), pp. 63--72.

Digital Library

[25]

Pascal, V., Hugo, L., Isabelle, L., Yoshua, B., and Antoine, M. P. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11, Dec (2010), 3371--3408.

Digital Library

[26]

Piczak, K. J. Environmental sound classification with convolutional neural networks. In 25th IEEE International Workshop on Machine Learning for Signal Processing (2015), pp. 1--6.

[27]

Rendle, S. Scaling factorization machines to relational data. VLDB Journal 6, 5 (2013), 337--348.

Digital Library

[28]

Resnick, P., and Varian, H. Recommender systems. Communications of the ACM 40, 3 (1997), 56--58.

Digital Library

[29]

Sarwar, B., Karypis, G., Konstan, J., and Riedl, J. Application of dimensionality reduction in recommender system-a case study. Tech. rep., Minnesota Univ Minneapolis Dept of Computer Science, 2000.

[30]

Sedhain, S., Menon, K. A., Sanner, S., and Xie, L. Autorec: Autoencoders meet collaborative filtering. In Proceedings of the 24th International Conference on World Wide Web (2015), ACM, pp. 111--112.

Digital Library

[31]

Steffen, R. Factorization machines. In IEEE 10th International Conference on Data Mining (ICDM) (2010), pp. 995--1000.

Digital Library

[32]

Steffen, R. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST) 3, 3 (2012), 57.

Digital Library

[33]

Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P. A. Extracting and composing robust features with denoising autoencoders. In Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML) (2008), pp. 1096--1103.

Digital Library

[34]

Wang, H., Shi, X., and Yeung, D.-Y. Relational stacked denoising autoencoder for tag recommendation. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015), pp. 3052--3058.

Digital Library

[35]

Wu, S., Ren, W., Yu, C., Chen, G., Zhang, D., and Zhu, J. Personal recommendation using deep recurrent neural networks in netease. In IEEE 32nd International Conference on Data Engineering (ICDE) (2016), IEEE, pp. 1218--1229.

[36]

Xiangnan, H., and Tat-Seng, C. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference (2017), ACM, pp. 355--364.

Digital Library

[37]

Xiwang, Y., Harald, S., and Yong, L. Circle-based recommendation in online social networks. In the 18th ACM international conference on Knowledge discovery and data mining (SIGKDD) (2012), pp. 1267--1275.

Digital Library

[38]

Xueming, Q., He, F., Guoshuai, Z., and Tao, M. Personalized recommendation combining user interest and social circle. IEEE Transactions on Knowledge and Data Engineering (TKDE) 26, 7 (2014), 1763--1777.

[39]

Zhang, S., Wang, W., Ford, J., and Makedon, F. Learning from incomplete ratings using non-negative matrix factorization. In Proceedings of the Sixth SIAM International Conference on Data Mining (2006), pp. 549--553.

[40]

Zhang, W., Du, T., and Wang, J. Deep learning over multi-field categorical data. In European conference on information retrieval (2016), Springer, pp. 45--57.

[41]

Zheng, L., Noroozi, V., and Yu, P. S. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM) (2017), ACM, pp. 425--434.

Digital Library

Cited By

Balashankar ABeutel ASubramanian LLewin-Eytan LCarmel DYom-Tov EAgichtein EGabrilovich E(2021)Enhancing Neural Recommender Models through Domain-Specific ConcordanceProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441784(1002-1010)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3437963.3441784

Index Terms

Deep Feature Fusion over Multi-field Categorical Data for Rating Prediction
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Neural Factorization Machines for Sparse Predictive Analytics
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Many predictive tasks of web applications need to model categorical variables, such as user IDs and demographics like genders and occupations. To apply standard machine learning techniques, these categorical predictors are always converted to a set of ...
Deep learning techniques for rating prediction: a survey of the state-of-the-art
Abstract
With the growth of online information, varying personalization drifts and volatile behaviors of internet users, recommender systems are effective tools for information filtering to overcome the information overload problem. Recommender systems ...
Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion
Abstract
Electroencephalography (EEG) motor imagery (MI) signals have recently gained a lot of attention as these signals encode a person’s intent of performing an action. Researchers have used MI signals to help disabled persons, control ...
Highlights
- Multi CNN models with different layers and filters for robust EEG feature extraction.

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AICCC '18: Proceedings of the 2018 Artificial Intelligence and Cloud Computing Conference

December 2018

206 pages

ISBN:9781450366236

DOI:10.1145/3299819

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 December 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AICCC '18

AICCC '18: 2018 Artificial Intelligence and Cloud Computing Conference

December 21 - 23, 2018

Tokyo, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
127
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Balashankar ABeutel ASubramanian LLewin-Eytan LCarmel DYom-Tov EAgichtein EGabrilovich E(2021)Enhancing Neural Recommender Models through Domain-Specific ConcordanceProceedings of the 14th ACM International Conference on Web Search and Data Mining10.1145/3437963.3441784(1002-1010)Online publication date: 8-Mar-2021
https://dl.acm.org/doi/10.1145/3437963.3441784

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents