research-article

Operation-aware Neural Networks for user response prediction

Authors:

Jian ZhaoAuthors Info & Claims

Volume 121, Issue C

Pages 161 - 168

https://doi.org/10.1016/j.neunet.2019.09.020

Published: 01 January 2020 Publication History

Abstract

User response prediction makes a crucial contribution to the rapid development of online advertising system and recommendation system. The importance of learning feature interactions has been emphasized by many works. Many deep models are proposed to automatically learn high-order feature interactions. Since most features in advertising systems and recommendation systems are high-dimensional sparse features, deep models usually learn a low-dimensional distributed representation for each feature in the bottom layer. Besides traditional fully-connected architectures, some new operations, such as convolutional operations and product operations, are proposed to learn feature interactions better. In these models, the representation is shared among different operations. However, the best representation for each operation may be different. In this paper, we propose a new neural model named Operation-aware Neural Networks (ONN) which learns different representations for different operations. Our experimental results on two large-scale real-world ad click/conversion datasets demonstrate that ONN consistently outperforms the state-of-the-art models in both offline-training environment and online-training environment.

References

[1]

Cheng H.-T., Koc L., Harmsen J., Shaked T., Chandra T., Aradhye H., et al., Wide & deep learning for recommender systems, in: Proceedings of the 1st workshop on deep learning for recommender systems, ACM, 2016, pp. 7–10.

[2]

Criteo, (2014). Kaggle display advertising challenge dataset, http://labs.criteo.com/2014/02/kaggle-display-advertising-challenge-dataset/.

[3]

Guo, H., Tang, R., Ye, Y., Li, Z., & He, X. (2017). Deepfm: A Factorization-machine based neural network for ctr prediction, arXiv preprint arXiv:1703.04247.

[4]

Iacobacci, I., Pilehvar, M. T., & Navigli, R. (2015). Sensembed: Learning sense embeddings for word and relational similarity. In: ACL Vol. 1, (pp. 95–105).

[5]

Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, In: International conference on machine learning (pp. 448–456).

[6]

Juan Y., Zhuang Y., Chin W.-S., Lin C.-J., Field-aware factorization machines for CTR prediction, in: Proceedings of the 10th ACM conference on recommender systems, ACM, 2016, pp. 43–50.

[7]

Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980.

[8]

Li, J., & Jurafsky, D. (2015). Do multi-sense embeddings improve natural language understanding?, arXiv preprint arXiv:1506.01070.

[9]

Liu Q., Yu F., Wu S., Wang L., A convolutional click prediction model, in: Proceedings of the 24th ACM international on conference on information and knowledge management, ACM, 2015, pp. 1743–1746.

[10]

McMahan H.B., Holt G., Sculley D., Young M., Ebner D., Grady J., et al., Ad click prediction: a view from the trenches, in: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, 2013, pp. 1222–1230.

[11]

Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines, In: Proceedings of the 27th international conference on machine learning (ICML-10), (pp. 807–814).

[12]

Qu Y., Cai H., Ren K., Zhang W., Yu Y., Wen Y., et al., Product-based neural networks for user response prediction, in: Data mining (ICDM), 2016 IEEE 16th international conference on, IEEE, 2016, pp. 1149–1154.

[13]

Rendle S., Factorization machines, in: Data mining (ICDM), 2010 IEEE 10th international conference on, IEEE, 2010, pp. 995–1000.

[14]

Rendle S., Schmidt-Thieme L., Pairwise interaction tensor factorization for personalized tag recommendation, in: Proceedings of the third ACM international conference on web search and data mining, ACM, 2010, pp. 81–90.

[15]

Srivastava N., Hinton G.E., Krizhevsky A., Sutskever I., Salakhutdinov R., Dropout: a simple way to prevent neural networks from overfitting., Journal of Machine Learning Research (JMLR) 15 (1) (2014) 1929–1958.

[16]

Tencent, (2018). Tencent ads algorithm competition, https://algo.qq.com/home/home/index.html.

[17]

Wang, R., Fu, B., Fu, G., & Wang, M. (2017). Deep & cross network for ad click predictions, (pp. 1–7).

[18]

Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., & Chua, T.-S. Attentional factorization machines: Learning the weight of feature interactions via attention networks, arXiv preprint arXiv:1708.04617.

[19]

Zhang W., Du T., Wang J., Deep learning over multi-field categorical data, in: European conference on information retrieval, Springer, 2016, pp. 45–57.

[20]

Zhou G., Zhu X., Song C., Fan Y., Zhu H., Ma X., et al., Deep interest network for click-through rate prediction, in: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, ACM, 2018, pp. 1059–1068.

Cited By

Yan CLi XTao RZhang ZWan Y(2024)MeFiNetIntelligent Data Analysis10.3233/IDA-22711328:1(261-278)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/IDA-227113
Zhang HPan JLiu DJiang JLi XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Deep Pattern Network for Click-Through Rate PredictionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657777(1189-1199)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657777
Kim TBaek SJeon TJung HKim JLee T(2023)Capturing Performance and Privacy by Assembling Avengers of Online AdvertisingProceedings of the Recommender Systems Challenge 202310.1145/3626221.3627286(28-32)Online publication date: 19-Sep-2023
https://dl.acm.org/doi/10.1145/3626221.3627286
Show More Cited By

Index Terms

Operation-aware Neural Networks for user response prediction

Index terms have been assigned to the content through auto-classification.

Recommendations

Deep Interest Network for Click-Through Rate Prediction
KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Click-through rate prediction is an essential task in industrial applications, such as online advertising. Recently deep learning based models have been proposed, which follow a similar Embedding&MLP paradigm. In these methods large scale sparse input ...
Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction
WWW '19: The World Wide Web Conference

Click-Through Rate prediction is an important task in recommender systems, which aims to estimate the probability of a user to click on a given item. Recently, many deep models have been proposed to learn low-order and high-order feature interactions ...
Improving signal prediction performance of neural networks through multiresolution learning approach

In this paper, we extend the original work on multiresolution learning for neural networks, and present new developments on the multiresolution learning paradigm. The contributions of this paper include: 1) proposing a new concept and method of ...

Comments

Information & Contributors

Information

Published In

cover image Neural Networks

Neural Networks Volume 121, Issue C

Jan 2020

536 pages

ISSN:0893-6080

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 01 January 2020

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yan CLi XTao RZhang ZWan Y(2024)MeFiNetIntelligent Data Analysis10.3233/IDA-22711328:1(261-278)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/IDA-227113
Zhang HPan JLiu DJiang JLi XHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Deep Pattern Network for Click-Through Rate PredictionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657777(1189-1199)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657777
Kim TBaek SJeon TJung HKim JLee T(2023)Capturing Performance and Privacy by Assembling Avengers of Online AdvertisingProceedings of the Recommender Systems Challenge 202310.1145/3626221.3627286(28-32)Online publication date: 19-Sep-2023
https://dl.acm.org/doi/10.1145/3626221.3627286
Tan XDeng YQu CXue SShi XZhang JQiu X(2023)Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural InteractionProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3624918.3625323(205-211)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3624918.3625323
Lv YWang SJin BYu YZhang YDong JWang YWang XWang D(2023)Deep Situation-Aware Interaction Network for Click-Through Rate PredictionProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608793(171-182)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608793
Zhang PZheng ZZhang JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)FiBiNet++: Reducing Model Size by Low Rank Feature Interaction Layer for CTR PredictionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615242(4425-4429)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615242
Wang FGu HLi DLu TZhang PGu NFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Towards Deeper, Lighter and Interpretable Cross Network for CTR PredictionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615089(2523-2533)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615089
Jiang LLi CChen HGao XZhong XQiu YYe SNiu DSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)AdSEE: Investigating the Impact of Image Style Editing on Advertisement AttractivenessProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599770(4239-4251)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599770
Dong JYu YZhang YLv YWang SJin BWang YWang XWang D(2023)A Deep Behavior Path Matching Network for Click-Through Rate PredictionCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3584662(538-542)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543873.3584662
Zhu YLin QLu HShi KLiu DChambua JWan SNiu Z(2023)Recommending Learning Objects Through Attentive Heterogeneous Graph Convolution and Operation-Aware Neural NetworkIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.312542435:4(4178-4189)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TKDE.2021.3125424
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents