ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling
Authors: 
Wen-Ji Zhou, Yuhang Zheng, Yinfu Feng, Yunan Ye, Rong Xiao, Long Chen, Xiaosong Yang, Jun XiaoAuthors Info & Claims
Wen-Ji Zhou, Yuhang Zheng, Yinfu Feng, Yunan Ye, Rong Xiao, Long Chen, Xiaosong Yang, Jun XiaoAuthors Info & ClaimsPages 265 - 277
Abstract
Long-term user behavior sequences are a goldmine for businesses to explore users’ interests to improve Click-Through Rate (CTR). However, it is very challenging to accurately capture users’ long-term interests from their long-term behavior sequences and give quick responses from the online serving systems. To meet such requirements, existing methods “inadvertently” destroy two basic requirements in long-term sequence modeling: <bold>R1</bold>) make full use of the entire sequence to keep the information as much as possible; <bold>R2</bold>) extract information from the most relevant behaviors to keep high relevance between learned interests and current target items. The performance of online serving systems is significantly affected by incomplete and inaccurate user interest information obtained by existing methods. To this end, we propose an efficient two-stage long-term sequence modeling approach, named as <bold>E</bold>fficie<bold>N</bold>t <bold>C</bold>lustering based tw<bold>O</bold>-stage interest mo<bold>DE</bold>ling (ENCODE), consisting of offline extraction stage and online inference stage. It not only meets the aforementioned two basic requirements but also achieves a desirable balance between online service efficiency and precision. Specifically, in the offline extraction stage, ENCODE clusters the entire behavior sequence and extracts accurate interests. To reduce the overhead of the clustering process, we design a metric learning-based dimension reduction algorithm that preserves the relative pairwise distances of behaviors in the new feature space. While in the online inference stage, ENCODE takes the off-the-shelf user interests to predict the associations with target items. Besides, to further ensure the relevance between user interests and target items, we adopt the same relevance metric throughout the whole pipeline of ENCODE. The extensive experiment and comparison with SOTA on both industrial and public datasets have demonstrated the effectiveness and efficiency of our proposed ENCODE.
References
[1]
Q. Pi, W. Bian, G. Zhou, X. Zhu, and K. Gai, “Practice on long sequential user behavior modeling for click-through rate prediction,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2019, pp. 2671–2679.
[2]
Q. Pi et al., “Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction,” in Proc. ACM Int. Conf. Inf. Knowl. Manage., 2020, pp. 2685–2692.
[3]
J. Qin, W. Zhang, X. Wu, J. Jin, Y. Fang, and Y. Yu, “User behavior retrieval for click-through rate prediction,” in Proc. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2020, pp. 2347–2356.
[4]
L. Wu, S. Li, C.-J. Hsieh, and J. Sharpnack, “SSE-PT: Sequential recommendation via personalized transformer,” in Proc. ACM Conf. Recommender Syst., 2020, pp. 328–337.
[5]
G. de Souza Pereira Moreira, S. Rabhi, J. M. Lee, R. Ak, and E. Oldridge, “Transformers4Rec: Bridging the gap between NLP and sequential/session-based recommendation,” in Proc. ACM Conf. Recommender Syst., 2021, pp. 143–153.
[6]
K. Ren et al., “Lifelong sequential modeling with personalized memorization for user response prediction,” in Proc. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2019, pp. 565–574.
[7]
X. Zhou and Y. Li, “Large-scale modeling of mobile user click behaviors using deep learning,” in Proc. ACM Conf. Recommender Syst., 2021, pp. 473–483.
[8]
C. Hansen et al., “Contextual and sequential user embeddings for large-scale music recommendation,” in Proc. ACM Conf. Recommender Syst., 2020, pp. 53–62.
[9]
W. Tao et al., “Sminet: State-aware multi-aspect interests representation network for cold-start users recommendation,” in Proc. AAAI Conf. Artif. Intell., 2022, pp. 8476–8484.
[10]
X. Xu, F. Dong, Y. Li, S. He, and X. Li, “Contextual-bandit based personalized recommendation with time-varying user interests,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 6518–6525.
[11]
W. Zhang, J. Qin, W. Guo, R. Tang, and X. He, “Deep learning for click-through rate estimation,” in Proc. Int. Joint Conf. Artif. Intell., 2021, pp. 4695–4703.
[12]
K. Ren et al., “Lifelong sequential modeling with personalized memorization for user response prediction,” in Proc. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2019, pp. 565–574.
[13]
Y. Cao et al., “Sampling is all you need on modeling long-term user behaviors for CTR prediction,” in Proc. ACM Int. Conf. Inf. Knowl. Manage., 2022, pp. 2974–2983.
[14]
S. Rendle, “Factorization machines,” in Proc. IEEE Int. Conf. Data Mining, 2010, pp. 995–1000.
[15]
J. Pan et al., “Field-weighted factorization machines for click-through rate prediction in display advertising,” in Proc. World Wide Web Conf., 2018, pp. 1349–1357.
[16]
Heng-Tze et al., “Wide & deep learning for recommender systems,” in Proc. ACM Conf. Recommender Syst., 2016, pp. 7–10.
[17]
J. Xiao, H. Ye, X. He, H. Zhang, F. Wu, and T.-S. Chua, “Attentional factorization machines: Learning the weight of feature interactions via attention networks,” in Proc. Int. Joint Conf. Artif. Intell., 2017, pp. 3119–3125.
[18]
H. Guo, R. Tang, Y. Ye, Z. Li, and X. He, “DeepFM: A factorization-machine based neural network for CTR prediction,” in Proc. Int. Joint Conf. Artif. Intell., 2017, pp. 1725–1731.
[19]
Y. Qu et al., “Product-based neural networks for user response prediction,” in Proc. IEEE Int. Conf. Data Mining, 2016, pp. 1149–1154.
[20]
J. Lian, X. Zhou, F. Zhang, Z. Chen, X. Xie, and G. Sun, “xDeepFM: Combining explicit and implicit feature interactions for recommender systems,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2018, pp. 1754–1763.
[21]
R. Wang, B. Fu, G. Fu, and M. Wang, “Deep & cross network for ad click predictions,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2017, pp. 12:1–12:7.
[22]
P. Covington, J. Adams, and E. Sargin, “Deep neural networks for YouTube recommendations,” in Proc. ACM Conf. Recommender Syst., 2016, pp. 191–198.
[23]
B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, “Session-based recommendations with recurrent neural networks,” in Proc. Int. Conf. Learn. Representations, 2016, pp. 1–10.
[24]
G. Zhou et al., “Deep interest evolution network for click-through rate prediction,” in Proc. AAAI Conf. Artif. Intell., 2019, pp. 5941–5948.
[25]
Y. Feng et al., “Deep session interest network for click-through rate prediction,” in Proc. Int. Joint Conf. Artif. Intell., 2019, pp. 2301–2307.
[26]
F. Sun et al., “BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer,” in Proc. ACM Int. Conf. Inf. Knowl. Manage., 2019, pp. 1441–1450.
[27]
G. Zhou et al., “Deep interest network for click-through rate prediction,” in Proc. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2018, pp. 1059–1068.
[28]
C. Li et al., “Multi-interest network with dynamic routing for recommendation at tmall,” in Proc. ACM Int. Conf. Inf. Knowl. Manage., 2019, pp. 2615–2623.
[29]
H. Jiang, W. Wang, Y. Wei, Z. Gao, Y. Wang, and L. Nie, “What aspect do you like: Multi-scale time-aware user interest modeling for micro-video recommendation,” in Proc. ACM Int. Conf. Multimedia, 2020, pp. 3487–3495.
[30]
Y. Cen, J. Zhang, X. Zou, C. Zhou, H. Yang, and J. Tang, “Controllable multi-interest framework for recommendation,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2020, pp. 2942–2951.
[31]
Q. Chen, Y. Xu, C. Pei, S. Lv, T. Zhuang, and J. Ge, “Efficient long sequential user data modeling for click-through rate prediction,” 2022,.
[32]
J. Chang et al., “TWIN: Two-stage interest network for lifelong user behavior modeling in CTR prediction at Kuaishou,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Mining, 2023, pp. 3785–3794.
[33]
A. Andoni, P. Indyk, T. Laarhoven, I. P. Razenshteyn, and L. Schmidt, “Practical and optimal LSH for angular distance,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2015, pp. 1225–1233.
[34]
M. S. Charikar, “Similarity estimation techniques from rounding algorithms,” in Proc. Annu. ACM Symp. Theory Comput., 2002, pp. 380–388.
[35]
E. Xing, M. Jordan, S. J. Russell, and A. Ng, “Distance metric learning with application to clustering with side-information,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2002, pp. 521–528.
[36]
F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 815–823.
[37]
J. J. McAuley, C. Targett, Q. Shi, and A. van den Hengel, “Image-based recommendations on styles and substitutes,” in Proc. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 2015, pp. 43–52.
[38]
Q. Lin, W.-J. Zhou, Y. Wang, Q. Da, Q.-G. Chen, and B. Wang, “Sparse attentive memory network for click-through rate prediction with long sequences,” in Proc. ACM Int. Conf. Inf. Knowl. Manage., 2022, pp. 3312–3321.
[39]
F. M. Harper and J. A. Konstan, “The movielens datasets: History and context,” ACM Trans. Interactive Intell. Syst., vol. 5, no. 4, pp. 1–19, 2015.
[40]
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014,.
[41]
K. Sohn, “Improved deep metric learning with multi-class n-pair loss objective,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2016, pp. 1857–1865.
Index Terms
- ENCODE: Breaking the Trade-Off Between Performance and Efficiency in Long-Term User Behavior Modeling
Index terms have been assigned to the content through auto-classification.
Recommendations
Long-term and session-specific user preferences in a mobile recommender system
IUI '08: Proceedings of the 13th international conference on Intelligent user interfacesUser preferences acquisition plays a very important role for recommender systems. In a previous paper, we proposed a critique-based mobile recommendation methodology exploiting both long-term and session-specific user preferences. In this paper, we ...
Sequential Recommendation Based on Long-Term and Short-Term User Behavior with Self-attention
Knowledge Science, Engineering and ManagementAbstractProduct recommenders based on users’ interests are becoming increasingly essential in e-commerce. With the continuous development of the recommendation system, the available information is further enriched. In the case, user’s click or purchase ...
Combining long-term and short-term user interest for personalized hashtag recommendation
Hashtags, terms prefixed by a hash-symbol #, are widely used and inserted anywhere within short messages (tweets) on micro-blogging systems as they present rich sentiment information on topics that people are interested in. In this paper, we focus on ...
Comments
Information & Contributors
Information
Published In
1041-4347 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 01 January 2025
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025