research-article

Dynamic Bayesian Contrastive Predictive Coding Model for Personalized Product Search

Authors:

Shangsong LiangAuthors Info & Claims

ACM Transactions on the Web, Volume 17, Issue 4

Article No.: 33, Pages 1 - 31

https://doi.org/10.1145/3609225

Published: 10 October 2023 Publication History

Abstract

In this article, we study the problem of dynamic personalized product search. Due to the data-sparsity problem in the real world, existing methods suffer from the challenge of data inefficiency. We address the challenge by proposing a Dynamic Bayesian Contrastive Predictive Coding model (DBCPC), which aims to capture the rich structured information behind search records to improve data efficiency. Our proposed DBCPC utilizes contrastive predictive learning to jointly learn dynamic embeddings with structure information of entities (i.e., users, products, and words). Specifically, our DBCPC employs structured prediction to tackle the intractability caused by non-linear output space and utilizes the time embedding technique to avoid designing different encoders each time in the Dynamic Bayesian models. In this way, our model jointly learns the underlying embeddings of entities (i.e., users, products, and words) via prediction tasks, which enables the embeddings to focus more on their general attributes and capture the general information during the preference evolution with time. For inferring the dynamic embeddings, we propose an inference algorithm combining the variational objective and the contrastive objectives. Experiments were conducted on an Amazon dataset and the experimental results show that our proposed DBCPC can learn the higher-quality embeddings and outperforms the state-of-the-art non-dynamic and dynamic models for product search.

References

[1]

Amina Adadi. 2021. A survey on data-efficient algorithms in big data era. J. Big Data 8, 1 (2021), 1–54.

[2]

Qingyao Ai, Daniel N. Hill, S. V. N. Vishwanathan, and W. Bruce Croft. 2019. A zero attention model for personalized product search. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 379–388.

Digital Library

[3]

Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W. Bruce Croft. 2017. Learning a hierarchical embedding model for personalized product search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 645–654.

Digital Library

[4]

Qingyao Ai, Yongfeng Zhang, Keping Bi, and W. Bruce Croft. 2019. Explainable product search with a dynamic relation embedding model. ACM Transactions on Information Systems (TOIS) 38, 1 (2019), 1–29.

Digital Library

[5]

Gökhan BakIr, Thomas Hofmann, Alexander J. Smola, Bernhard Schölkopf, and Ben Taskar. 2007. Predicting Structured Data.

Digital Library

[6]

Robert Bamler and Stephan Mandt. 2017. Dynamic word embeddings. In International Conference on Machine Learning. PMLR, 380–389.

[7]

Claudia Beleites, Ute Neugebauer, Thomas Bocklitz, Christoph Krafft, and Jürgen Popp. 2013. Sample size planning for classification models. Analytica chimica acta 760 (2013), 25–33.

[8]

Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R. Devon Hjelm. 2018. Mine: mutual information neural estimation. arXiv preprint arXiv:1801.04062 (2018).

[9]

Keping Bi, Qingyao Ai, and W. Bruce Croft. 2020. A transformer-based embedding model for personalized product search. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1521–1524.

Digital Library

[10]

David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. 2017. Variational inference: A review for statisticians. Journal of the American Statistical Association 112, 518 (2017), 859–877.

[11]

Aleksandar Bojchevski and Stephan Günnemann. 2017. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. arXiv preprint arXiv:1707.03815 (2017).

[12]

Ricky T. Q. Chen, Xuechen Li, Roger Grosse, and David Duvenaud. 2018. Isolating sources of disentanglement in VAEs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2615–2625.

Digital Library

[13]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning. PMLR, 1597–1607.

[14]

Dian Cheng, Jiawei Chen, Wenjun Peng, Wenqin Ye, Fuyu Lv, Tao Zhuang, Xiaoyi Zeng, and Xiangnan He. 2022. IHGNN: Interactive hypergraph neural network for personalized product search. In Proceedings of the ACM Web Conference 2022. 256–265.

Digital Library

[15]

W. Bruce Croft, Donald Metzler, and Trevor Strohman. 2010. Search Engines: Information Retrieval in Practice. Vol. 520. Addison-Wesley, Reading.

Digital Library

[16]

Huizhong Duan and ChengXiang Zhai. 2015. Mining coordinated intent representation for entity search and recommendation. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 333–342.

Digital Library

[17]

Huizhong Duan, ChengXiang Zhai, Jinxing Cheng, and Abhishek Gattani. 2013. Supporting keyword search in product database: A probabilistic approach. Proceedings of the VLDB Endowment 6, 14 (2013), 1786–1797.

Digital Library

[18]

Lu Fan, Qimai Li, Bo Liu, Xiao-Ming Wu, Xiaotong Zhang, Fuyu Lv, Guli Lin, Sen Li, Taiwei Jin, and Keping Yang. 2022. Modeling user behavior with graph convolution for personalized product search. In Proceedings of the ACM Web Conference 2022. 203–212.

Digital Library

[19]

Songwei Ge, Zhicheng Dou, Zhengbao Jiang, Jian-Yun Nie, and Ji-Rong Wen. 2018. Personalizing search results using hierarchical RNN with query-aware attention. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 347–356.

Digital Library

[20]

Samuel Gershman and Noah Goodman. 2014. Amortized inference in probabilistic reasoning. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 36.

[21]

Miha Grčar, Dunja Mladenič, Blaž Fortuna, and Marko Grobelnik. 2005. Data sparsity issues in the collaborative filtering framework. In International Workshop on Knowledge Discovery on the Web. Springer, 58–76.

[22]

Yangyang Guo, Zhiyong Cheng, Liqiang Nie, Yinglong Wang, Jun Ma, and Mohan Kankanhalli. 2019. Attentive long short-term preference modeling for personalized product search. ACM Transactions on Information Systems (TOIS) 37, 2 (2019), 1–27.

Digital Library

[23]

Yangyang Guo, Zhiyong Cheng, Liqiang Nie, Xin-Shun Xu, and Mohan Kankanhalli. 2018. Multi-modal preference modeling for product search. In Proceedings of the 26th ACM International Conference on Multimedia. 1865–1873.

Digital Library

[24]

Michael U. Gutmann and Aapo Hyvärinen. 2012. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research 13, 2 (2012), 307–361.

[25]

Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning. PMLR, 4116–4126.

[26]

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9729–9738.

[27]

Olivier Henaff. 2020. Data-efficient image recognition with contrastive predictive coding. In International Conference on Machine Learning. PMLR, 4182–4192.

Digital Library

[28]

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016. beta-vae: Learning basic visual concepts with a constrained variational framework. (2016).

[29]

R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2018. Learning deep representations by mutual information estimation and maximization. In 5th International Conference on Learning Representations (ICLRz’17).

[30]

Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin. 2017. Collaborative metric learning. In Proceedings of the 26th International Conference on World Wide Web. 193–201.

Digital Library

[31]

Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422–446.

Digital Library

[32]

Wenjun Jiang, Jing Chen, Xiaofei Ding, Jie Wu, Jiawei He, and Guojun Wang. 2021. Review summary generation in online systems: Frameworks for supervised and unsupervised scenarios. ACM Transactions on the Web 15, 3, Article 13 (may 2021), 33 pages.

Digital Library

[33]

Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013).

[34]

Thomas Kipf, Elise van der Pol, and Max Welling. 2019. Contrastive learning of structured world models. In International Conference on Learning Representations.

[35]

Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. 2015. Human-level concept learning through probabilistic program induction. Science 350, 6266 (2015), 1332–1338.

[36]

Phuc H. Le-Khac, Graham Healy, and Alan F. Smeaton. 2020. Contrastive representation learning: A framework and review. IEEE Access 8 (2020), 193907–193934.

[37]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.

[38]

Jure Leskovec, Lada A. Adamic, and Bernardo A. Huberman. 2007. The Dynamics of viral marketing. ACM Transactions on the Web 1, 1 (May 2007), 5–es.

Digital Library

[39]

Shangsong Liang, Xiangliang Zhang, Zhaochun Ren, and Evangelos Kanoulas. 2018. Dynamic embeddings for user profiling in Twitter. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1764–1773.

Digital Library

[40]

Shang Liu, Wanli Gu, Gao Cong, and Fuzheng Zhang. 2020. Structural relationship representation learning with graph embedding for personalized product search. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 915–924.

Digital Library

[41]

Siwei Liu, Iadh Ounis, Craig Macdonald, and Zaiqiao Meng. 2020. A heterogeneous graph neural model for cold-start recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2029–2032.

Digital Library

[42]

Yining Liu, Yong Liu, Yanming Shen, and Keqiu Li. 2017. Recommendation in a changing world: Exploiting temporal dynamics in ratings and reviews. ACM Transactions on the Web 12, 1, Article 3 (Aug. 2017), 20 pages.

Digital Library

[43]

Yuanfu Lu, Yuan Fang, and Chuan Shi. 2020. Meta-learning on heterogeneous information networks for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1563–1573.

Digital Library

[44]

Yunshan Ma, Yingzhi He, An Zhang, Xiang Wang, and Tat-Seng Chua. 2022. CrossCBR: Cross-view contrastive learning for bundle recommendation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22). Association for Computing Machinery, 1233–1241.

Digital Library

[45]

Zhuang Ma and Michael Collins. 2018. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP’18).

[46]

Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 43–52.

Digital Library

[47]

Zaiqiao Meng, Shangsong Liang, Hongyan Bao, and Xiangliang Zhang. 2019. Co-embedding attributed networks. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 393–401.

Digital Library

[48]

Zaiqiao Meng, Shangsong Liang, Xiangliang Zhang, Richard McCreadie, and Iadh Ounis. 2020. Jointly learning representations of nodes and attributes for attributed networks. ACM Transactions on Information Systems (TOIS) 38, 2 (2020), 1–32.

Digital Library

[49]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[50]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. arXiv preprint arXiv:1310.4546 (2013).

[51]

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).

[52]

Yaoxin Pan, Shangsong Liang, Jiaxin Ren, Zaiqiao Meng, and Qiang Zhang. 2021. Personalized, sequential, attentive, metric-aware product search. ACM Transactions on Information Systems (TOIS) 40, 2 (2021), 1–29.

Digital Library

[53]

Jay M. Ponte and W. Bruce Croft. 1998. A language modeling approach to information retrieval. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 275–281.

Digital Library

[54]

Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. 2019. On variational bounds of mutual information. In International Conference on Machine Learning. PMLR, 5171–5180.

[55]

Xin Qian, Ryan A. Rossi, Fan Du, Sungchul Kim, Eunyee Koh, Sana Malik, Tak Yeon Lee, and Nesreen K. Ahmed. 2022. Personalized visualization recommendation. ACM Transactions on the Web 16, 3, Article 11 (Sept. 2022), 47 pages.

Digital Library

[56]

Tom Rainforth, Rob Cornish, Hongseok Yang, Andrew Warrington, and Frank Wood. 2018. On nesting Monte Carlo estimators. In International Conference on Machine Learning. PMLR, 4267–4276.

[57]

Stefan Richthofer and Laurenz Wiskott. 2015. Predictable feature analysis. In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA’15). IEEE, 190–196.

[58]

B. T. C. G. D. Roller, C. Taskar, and D. Guestrin. 2004. Max-margin Markov networks. Advances in Neural Information Processing Systems 16 (2004), 25.

[59]

Monika Singh. 2020. Scalability and sparsity issues in recommender datasets: A survey. Knowledge and Information Systems 62 (2020), 1–43.

Digital Library

[60]

Jiaming Song and Stefano Ermon. 2020. Multi-label contrastive predictive coding. Advances in Neural Information Processing Systems 33 (2020), 8161–8173.

[61]

Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive multiview coding. In Proceedings of the 16th European Conference on Computer Vision (ECCV’20), Part XI 16. Springer, 776–794.

Digital Library

[62]

Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann, Yasemin Altun, and Yoram Singer. 2005. Large margin methods for structured and interdependent output variables. Journal of Machine Learning Research 6, 9 (2005), 1453–1484.

[63]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008), 2579–2605.

[64]

Christophe Van Gysel, Maarten de Rijke, and Evangelos Kanoulas. 2016. Learning latent vector spaces for product search. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. 165–174.

Digital Library

[65]

Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2018. Deep graph Infomax. In International Conference on Learning Representations.

[66]

Lei Wang and Ee-Peng Lim. 2023. Zero-shot next-item recommendation using large pretrained language models. arXiv preprint arXiv:2304.03153 (2023).

[67]

Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning. PMLR, 9929–9939.

[68]

Gary Bishop and Greg Welch. 2001. An introduction to the kalman filter. Proceeding of SIGGRAPH, Course 8.27599-23175.

[69]

Laurenz Wiskott and Terrence J. Sejnowski. 2002. Slow feature analysis: Unsupervised learning of invariances. Neural Computation 14, 4 (2002), 715–770.

Digital Library

[70]

Bin Wu, Zaiqiao Meng, Qiang Zhang, and Shangsong Liang. 2022. Meta-learning helps personalized product search. In Proceedings of the ACM Web Conference 2022. 2277–2287.

Digital Library

[71]

Teng Xiao, Jiaxin Ren, Zaiqiao Meng, Huan Sun, and Shangsong Liang. 2019. Dynamic Bayesian metric learning for personalized product search. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1693–1702.

Digital Library

[72]

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 974–983.

Digital Library

[73]

Lei Zheng, Chaozhuo Li, Chun-Ta Lu, Jiawei Zhang, and Philip S. Yu. 2019. Deep distribution network: Addressing the data sparsity issue for top-N recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1081–1084.

Digital Library

[74]

Deyu Zhou, Meng Zhang, Linhai Zhang, and Yulan He. 2021. A neural group-wise sentiment analysis model with data sparsity awareness. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 14594–14601.

Cited By

Nicola VDelgado KLauretto M(2024)Imbalance-Robust Multi-Label Self-Adjusting kNNACM Transactions on Knowledge Discovery from Data10.1145/366357518:8(1-30)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3663575
Mao QLiu QLi ZWu LLv BZhang ZHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Cross-reconstructed Augmentation for Dual-target Cross-domain RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657902(2352-2356)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657902

Index Terms

Dynamic Bayesian Contrastive Predictive Coding Model for Personalized Product Search
1. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Personalization

Recommendations

Dynamic Bayesian Metric Learning for Personalized Product Search
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

In this paper, we study the problem of personalized product search under streaming scenarios. We address the problem by proposing a Dynamic Bayesian Metric Learning model, abbreviated as DBML, which can collaboratively track the evolutions of latent ...
Integrating collaborative filtering and matching-based search for product recommendations

Currently, recommender systems (RS) have been widely applied in many commercial e-commerce sites to help users deal with the information overload problem. Recommender systems provide personalized recommendations to users and, thus, help in making good ...
Contrastive Learning for User Sequence Representation in Personalized Product Search
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Providing personalization in product search has attracted increasing attention in both industry and research communities. Most existing personalized product search methods model users' individual search interests based on their historical search logs to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on the Web

ACM Transactions on the Web Volume 17, Issue 4

November 2023

331 pages

ISSN:1559-1131

EISSN:1559-114X

DOI:10.1145/3608910

Editor:
Ryen White
Microsoft Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2023

Online AM: 13 July 2023

Accepted: 04 July 2023

Revised: 07 June 2023

Received: 29 October 2022

Published in TWEB Volume 17, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
226
Total Downloads

Downloads (Last 12 months)200
Downloads (Last 6 weeks)8

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nicola VDelgado KLauretto M(2024)Imbalance-Robust Multi-Label Self-Adjusting kNNACM Transactions on Knowledge Discovery from Data10.1145/366357518:8(1-30)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3663575
Mao QLiu QLi ZWu LLv BZhang ZHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Cross-reconstructed Augmentation for Dual-target Cross-domain RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657902(2352-2356)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657902

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents