research-article

A Personalized Dense Retrieval Framework for Unified Information Access

Authors:

Surya Kallumadi,

Rodrigo Nogueira,

Hamed ZamaniAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 121 - 130

https://doi.org/10.1145/3539618.3591626

Published: 18 July 2023 Publication History

Abstract

Developing a universal model that can efficiently and effectively respond to a wide range of information access requests-from retrieval to recommendation to question answering---has been a long-lasting goal in the information retrieval community. This paper argues that the flexibility, efficiency, and effectiveness brought by the recent development in dense retrieval and approximate nearest neighbor search have smoothed the path towards achieving this goal. We develop a generic and extensible dense retrieval framework, called framework, that can handle a wide range of (personalized) information access requests, such as keyword search, query by example, and complementary item recommendation. Our proposed approach extends the capabilities of dense retrieval models for ad-hoc retrieval tasks by incorporating user-specific preferences through the development of a personalized attentive network. This allows for a more tailored and accurate personalized information access experience. Our experiments on real-world e-commerce data suggest the feasibility of developing universal information access models by demonstrating significant improvements even compared to competitive baselines specifically developed for each of these individual information access tasks. This work opens up a number of fundamental research directions for future exploration.

References

[1]

Qingyao Ai, Daniel N. Hill, S. V. N. Vishwanathan, and W. Bruce Croft. 2019. A Zero Attention Model for Personalized Product Search. Proceedings of the 28th ACM International Conference on Information and Knowledge Management (2019).

Digital Library

[2]

Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W. Bruce Croft. 2017. Learning a Hierarchical Embedding Model for Personalized Product Search. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (2017).

Digital Library

[3]

Nicholas J. Belkin and W. Bruce Croft. 1992. Information filtering and information retrieval: two sides of the same coin? Commun. ACM, Vol. 35 (1992), 29--38.

Digital Library

[4]

Daniel Fernando Campos, Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, Li Deng, and Bhaskar Mitra. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. ArXiv, Vol. abs/1611.09268 (2016).

[5]

Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (2017).

Digital Library

[6]

Alexis Conneau and Douwe Kiela. 2018. SentEval: An Evaluation Toolkit for Universal Sentence Representations. ArXiv, Vol. abs/1803.05449 (2018).

[7]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL.

[8]

Luyu Gao and Jamie Callan. 2021. Condenser: a Pre-training Architecture for Dense Retrieval. In EMNLP.

[9]

Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. NAIS: Neural Attentive Item Similarity Model for Recommendation. IEEE Transactions on Knowledge and Data Engineering, Vol. 30 (2018), 2354--2366.

Digital Library

[10]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. Proceedings of the 26th International Conference on World Wide Web (2017).

Digital Library

[11]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. CoRR, Vol. abs/1511.06939 (2016).

[12]

Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy J. Lin, and Allan Hanbury. 2021. Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021).

Digital Library

[13]

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, Vol. 7, 3 (2019), 535--547.

[14]

Santosh Kabbur, Xia Ning, and George Karypis. 2013. FISM: factored item similarity models for top-N recommender systems. Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining (2013).

Digital Library

[15]

Wang-Cheng Kang, Chen Fang, Zhaowen Wang, and Julian McAuley. 2017. Visually-Aware Fashion Recommendation and Design with Generative Image Models. 2017 IEEE International Conference on Data Mining (ICDM) (2017), 207--216.

[16]

Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recommendation. 2018 IEEE International Conference on Data Mining (ICDM) (2018), 197--206.

[17]

Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Yu Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. ArXiv, Vol. abs/2004.04906 (2020).

[18]

Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, and Hwanjo Yu. 2016. Convolutional Matrix Factorization for Document Context-Aware Recommendation. Proceedings of the 10th ACM Conference on Recommender Systems (2016).

Digital Library

[19]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. CoRR, Vol. abs/1412.6980 (2015).

[20]

Yehuda Koren and Robert M. Bell. 2011. Advances in Collaborative Filtering. In Recommender Systems Handbook.

[21]

Quoc V. Le and Tomas Mikolov. 2014. Distributed Representations of Sentences and Documents. In ICML.

Digital Library

[22]

Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy J. Lin. 2021. In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval. In REPL4NLP.

[23]

Prafull Prakash, Julia Killingback, and Hamed Zamani. 2021. Learning Robust Dense Retrieval Models from Incomplete Relevance Labels. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021).

Digital Library

[24]

Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021. RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering. In NAACL.

[25]

Chandan K. Reddy, Lluís Màrquez, Fran Valero, Nikhil Rao, Hugo Zaragoza, Sambaran Bandyopadhyay, Arnab Biswas, Anlu Xing, and Karthik Subbian. 2022. Shopping Queries Dataset: A Large-Scale ESCI Benchmark for Improving Product Search. (2022). arxiv: 2206.06588

[26]

Stephen E. Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr., Vol. 3 (2009), 333--389.

Digital Library

[27]

Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2022. One Embedder, Any Task: Instruction-Finetuned Text Embeddings. ArXiv, Vol. abs/2212.09741 (2022).

[28]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. Proceedings of the 28th ACM International Conference on Information and Knowledge Management (2019).

Digital Library

[29]

Nandan Thakur, Nils Reimers, Andreas Ruckl'e, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. ArXiv, Vol. abs/2104.08663 (2021).

[30]

Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS.

[31]

Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative Deep Learning for Recommender Systems. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015).

Digital Library

[32]

Suhang Wang, Yilin Wang, Jiliang Tang, Kai Shu, Suhas Ranganath, and Huan Liu. 2017. What Your Images Reveal: Exploiting Visual Contents for Point-of-Interest Recommendation. Proceedings of the 26th International Conference on World Wide Web (2017).

Digital Library

[33]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace's Transformers: State-of-the-art Natural Language Processing. ArXiv, Vol. abs/1910.03771 (2019).

[34]

Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. ArXiv, Vol. abs/2007.00808 (2021).

[35]

Liu Yang, Qi Guo, Yang Song, Sha Meng, Milad Shokouhi, Kieran McDonald, and W. Bruce Croft. 2016. Modelling User Interest for Zero-query Ranking. In European Conference on Information Retrieval (ECIR 2016).

[36]

Hamed Zamani. 2020. Learning a Joint Search and Recommendation Model from User-Item Interactions. Proceedings of the 13th International Conference on Web Search and Data Mining (2020).

Digital Library

[37]

Hamed Zamani, Michael Bendersky, Xuanhui Wang, and Mingyang Zhang. 2017. Situational Context for Ranking in Personal Search. Proceedings of the 26th International Conference on World Wide Web (2017).

Digital Library

[38]

Hamed Zamani and W. Bruce Croft. 2018. Joint Modeling and Optimization of Search and Recommendation. ArXiv, Vol. abs/1807.05631 (2018).

[39]

Hansi Zeng, Hamed Zamani, and Vishwa Vinay. 2022. Curriculum Learning for Dense Retrieval Distillation. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022).

Digital Library

[40]

Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, M. Zhang, and Shaoping Ma. 2021. Optimizing Dense Retrieval Model Training with Hard Negatives. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021).

Digital Library

[41]

Kai Zhao, Yukun Zheng, Tao Zhuang, Xiang Li, and Xiaoyi Zeng. 2022. Joint Learning of E-commerce Search and Recommendation with a Unified Graph Neural Network. Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (2022).

Digital Library

Cited By

Zhou XGao YJie XCai XBu JWang HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)EASE-DR: Enhanced Sentence Embeddings for Dense RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657925(2374-2378)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657925
Salemi AKallumadi SZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Optimization Methods for Personalizing Large Language Models through Retrieval AugmentationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657783(752-762)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657783
Zeng HLuo CZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous DecodingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657746(469-480)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657746
Show More Cited By

Index Terms

A Personalized Dense Retrieval Framework for Unified Information Access
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Retrieval models and ranking
      1. Learning to rank

Recommendations

Lexically-Accelerated Dense Retrieval
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Retrieval approaches that score documents based on learned dense vectors (i.e., dense retrieval) rather than lexical signals (i.e., conventional retrieval) are increasingly popular. Their ability to identify related documents that do not necessarily ...
Generative Retrieval as Multi-Vector Dense Retrieval
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

For a given query generative retrieval generates identifiers of relevant documents in an end-to-end manner using a sequence-to-sequence architecture. The relation between generative retrieval and other retrieval methods, especially those based on ...
A Hybrid Approach of Personalized Web Information Retrieval
WI-IAT '10: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01

This paper proposes a hybrid approach of personalized Web Information Retrieval that utilizes (1) ontology for retrieval of user’s context (2) user profile that is temporarily updated according to users’ browsing behavior and (3) collaborative filtering ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
291
Total Downloads

Downloads (Last 12 months)239
Downloads (Last 6 weeks)10

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou XGao YJie XCai XBu JWang HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)EASE-DR: Enhanced Sentence Embeddings for Dense RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657925(2374-2378)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657925
Salemi AKallumadi SZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Optimization Methods for Personalizing Large Language Models through Retrieval AugmentationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657783(752-762)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657783
Zeng HLuo CZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous DecodingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657746(469-480)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657746
Xu ZChen DKuang JYi ZLi YShen YHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Dynamic Demonstration Retrieval and Cognitive Understanding for Emotional Support ConversationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657695(774-784)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657695
Zeng HLuo CJin BSarwar SWei TZamani HChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Scalable and Effective Generative Information RetrievalProceedings of the ACM on Web Conference 202410.1145/3589334.3645477(1441-1452)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645477
He XWei THe JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Robust Basket Recommendation via Noise-tolerated Graph Contrastive LearningProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615039(709-719)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615039

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents