Article

Improving web search ranking by incorporating user behavior information

Authors:

Eugene Agichtein,

Eric Brill, and

Susan DumaisAuthors Info & Claims

SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval

August 2006

Pages 19 - 26

https://doi.org/10.1145/1148170.1148177

Published: 06 August 2006 Publication History

Abstract

We show that incorporating user behavior data can significantly improve ordering of top results in real web search setting. We examine alternatives for incorporating feedback into the ranking process and explore the contributions of user feedback compared to other common web search features. We report results of a large scale evaluation over 3,000 queries and 12 million user interactions with a popular web search engine. We show that incorporating implicit feedback can augment other features, improving the accuracy of a competitive web search ranking algorithms by as much as 31% relative to the original performance.

References

[1]

E. Agichtein, E. Brill, S. Dumais, and R.Ragno, Learning User Interaction Models for Predicting Web Search Result Preferences. In Proceedings of the ACM Conference on Research and Development on Information Retrieval (SIGIR), 2006

Digital Library

[2]

J. Allan, HARD Track Overview in TREC 2003, High Accuracy Retrieval from Documents, 2003

[3]

R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley, 1999.

Digital Library

[4]

S. Brin and L. Page, The Anatomy of a Large-scale Hypertextual Web Search Engine, in Proceedings of WWW, 1997

Digital Library

[5]

C.J.C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, G. Hullender, Learning to Rank using Gradient Descent, in Proceedings of the International Conference on Machine Learning, 2005

Digital Library

[6]

D.M. Chickering, The WinMine Toolkit, Microsoft Technical Report MSR-TR-2002-103, 2002

[7]

M. Claypool, D. Brown, P. Lee and M. Waseda. Inferring user interest. IEEE Internet Computing. 2001

Digital Library

[8]

S. Fox, K. Karnawat, M. Mydland, S. T. Dumais and T. White. Evaluating implicit measures to improve the search experience. In ACM Transactions on Information Systems, 2005

Digital Library

[9]

J. Goecks and J. Shavlick. Learning users' interests by unobtrusively observing their normal behavior. In Proceedings of the IJCAI Workshop on Machine Learning for Information Filtering. 1999.

[10]

K Jarvelin and J. Kekalainen. IR evaluation methods for retrieving highly relevant documents. In Proceedings of the ACM Conference on Research and Development on Information Retrieval (SIGIR), 2000

Digital Library

[11]

T. Joachims, Optimizing Search Engines Using Clickthrough Data. In Proceedings of the ACM Conference on Knowledge Discovery and Datamining (SIGKDD), 2002

Digital Library

[12]

T. Joachims, L. Granka, B. Pang, H. Hembrooke, and G. Gay, Accurately Interpreting Clickthrough Data as Implicit Feedback, Proceedings of the ACM Conference on Research and Development on Information Retrieval (SIGIR), 2005

Digital Library

[13]

T. Joachims, Making Large-Scale SVM Learning Practical. Advances in Kernel Methods, in Support Vector Learning, MIT Press, 1999

Digital Library

[14]

D. Kelly and J. Teevan, Implicit feedback for inferring user preference: A bibliography. In SIGIR Forum, 2003

Digital Library

[15]

J. Konstan, B. Miller, D. Maltz, J. Herlocker, L. Gordon, and J. Riedl. GroupLens: Applying collaborative filtering to usenet news. In Communications of ACM, 1997.

Digital Library

[16]

M. Morita, and Y. Shinoda, Information filtering based on user behavior analysis and best match text retrieval. Proceedings of the ACM Conference on Research and Development on Information Retrieval (SIGIR), 1994

Digital Library

[17]

D. Oard and J. Kim. Implicit feedback for recommender systems. In Proceedings of the AAAI Workshop on Recommender Systems. 1998

[18]

D. Oard and J. Kim. Modeling information content using observable behavior. In Proceedings of the 64th Annual Meeting of the American Society for Information Science and Technology. 2001

[19]

N. Pharo, N. and K. Järvelin. The SST method: a tool for analyzing web information search processes. In Information Processing & Management, 2004

Digital Library

[20]

P. Pirolli, The Use of Proximal Information Scent to Forage for Distal Content on the World Wide Web. In Working with Technology in Mind: Brunswikian. Resources for Cognitive Science and Engineering, Oxford University Press, 2004

[21]

F. Radlinski and T. Joachims, Query Chains: Learning to Rank from Implicit Feedback. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (SIGKDD), 2005.

Digital Library

[22]

F. Radlinski and T. Joachims, Evaluating the Robustness of Learning from Implicit Feedback, in Proceedings of the ICML Workshop on Learning in Web Search, 2005

[23]

S. E. Robertson, H. Zaragoza, and M. Taylor, Simple BM25 extension to multiple weighted fields, in Proceedings of the Conference on Information and Knowledge Management (CIKM), 2004

Digital Library

[24]

G. Salton & M. McGill. Introduction to modern information retrieval. McGraw-Hill, 1983

Digital Library

[25]

E.M. Voorhees, D. Harman, Overview of TREC, 2001

[26]

G.R. Xue, H.J. Zeng, Z. Chen, Y. Yu, W.Y. Ma, W.S. Xi, and W.G. Fan, Optimizing web search using web click-through data, in Proceedings of the Conference on Information and Knowledge Management (CIKM), 2004

Digital Library

[27]

H. Zaragoza, N. Craswell, M. Taylor, S. Saria, and S. Robertson. Microsoft Cambridge at TREC 13: Web and Hard Tracks. In Proceedings of TREC 2004

Cited By

Rieger AKulane SGadiraju UPera M(2024)Disentangling Web Search on Debated Topics: A User-Centered ExplorationProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659559(24-35)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659559
Yang THan CLuo CGupta PPhillips JAi QChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Mitigating Exploitation Bias in Learning to Rank with an Uncertainty-aware Empirical Bayes ApproachProceedings of the ACM on Web Conference 202410.1145/3589334.3645487(1486-1496)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645487
Skjæveland MBalog KBernard NŁajewska WLinjordet T(2024)An ecosystem for personal knowledge graphs: A survey and research roadmapAI Open10.1016/j.aiopen.2024.01.0035(55-69)Online publication date: 2024
https://doi.org/10.1016/j.aiopen.2024.01.003
Show More Cited By

Index Terms

Improving web search ranking by incorporating user behavior information
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Relevance assessment
    2. Information retrieval query processing
  2. World Wide Web
    1. Web applications
    2. Web services

Recommendations

Improving Web Search Ranking by Incorporating User Behavior Information

We show that incorporating user behavior data can significantly improve ordering of top results in real web search setting. We examine alternatives for incorporating feedback into the ranking process and explore the contributions of user feedback ...
Read More
Improving Ranking Consistency for Web Search by Leveraging a Knowledge Base and Search Logs
CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management

In this paper, we propose a new idea called ranking consistency in web search. Relevance ranking is one of the biggest problems in creating an effective web search system. Given some queries with similar search intents, conventional approaches typically ...
Read More
Identifying popular search goals behind search queries to improve web search ranking
AIRS'11: Proceedings of the 7th Asia conference on Information Retrieval Technology

Web users usually have a certain search goal before they submit a search query. However, many laypersons can't transform their search goals into suitable queries. Thus, understanding original search goals behind a query is very important for search ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '06: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval

August 2006

768 pages

ISBN:1595933697

DOI:10.1145/1148170

General Chair:
Efthimis N. Efthimiadis
University of Washington
,
Program Chairs:
Susan Dumais
Microsoft Research, Redmond
,
David Hawking
CSIRO ICT Centre, Canberra, Australia
,
Kalervo Järvelin,
University of Tampere, Finland

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 August 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGIR06

Sponsor:

SIGIR06: The 29th Annual International SIGIR Conference

August 6 - 11, 2006

Washington, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

698
Total Citations
View Citations
8,102
Total Downloads

Downloads (Last 12 months)91
Downloads (Last 6 weeks)4

Other Metrics

View Author Metrics

Citations

Cited By

Rieger AKulane SGadiraju UPera M(2024)Disentangling Web Search on Debated Topics: A User-Centered ExplorationProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659559(24-35)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659559
Yang THan CLuo CGupta PPhillips JAi QChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Mitigating Exploitation Bias in Learning to Rank with an Uncertainty-aware Empirical Bayes ApproachProceedings of the ACM on Web Conference 202410.1145/3589334.3645487(1486-1496)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645487
Skjæveland MBalog KBernard NŁajewska WLinjordet T(2024)An ecosystem for personal knowledge graphs: A survey and research roadmapAI Open10.1016/j.aiopen.2024.01.0035(55-69)Online publication date: 2024
https://doi.org/10.1016/j.aiopen.2024.01.003
Benzarti STebourski WKaraa W(2024)Toward a Deep Multimodal Interactive Query Expansion for Healthcare Information Retrieval EffectivenessAdvanced Information Networking and Applications10.1007/978-3-031-57853-3_31(369-379)Online publication date: 10-Apr-2024
https://doi.org/10.1007/978-3-031-57853-3_31
Ssemwogerere RSajo AMutwalibi NMzee A(2023)A Survey About the Application of Artificial Intelligence in Search EnginesHandbook of Research on AI Methods and Applications in Computer Engineering10.4018/978-1-6684-6937-8.ch008(161-178)Online publication date: 3-Mar-2023
https://doi.org/10.4018/978-1-6684-6937-8.ch008
Pergantis MVarlamis IKanellopoulos NGiannakoulopoulos A(2023)Searching Online for Art and Culture: User Behavior AnalysisFuture Internet10.3390/fi1506021115:6(211)Online publication date: 11-Jun-2023
https://doi.org/10.3390/fi15060211
Pergantis MKouretsis AGiannakoulopoulos A(2023)Investigating Online Art Search through Quantitative Behavioral Data and Machine Learning TechniquesAnalytics10.3390/analytics20200212:2(359-392)Online publication date: 26-Apr-2023
https://doi.org/10.3390/analytics2020021
Li YLiu YDai XLin JLai HLiu YYu Y(2023)Adversarially Trained Environment Models Are Effective Policy Evaluators and Improvers - An Application to Information RetrievalProceedings of the Fifth International Conference on Distributed Artificial Intelligence10.1145/3627676.3627680(1-12)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3627676.3627680
Chi AAnderson BReiter M(2023)Prioritizing Remediation of Enterprise Hosts by Malware Execution RiskProceedings of the 39th Annual Computer Security Applications Conference10.1145/3627106.3627180(550-564)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3627106.3627180
Breuer TFuhr NSchaer P(2023)Validating Synthetic Usage Data in Living Lab EnvironmentsJournal of Data and Information Quality10.1145/3623640Online publication date: 24-Sep-2023
https://dl.acm.org/doi/10.1145/3623640
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents