research-article

Utilizing a geometry of context for enhanced implicit feedback

Authors:

Massimo Melucci,

Ryen W. WhiteAuthors Info & Claims

CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management

Pages 273 - 282

https://doi.org/10.1145/1321440.1321480

Published: 06 November 2007 Publication History

Abstract

Implicit feedback algorithms utilize interaction between searchers and search systems to learn more about users' needs and interests than expressed in query statements alone. This additional information can be used to formulate improved queries or directly improve retrieval performance. In this paper we present a geometric framework that utilizes multiple sources of evidence present in this interaction context (e.g., display time, document retention) to develop enhanced implicit feedback models personalized for each user and tailored for each search task. We use rich interaction logs (and associated metadata such as relevance judgments), gathered during a longitudinal user study, as relevance stimuli to compare an implicit feedback algorithm developed using the framework with alternative algorithms. Our findings demonstrate both the effectiveness of our proposed algorithm and the potential value of incorporating multiple sources of interaction evidence when developing implicit feedback algorithms.

References

[1]

N. J. Belkin. Helping people find what they don't know. Comm. of ACM, 43(8):58--61, 2000.

Digital Library

[2]

G. W. Furnas, T. K. Landauer, L. M. Gomez, and S. Dumais. The vocabulary problem in human-system communication. Comm. of the ACM, 30(11):964--971, 1987.

Digital Library

[3]

G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4):288--297, 1990.

[4]

C. Buckley, G. Salton, and J. Allan. The effect of adding relevance information in a relevance feedback environment. Proc. of SIGIR, pp. 292--300, New York, NY, USA, 1994. Springer-Verlag New York, Inc.

Digital Library

[5]

M. Beaulieu. Experiments with interfaces to support query expansion. J. of IR, 53(1):8--19, 1997.

[6]

D. Kelly and J. Teevan. Implicit feedback for inferring user preference: a bibliography. SIGIR Forum, 37(2):18--28, 2003.

Digital Library

[7]

M. Morita and Y. Shinoda. Information filtering based on user behavior analysis and best match text retrieval. Proc. of SIGIR, pp. 272--281, New York, NY, USA, 1994. Springer-Verlag New York, Inc.

Digital Library

[8]

M. Claypool, P. Le, M. Wased, and D. Brown. Implicit interest indicators. Proc. of the Int.l Conf. on Intelligent User Interfaces, pp. 33--40, New York, NY, USA, 2001.

Digital Library

[9]

C. Vogt. Passive feedback collection - an attempt to debunk the myth of clickthroughs. Proc. of TREC, pp. 141--150, Gaithersburg, MD, 2000.

[10]

E. Agichtein, E. Brill, and S. Dumais. Improving web search ranking by incorporating user behavior information. Proc. of SIGIR, pp. 19--26, New York, NY, USA, 2006. ACM Press.

Digital Library

[11]

R. W. White and D. Kelly. A study on the effects of personalization and task information on implicit feedback performance. Proc.of CIKM, pp. 297--306, New York, NY, USA, 2006. ACM Press.

Digital Library

[12]

J. Teevan, S. Dumais, and E. Horvitz. Personalizing search via automated analysis of interests and activities. Proc. of SIGIR, pp. 449--456, New York, NY, USA, 2005. ACM Press.

Digital Library

[13]

S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T. White. Evaluating implicit measures to improve web search. ACM TOIS, 23(2):147--168, 2005.

Digital Library

[14]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. Proc. of SIGIR, pp. 154--161, New York, NY, USA, 2005. ACM Press.

Digital Library

[15]

D. Kelly and N. J. Belkin. Display time as implicit feedback: understanding task effects. Proc. of SIGIR, pp. 377--384, New York, NY, USA, 2004. ACM Press.

Digital Library

[16]

R. W. White, I. Ruthven, and J. M. Jose. A study of factors affecting the utility of implicit relevance feedback. Proc. of SIGIR, pp. 35--42, New York, NY, USA, 2005. ACM Press.

Digital Library

[17]

J. Budzik and K. Hammond. Watson: Anticipating and contextualizing information needs. Meeting of the ASIS, pp. 727--740, Medford, NJ, 1999.

[18]

R. W. White, I. Ruthven, and J. M. Jose. Finding relevant documents using top ranking sentences: an evaluation of two alternative schemes. Proc. of SIGIR, pp. 57--64, New York, NY, USA, 2002. ACM Press.

Digital Library

[19]

J. Kim, D. W. Oard, and K. Romanik. Using implicit feedback for user modeling in internet and intranet searching. Technical report, College of Library and Information Services, University of Maryland at College Park, 2000.

[20]

H. Lieberman. Letizia: An agent that assists web browsing. In Chris S. Mellish, editor, Proc. of IJCAI, pp. 924--929, Montreal, Quebec, Canada, 1995. Morgan Kaufmann publishers Inc.: San Mateo, CA, USA.

Digital Library

[21]

T. Joachims, D. Freitag, and T. Mitchell. Webwatcher: A tour guide for the world wide web. Proc. of IJCAI, pp. 770--777, San Francisco, CA, 1997. Morgan Kaufmann.

[22]

E. Horvitz, J. Breese, D. Heckerman, D. Hovel, and K. Rommelse. The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users. Proc. of the Conf. on Uncertainty in Artificial Intelligence, pp. 256--265, Madison, WI, July 1998.

Digital Library

[23]

T. Joachims. Optimizing search engines using clickthrough data. Proc. of SIGKDD, pp. 133--142, New York, NY, USA, 2002. ACM Press.

Digital Library

[24]

F. Radlinski and T. Joachims. Query chains: learning to rank from implicit feedback. Proc. of SIGKDD, pp. 239--248, New York, NY, USA, 2005.

Digital Library

[25]

R. W. White, M. Bilenko, and S. Cucerzan. Studying the use of popular destinations to enhance web search interaction. Proc. of SIGIR, pp. 159--166, New York, NY, USA, 2007.

Digital Library

[26]

G. Salton, A. Wong, and C. S. Yang. A vector space model for automatic indexing. Comm. of the ACM, 18(11):613--620, November 1975.

Digital Library

[27]

G. Salton. Automatic Text Processing. Addison-Wesley, 1989.

Digital Library

[28]

S. K. M. Wong and V. V. Raghavan. Vector space model of information retrieval - a reevaluation. Proc. of SIGIR, pp. 167--185, Cambridge, England, 1984.

Digital Library

[29]

D. Dubin. The most influential paper Gerard Salton never wrote. Library Trends, 52(4):748--764, 2004.

[30]

C. J. van Rijsbergen. The Geometry of Information Retrieval. Cambridge University Press, UK, 2004.

Digital Library

[31]

M. Melucci. Context modeling and discovery using vector space bases. Proc. of CIKM, pp. 808--815, Bremen, Germany, November 2005.

Digital Library

[32]

M. Melucci. Ranking in context using vector spaces. Proc. of CIKM, pp. 477--478, Arlington, VA, USA, November 2006.

Digital Library

[33]

M. Melucci. Exploring a mechanics for context-aware information retrieval. Proc. of the AAAI Spring Symposium on Quantum Interaction, Stanford, CA, USA, March 2007. AAAI Press.

[34]

G. Salton. Mathematics and information retrieval. J. of Documentation, 35(1):1--29, 1979.

[35]

S. Deerwester, S. T. Dumais, G. W. Furnas, T. Landauer, and R. Harshman. Indexing by latent semantic analysis. J. of Am. Soc. Inf. Science, 41(6):391--407, 1990.

[36]

P. Ingwersen and K. Jäarvelin. The Turn: Integration of Information Seeking and Retrieval in Context. Springer, 2005.

Digital Library

[37]

D. Kelly. Understanding Implicit Feedback And Document Preference: A Naturalistic User Study. PhD thesis, Rutgers, The State University of New Jersey, 2004.

Digital Library

[38]

W. Kraaij, T. Westerveld, and D. Hiemstra. The importance of prior probabilities for entry page search. Proc. of SIGIR, pp. 27--34, Tampere, Finland, 2002. ACM Press

Digital Library

[39]

K. Jarväelin and J. Kekäläinen. Cumulated gain-based evaluation of IR techniques. ACM TOIS, 20(4):422--446, 2002.

Digital Library

[40]

A. Singhal, C. Buckley, and M. Mitra. Pivoted document length normalization. Proc. of SIGIR, pp. 21--29, Zurich, Switzerland, 1996. ACM Press.

Digital Library

[41]

I. Campbell. Interactive evaluation of the ostensive model using a new test collection of images with multiple relevance assessments. J. of IR, 2(1):85--112, 2000.

Digital Library

Cited By

Yin XHuang JLi Z(2018)Re-ranking with context for high-performance biomedical information retrievalInternational Journal of Data Mining and Bioinformatics10.1504/IJDMB.2012.0481726:2(115-129)Online publication date: 23-Dec-2018
https://dl.acm.org/doi/10.1504/IJDMB.2012.048172
Melucci MMelucci M(2015)Future WorkIntroduction to Information Retrieval and Quantum Mechanics10.1007/978-3-662-48313-8_4(189-195)Online publication date: 9-Dec-2015
https://doi.org/10.1007/978-3-662-48313-8_4
Melucci MMelucci M(2015)Quantum Mechanics and Information RetrievalIntroduction to Information Retrieval and Quantum Mechanics10.1007/978-3-662-48313-8_3(101-188)Online publication date: 9-Dec-2015
https://doi.org/10.1007/978-3-662-48313-8_3
Show More Cited By

Index Terms

Utilizing a geometry of context for enhanced implicit feedback
1. Information systems
  1. Information retrieval

Recommendations

A study of factors affecting the utility of implicit relevance feedback
SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval

Implicit relevance feedback (IRF) is the process by which a search system unobtrusively gathers evidence on searcher interests from their interaction with the system. IRF is a new method of gathering information on user interest and, if IRF is to be ...
An effective implicit relevance feedback technique using affective, physiological and behavioural features
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

The effectiveness of various behavioural signals for implicit relevance feedback models has been exhaustively studied. Despite the advantages of such techniques for a real time information retrieval system, most of the behavioural signals are noisy and ...
Evaluating the implicit feedback models for adaptive video retrieval
MIR '07: Proceedings of the international workshop on Workshop on multimedia information retrieval

Interactive video retrieval systems are becoming popular. On the one hand, these systems try to reduce the effect of the semantic gap, an issue currently being addressed by the multimedia retrieval community. On the other hand, such systems enhance the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management

November 2007

1048 pages

ISBN:9781595938039

DOI:10.1145/1321440

Co-chair:
Alberto H. F. Laender,
Conference Chairs:
André O. Falcão
Universidade de Lisboa, Portugal
,
Øystein Haug Olsen,
General Chair:
Mário J. Silva
(Universidade de Lisboa, Portugal)
,
Program Chairs:
Ricardo Baeza-Yates,
Deborah L. McGuinness,
Bjorn Olstad

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM07

Sponsor:

CIKM07: Conference on Information and Knowledge Management

November 6 - 10, 2007

Lisbon, Portugal

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
392
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yin XHuang JLi Z(2018)Re-ranking with context for high-performance biomedical information retrievalInternational Journal of Data Mining and Bioinformatics10.1504/IJDMB.2012.0481726:2(115-129)Online publication date: 23-Dec-2018
https://dl.acm.org/doi/10.1504/IJDMB.2012.048172
Melucci MMelucci M(2015)Future WorkIntroduction to Information Retrieval and Quantum Mechanics10.1007/978-3-662-48313-8_4(189-195)Online publication date: 9-Dec-2015
https://doi.org/10.1007/978-3-662-48313-8_4
Melucci MMelucci M(2015)Quantum Mechanics and Information RetrievalIntroduction to Information Retrieval and Quantum Mechanics10.1007/978-3-662-48313-8_3(101-188)Online publication date: 9-Dec-2015
https://doi.org/10.1007/978-3-662-48313-8_3
Melucci MMelucci M(2015)Elements of Quantum MechanicsIntroduction to Information Retrieval and Quantum Mechanics10.1007/978-3-662-48313-8_2(53-100)Online publication date: 9-Dec-2015
https://doi.org/10.1007/978-3-662-48313-8_2
Melucci MMelucci M(2015)Elements of Information RetrievalIntroduction to Information Retrieval and Quantum Mechanics10.1007/978-3-662-48313-8_1(1-51)Online publication date: 9-Dec-2015
https://doi.org/10.1007/978-3-662-48313-8_1
Li JSong DZhang PWen JDou Z(2014)Personalizing Web Search Results Based on Subspace ProjectionInformation Retrieval Technology10.1007/978-3-319-12844-3_14(160-171)Online publication date: 2014
https://doi.org/10.1007/978-3-319-12844-3_14
Di Buccio EMelucci MSong D(2011)Combining interaction and content for feedback-based rankingProceedings of the Second international conference on Multidisciplinary information retrieval facility10.5555/2018142.2018150(46-61)Online publication date: 6-Jun-2011
https://dl.acm.org/doi/10.5555/2018142.2018150
Di Buccio EMelucci MNica AVarde A(2010)Toward the design of a methodology to predict relevance through multiple sources of evidenceProceedings of the 3rd workshop on Ph.D. students in information and knowledge management10.1145/1871902.1871918(83-86)Online publication date: 30-Oct-2010
https://dl.acm.org/doi/10.1145/1871902.1871918
Lee DPark SKahng MLee SLee S(2010)Exploiting Contextual Information from Event Logs for Personalized RecommendationComputer and Information Science 201010.1007/978-3-642-15405-8_11(121-139)Online publication date: 2010
https://doi.org/10.1007/978-3-642-15405-8_11
Yin XHuang XLi Z(2009)Towards a Better Ranking for Biomedical Information Retrieval Using ContextProceedings of the 2009 IEEE International Conference on Bioinformatics and Biomedicine10.1109/BIBM.2009.82(344-349)Online publication date: 1-Nov-2009
https://dl.acm.org/doi/10.1109/BIBM.2009.82
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten