Article

Viewing term proximity from a different perspective

Authors:

Michael J. Taylor,

Hsiao-Wuen Hon,

Yong YuAuthors Info & Claims

ECIR'08: Proceedings of the IR research, 30th European conference on Advances in information retrieval

Pages 346 - 357

Published: 30 March 2008 Publication History

Abstract

This paper extends the state-of-the-art probabilistic model BM25 to utilize term proximity from a new perspective. Most previous work only consider dependencies between pairs of terms, and regard phrases as additional independent evidence. It is difficult to estimate the importance of a phrase and its extra contribution to a relevance score, as the phrase actually overlaps with the component terms. This paper proposes a new approach. First, query terms are grouped locally into non-overlapping phrases that may contain one or more query terms. Second, these phrases are not scored independently but are instead treated as providing a context for the component query terms. The relevance contribution of a term occurrence is measured by how many query terms occur in the context phrase and how compact they are. Third, we replace term frequency by the accumulated relevance contribution. Consequently, term proximity is easily integrated into the probabilistic model. Experimental results on TREC-10 and TREC-11 collections show stable improvements in terms of average precision and significant improvements in terms of top precisions.

References

[1]

Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. In: Proceedings of the 7th International World Wide Web Conference, Brisbane, Australia (1998).

Digital Library

[2]

Büttcher, S., Clarke, C.L.A., Lushman, B.: Term proximity scoring for ad-hoc retrieval on very large text collections. In: Proceedings of 29th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval (2006).

Digital Library

[3]

Clarke, C.L.A., Cormack, G.V., Burkowski, F.J.: Shortest substring ranking (multitext experiments for TREC-4). In: Proceedings of TREC-4 (1995).

[4]

Clarke, C.L.A., Cormack, G.V., Tudhope, E.A.: Relevance ranking for one to three term queries. Information Processing & Management 36(2), 291-311 (2000).

Digital Library

[5]

Croft, W.B., Turtle, H.R., Lewis, D.D.: The use of phrases and structured queries in information retrieval. In: Proceedings of 14th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 32-45 (1991).

Digital Library

[6]

Croft, W.B.: Boolean queries and term dependencies in probabilistic retrieval models. JASIS 37(2), 71-77 (1986).

[7]

CSIRO, TREC Web Tracks home page, http://www.ted.cmis.csiro.au/TRECWeb/

[8]

Fagan, J.L.: Automatic phrase indexing for document retrieval: An examination of syntactic and non-syntactic methods. In: Proceedings of 10th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 91-101 (1987).

Digital Library

[9]

Fox, C.: A stop list for general text. In: SIGIR Forum, December 1990, vol. 24(4), pp. 19- 35. ACM Press, New York (1990).

Digital Library

[10]

Gao, J., Nie, J.-Y., Wu, G., Cao, G.: Dependence language model for information retrieval. In: Proceedings of 27th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 170-177 (2004).

Digital Library

[11]

Harman, D.K.: Overview of the fourth Text Retrieval Conference (TREC-4). In: Proceedings of TREC-4, pp. 1-24.

[12]

Harper, D.J., van Rijsbergen, C.J.: An evaluation of feedback in document retrieval using co-occurrence data. Journal of Documentation 34, 189-216.

[13]

Harper, D.J., van Rijsbergen, C.J.: An evaluation of feedback in document retrieval using co-occurrence data. Journal of Documentation 34, 189-216.

[14]

Hawking, D., Thistlewaite, P.: Proximity operators - So near and yet so far. In: Proceedings of TREC-4, pp. 131-143 (1995).

[15]

Hawking, D., Thistlewaite, P.: Relevance weighting using distance between term occurrences. Computer Science Technical Report TR-CS-96-08, Australian National University (August 1996).

[16]

Losee Jr., R.M.: Term dependence: truncating the Bahadur Lazarsfeld expansion. Information Processing and Management 30, 293-303 (1994).

Digital Library

[17]

Metzler, D., Croft, W.B.: A Markov random field model for term dependencies. In: Proceedings of 28th Ann. Intl. ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 472-479 (2005).

Digital Library

[18]

Mishne, G., de Rijke, M.: Boosting web retrieval through query operations. In: Losada, D.E., Fernández-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 502-516. Springer, Heidelberg (2005).

Digital Library

[19]

Nallapati, R., Allan, J.: Capturing term dependencies using a language model on sentence trees. In: Proceedings of the 2002 ACM CIKM Intl. Conf. on Information and Knowledge Management, pp. 383-390 (2002).

Digital Library

[20]

Porter, M.: An algorithm for suffix stripping. Program 14(3), 130-137 (1980).

[21]

Pratt, E.J.: Complete poems. University of Toronto Press (1989).

[22]

Rasolofo, Y., Savoy, J.: Term proximity scoring for keyword-based retrieval systems. In: Sebastiani, F. (ed.) ECIR 2003. LNCS, vol. 2633, pp. 207-218. Springer, Heidelberg (2003).

Digital Library

[23]

Robertson, S.E., Spark Jones, K.: Relevance weighting for search terms. Journal of the American Society for Information Science 27(3), 129-146 (1976).

[24]

Robertson, S.E., Walker, S., Beaulieu, M.: Experimentation as a way of life: Okapi at TREC. Information Processing & Management 36(1), 95-108 (2000).

Digital Library

[25]

Song, F., Croft, W.B.: A general language model for information retrieval. In: Proceedings of CIKM 1999, pp. 316-321 (1999).

Digital Library

[26]

Spink, A., Wolfram, D., Jansen, B.J., Saracevic, T.: Searching the Web: The public and their queries. Journal of the American Society for Information Science and Technology 52(3), 226-234 (2001).

Digital Library

[27]

Srikanth, M., Srikanth, R.: Biterm language models for document retrieval. In: Proceedings of SIGIR 2002, pp. 425-426 (2002).

Digital Library

[28]

van Rijsbergen, C.J.: A theoretical basis for the use of cooccurrence data in retrieval. Journal of Documentation 33(2), 106-119 (1977).

[29]

Yu, C.T., Buckley, C., Lam, K., Salton, G.: A generalized term dependence in information retrieval. Technical report (1983).

Digital Library

Cited By

Kumar RBhanodai GPamula R(2019)Book search using social information, user profiles and query expansion with Pseudo Relevance FeedbackApplied Intelligence10.1007/s10489-018-1383-z49:6(2178-2200)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s10489-018-1383-z
Pitis SKeppens JGovernatori G(2017)Methods for retrieving alternative contract language using a prototypeProceedings of the 16th edition of the International Conference on Articial Intelligence and Law10.1145/3086512.3086530(179-187)Online publication date: 12-Jun-2017
https://dl.acm.org/doi/10.1145/3086512.3086530
Lu XMoffat ACulpepper JCarterette BFang HLalmas MNie J(2016)Efficient and Effective Higher Order Proximity ModelingProceedings of the 2016 ACM International Conference on the Theory of Information Retrieval10.1145/2970398.2970404(21-30)Online publication date: 12-Sep-2016
https://dl.acm.org/doi/10.1145/2970398.2970404
Show More Cited By

Viewing term proximity from a different perspective
1. Information systems
  1. Information retrieval

Recommendations

Hybrid term indexing for different IR models
IRAL '00: Proceedings of the fifth international workshop on on Information retrieval with Asian languages

Retrieval effectiveness depends on how terms are extracted and indexed. For Chinese text (and others like Japanese and Korean), there are no space to delimit words. Indexing using hybrid terms (i.e. words and bigrams) were able to achieve the best ...
Term Proximity Constraints for Pseudo-Relevance Feedback
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pseudo-relevance feedback (PRF) refers to a query expansion strategy based on top-retrieved documents, which has been shown to be highly effective in many retrieval models. Previous work has introduced a set of constraints (axioms) that should be ...
Lexical cohesion and term proximity in document ranking

We demonstrate effective new methods of document ranking based on lexical cohesive relationships between query terms. The proposed methods rely solely on the lexical relationships between original query terms, and do not involve query expansion or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ECIR'08: Proceedings of the IR research, 30th European conference on Advances in information retrieval

March 2008

718 pages

ISBN:3540786457

Editors:
Craig Macdonald
University of Glasgow, Department of Computing Science, Glasgow, UK
,
Iadh Ounis
University of Glasgow, Department of Computing Science, Glasgow, UK
,
Vassilis Plachouras
Yahoo! Research, Barcelona, Spain
,
Ian Ruthven
University of Strathclyde, Department of Computing and Information Sciences, Glasgow, UK
,
Ryen W. White
Microsoft Research, Redmond, WA

Sponsors

Yahoo! Research
Google Inc.
Microsoft Research: Microsoft Research
Matrixware Information Services

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval
British Computer Society: BCS

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 30 March 2008

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
8
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kumar RBhanodai GPamula R(2019)Book search using social information, user profiles and query expansion with Pseudo Relevance FeedbackApplied Intelligence10.1007/s10489-018-1383-z49:6(2178-2200)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s10489-018-1383-z
Pitis SKeppens JGovernatori G(2017)Methods for retrieving alternative contract language using a prototypeProceedings of the 16th edition of the International Conference on Articial Intelligence and Law10.1145/3086512.3086530(179-187)Online publication date: 12-Jun-2017
https://dl.acm.org/doi/10.1145/3086512.3086530
Lu XMoffat ACulpepper JCarterette BFang HLalmas MNie J(2016)Efficient and Effective Higher Order Proximity ModelingProceedings of the 2016 ACM International Conference on the Theory of Information Retrieval10.1145/2970398.2970404(21-30)Online publication date: 12-Sep-2016
https://dl.acm.org/doi/10.1145/2970398.2970404
Ye ZHuang J(2016)A learning to rank approach for quality-aware pseudo-relevance feedbackJournal of the Association for Information Science and Technology10.1002/asi.2343067:4(942-959)Online publication date: 1-Apr-2016
https://dl.acm.org/doi/10.1002/asi.23430
Saha Roy RKatare RGanguly NLaxman SChoudhury M(2015)Discovering and understanding word level user intent in Web search queriesWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2014.07.01030:C(22-38)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.websem.2014.07.010
Li HXu J(2014)Semantic Matching in SearchFoundations and Trends in Information Retrieval10.1561/15000000357:5(343-469)Online publication date: 12-Jun-2014
https://dl.acm.org/doi/10.1561/1500000035
Lu XMoffat ACulpepper J(2014)How Effective are Proximity Scores in Term Dependency Models?Proceedings of the 19th Australasian Document Computing Symposium10.1145/2682862.2682876(89-92)Online publication date: 26-Nov-2014
https://dl.acm.org/doi/10.1145/2682862.2682876
Huston SCroft WLi JWang XGarofalakis MSoboroff ISuel TWang M(2014)A Comparison of Retrieval Models using Term DependenciesProceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management10.1145/2661829.2661894(111-120)Online publication date: 3-Nov-2014
https://dl.acm.org/doi/10.1145/2661829.2661894
Ye ZHuang JGeva STrotman ABruza PClarke CJärvelin K(2014)A simple term frequency transformation model for effective pseudo relevance feedbackProceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval10.1145/2600428.2609636(323-332)Online publication date: 3-Jul-2014
https://dl.acm.org/doi/10.1145/2600428.2609636
Zhao JHuang JGeva STrotman ABruza PClarke CJärvelin K(2014)An enhanced context-sensitive proximity model for probabilistic information retrievalProceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval10.1145/2600428.2609527(1131-1134)Online publication date: 3-Jul-2014
https://dl.acm.org/doi/10.1145/2600428.2609527
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

Media

Figures

Other

Tables

View Table of Contents