Article

Condorcet fusion for improved retrieval

Authors:

Javed A. AslamAuthors Info & Claims

CIKM '02: Proceedings of the eleventh international conference on Information and knowledge management

Pages 538 - 548

https://doi.org/10.1145/584792.584881

Published: 04 November 2002 Publication History

Abstract

We present a new algorithm for improving retrieval results by combining document ranking functions: Condorcet-fuse. Beginning with one of the two major classes of voting procedures from Social Choice Theory, the Condorcet procedure, we apply a graph-theoretic analysis that yields a sorting-based algorithm that is elegant, efficient, and effective. The algorithm performs very well on TREC data, often outperforming existing metasearch algorithms whether or not relevance scores and training data is available. Condorcet-fuse significantly outperforms Borda-fuse, the analogous representative from the other major class of voting algorithms.

References

[1]

J. A. Aslam and M. Montague. Models for metasearch. In Croft et~al. {7}, pages 276--284.

Digital Library

[2]

B. T. Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, University of California, San Diego, 1994.

Digital Library

[3]

B. T. Bartell, G. W. Cottrell, and R. K. Belew. Automatic combination of multiple ranked retrieval systems. In W. B. Croft and C. van Rijsbergen, editors, SIGIR'94, Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 173--181, Dublin, Ireland, July 1994. Springer-Verlag, London.

Digital Library

[4]

N. Belkin, P. Kantor, C. Cool, and R. Quatrain. Combining evidence for information retrieval. In Harman {15}, pages 35--43.

[5]

N. Craswell, D. Hawking, and P. Thistlewaite. Merging results from isolated search engines. In Proceedings of the Tenth Australasian Database Conference, Aukland, New Zealand, Jan. 1999. Springer-Verlag.

[6]

W. B. Croft. Combining approaches to information retrieval. In W. B. Croft, editor, Advances in Information Retrieval: Recent Research from the Center for Intelligent Information Retrieval, The Kluwer International Series on Information Retrieval, chapter~1. Kluwer Academic Publishers, 2000.

Digital Library

[7]

W. B. Croft, D. J. Harper, D. H. Kraft, and J. Zobel, editors. SIGIR'01, Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New Orleans, Louisiana, USA, Sept. 2001. ACM Press, New York.

[8]

J. C. de~Borda. Mémoire sur les élections au scrutin. In Histoire de l'Academie Royale des Sciences. Paris, 1781.

[9]

M. de~Condorcet. Essai sur l'application de l'analyse à la probabilité des decisions rendues à la pluralité des voix, 1785.

[10]

H. L. Fisher and D. R. Elchesen. Effectiveness of combining title words and index terms in machine retrieval searches. Nature, 238:109--110, July 1972.

[11]

E. Fox, P. Ingwersen, and R. Fidel, editors. SIGIR'95, Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, July 1995. ACM Press, New York.

[12]

E. A. Fox, M. P. Koushik, J. Shaw, R. Modlin, and D. Rao. Combining evidence from multiple searches. In D. Harman, editor, The First Text REtrieval Conference (TREC-1), pages 319--328, Gaithersburg, MD, USA, Mar. 1993. U.S. Government Printing Office, Washington D.C.

[13]

E. A. Fox and J. A. Shaw. Combination of multiple searches. In Harman {15}, pages 243--249.

[14]

K. L. Fox, O. Frieder, M. Knepper, and E. Snowberg. SENTINEL: A multiple engine information retrieval and visualization system. Journal of the ASIS, 50(7), May 1999.

Digital Library

[15]

D. Harman, editor. The Second Text REtrieval Conference (TREC-2), Gaithersburg, MD, USA, Mar. 1994. U.S. Government Printing Office, Washington D.C.

[16]

D. A. Hull, J. O. Pedersen, and H. Schütze. Method combination for document filtering. In H.-P. Frei, D. Harman, P. Schäuble, and R. Wilkinson, editors, SIGIR'96, Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 279--287, Zurich, Switzerland, Aug. 1996. ACM Press, New York.

Digital Library

[17]

J. S. Kelly. Social Choice Theory: An Introduction. Springer-Verlag, 1988.

[18]

J. H. Lee. Combining multiple evidence from different properties of weighting schemes. In Fox et~al. {11}, pages 180--188.

Digital Library

[19]

J. H. Lee. Analyses of multiple evidence combination. In N. J. Belkin, A. D. Narasimhalu, and P. Willett, editors, SIGIR'97, Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 267--275, Philadelphia, Pennsylvania, USA, July 1997. ACM Press, New York.

Digital Library

[20]

M. Montague and J. A. Aslam. Metasearch consistency. In Croft et~al. {7}, pages 386--387.

Digital Library

[21]

H. Moulin. Axioms of Cooperative Decision Making. Cambridge University Press, 1988.

[22]

K. B. Ng. An Investigation of the Conditions for Effective Data Fusion in Information Retrieval. PhD thesis, School of Communication, Information, and Library Studies, Rutgers University, 1998.

[23]

K. B. Ng and P. B. Kantor. An investigation of the preconditions for effective data fusion in IR: A pilot study. In Proceedings of the 61th Annual Meeting of the American Society for Information Science, 1998.

[24]

K. B. Ng, D. Loewenstern, C. Basu, H. Hirsh, and P. B. Kantor. Data fusion of machine learning methods for the TREC5 routing task (and other work). In Voorhees and Harman {35}, pages 477--487.

[25]

W. H. Riker. Liberalism Against Populism. Waveland Press, 1982.

[26]

E. W. Selberg. Towards Comprehensive Web Search. PhD thesis, University of Washington, 1999.

Digital Library

[27]

J. A. Shaw and E. A. Fox. Combination of multiple searches. In D. Harman, editor, Overview of the Third Text REtrieval Conference (TREC-3), pages 105--108, Gaithersburg, MD, USA, Apr. 1995. U.S. Government Printing Office, Washington D.C.

[28]

B. Shu and S. Kak. A neural network-based intelligent metasearch engine. Information Sciences, 120:1--11, 1999.

Digital Library

[29]

P. Thompson. A combination of expert opinion approach to probabilistic information retrieval, part 1: the conceptual model. Information Processing and Management, 26(3):371--382, 1990.

Digital Library

[30]

P. Thompson. A combination of expert opinion approach to probabilistic information retrieval, part 2: mathematical treatment of CEO model 3. Information Processing and Management, 26(3):383--394, 1990.

Digital Library

[31]

C. C. Vogt. Adaptive Combination of Evidence for Information Retrieval. PhD thesis, University of California, San Diego, 1999.

Digital Library

[32]

C. C. Vogt. How much more is better? Characterizing the effects of adding more IR systems to a combination. In Content-Based Multimedia Information Access (RIAO), pages 457--475, Paris, France, Apr. 2000.

[33]

C. C. Vogt and G. W. Cottrell. Fusion via a linear combination of scores. Information Retrieval, 1(3):151--173, Oct. 1999.

Digital Library

[34]

C. C. Vogt, G. W. Cottrell, R. K. Belew, and B. T. Bartell. Using relevance to train a linear mixture of experts. In Voorhees and Harman {35}, pages 503--515.

[35]

E. Voorhees and D. Harman, editors. The Fifth Text REtrieval Conference (TREC-5), Gaithersburg, MD, USA, 1997. U.S. Government Printing Office, Washington D.C.

[36]

E. M. Voorhees, N. K. Gupta, and B. Johnson-Laird. Learning collection fusion strategies. In Fox et~al. {11}, pages 172--179.

Digital Library

Cited By

Bałchanowski MBoryczka U(2024)How Normalization Strategies Affect the Quality of Rank Aggregation Methods in Recommendation SystemsProcedia Computer Science10.1016/j.procs.2023.10.174225:C(1843-1852)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.procs.2023.10.174
Askari AAbolghasemi APasi GKraaij WVerberne S(2024)Injecting the score of the first-stage retriever as text improves BERT-based re-rankersDiscover Computing10.1007/s10791-024-09435-827:1Online publication date: 26-Jun-2024
https://doi.org/10.1007/s10791-024-09435-8
Tan XYu WTan L(2024)Large-Scale Rank Aggregation from Multiple Data Sources Based D3MOPSO MethodWeb and Big Data10.1007/978-981-97-2303-4_5(63-80)Online publication date: 29-May-2024
https://doi.org/10.1007/978-981-97-2303-4_5
Show More Cited By

Condorcet fusion for improved retrieval
1. Information systems

Recommendations

Reciprocal rank fusion outperforms condorcet and individual rank learning methods
SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval

Reciprocal Rank Fusion (RRF), a simple method for combining the document rankings from multiple IR systems, consistently yields better results than any individual system, and better results than the standard method Condorcet Fuse. This result is ...
The weighted Condorcet fusion in information retrieval

The Condorcet fusion is a distinctive fusion method and was found useful in information retrieval. Two basic requirements for the Condorcet fusion to improve retrieval effectiveness are: (1) all component systems involved should be more or less equally ...
Condorcet Fusion for Blog Opinion Retrieval
DEXA '12: Proceedings of the 2012 23rd International Workshop on Database and Expert Systems Applications

Blogs have been popular social networking platforms in recent years. Blog opinion retrieval is one of the key issues that needs to be solved. In this paper, we investigate if the Condorcet fusion and the weighted Condorcet fusion can be used for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '02: Proceedings of the eleventh international conference on Information and knowledge management

November 2002

704 pages

ISBN:1581134924

DOI:10.1145/584792

General Chair:
Charles Nicholas
University of Maryland Baltimore County
,
Program Chairs:
David Grossman
Illinois Institute of Technology
,
Konstantinos Kalpakis
University of Maryland Baltimore County
,
Sajda Qureshi
Erasmus University, Rotterdam
,
Han van Dissel
Erasmus University, Rotterdam
,
Len Seligman
The MITRE Corporation

Copyright © 2002 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 November 2002

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

CIKM02

Sponsor:

CIKM02: Eleventh ACM International Conference on Information and Knowledge Management

November 4 - 9, 2002

Virginia, McLean, USA

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

209
Total Citations
View Citations
1,239
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bałchanowski MBoryczka U(2024)How Normalization Strategies Affect the Quality of Rank Aggregation Methods in Recommendation SystemsProcedia Computer Science10.1016/j.procs.2023.10.174225:C(1843-1852)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.procs.2023.10.174
Askari AAbolghasemi APasi GKraaij WVerberne S(2024)Injecting the score of the first-stage retriever as text improves BERT-based re-rankersDiscover Computing10.1007/s10791-024-09435-827:1Online publication date: 26-Jun-2024
https://doi.org/10.1007/s10791-024-09435-8
Tan XYu WTan L(2024)Large-Scale Rank Aggregation from Multiple Data Sources Based D3MOPSO MethodWeb and Big Data10.1007/978-981-97-2303-4_5(63-80)Online publication date: 29-May-2024
https://doi.org/10.1007/978-981-97-2303-4_5
Mukhopadhyay ARay SMaulik UBandyopadhyay SMukhopadhyay ARay SMaulik UBandyopadhyay S(2024)Multiobjective Rank Aggregation for Gene PrioritizationMultiobjective Optimization Algorithms for Bioinformatics10.1007/978-981-97-1631-9_3(41-73)Online publication date: 4-Mar-2024
https://doi.org/10.1007/978-981-97-1631-9_3
Bałchanowski MBoryczka U(2023)A Comparative Study of Rank Aggregation Methods in Recommendation SystemsEntropy10.3390/e2501013225:1(132)Online publication date: 9-Jan-2023
https://doi.org/10.3390/e25010132
Feng SDeng QWang SSong LLiang C(2023)An Experimental Study of Unsupervised Rank Aggregation Methods in World University Rankings2023 International Conference on Intelligent Education and Intelligent Research (IEIR)10.1109/IEIR59294.2023.10391254(1-8)Online publication date: 5-Nov-2023
https://doi.org/10.1109/IEIR59294.2023.10391254
Rashidpoor Toochaei MMoeini F(2023)Evaluating the performance of ensemble classifiers in stock returns prediction using effective featuresExpert Systems with Applications10.1016/j.eswa.2022.119186213(119186)Online publication date: Mar-2023
https://doi.org/10.1016/j.eswa.2022.119186
Garba AWu SKhalid S(2023)Federated search techniques: an overview of the trends and state of the artKnowledge and Information Systems10.1007/s10115-023-01922-665:12(5065-5095)Online publication date: 10-Jul-2023
https://doi.org/10.1007/s10115-023-01922-6
Zhang ZWu S(2023)Data Fusion Performance Prophecy: A Random Forest RevelationInformation Integration and Web Intelligence10.1007/978-3-031-48316-5_20(192-200)Online publication date: 22-Nov-2023
https://doi.org/10.1007/978-3-031-48316-5_20
Ma TSu YRong HQian YAl-Nabhan N(2022)Rule Fusion of Privacy Protection Strategies for Co-Ownership Data SharingMathematics10.3390/math1006096910:6(969)Online publication date: 18-Mar-2022
https://doi.org/10.3390/math10060969
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents