Article

CLEF 2006: ad hoc track overview

Authors:

Giorgio M. Di Nunzio,

Carol PetersAuthors Info & Claims

CLEF'06: Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval

Pages 21 - 34

Published: 20 September 2006 Publication History

Abstract

We describe the objectives and organization of the CLEF 2006 ad hoc track and discuss the main characteristics of the tasks offered to test monolingual, bilingual, and multilingual textual document retrieval systems. The track was divided into two streams. The main stream offered mono- and bilingual tasks using the same collections as CLEF 2005: Bulgarian, English, French, Hungarian and Portuguese. The second stream, designed for more experienced participants, offered the so-called "robust task" which used test collections from previous years in six languages (Dutch, English, French, German, Italian and Spanish) with the objective of privileging experiments which achieve good stable performance over all queries rather than high average performance. The performance achieved for each task is presented and the results are commented. The document collections used were taken from the CLEF multilingual comparable corpus of news documents.

References

[1]

Braschler, M.: CLEF 2003 - Overview of results. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 44-63. Springer, Heidelberg (2004).

[2]

Di Nunzio, G.M., Ferro, N.: Appendix A. Results of the Core Tracks. In: Nardi, A., Peters, C., Vicedo, J.L. (eds.) Working Notes for the CLEF 2006 Workshop (2006), Published Online at www.clef-campaign.org.

[3]

Braschler, M., Peters, C.: CLEF 2003 Methodology and Metrics. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 7-20. Springer, Heidelberg (2004).

[4]

Halácsy, P., Trón, V.: Benefits of Deep NLP-based Lemmatization for Information Retrieval. {In this volume}.

[5]

Moreira Orengo, V., Buriol, L.S., Ramos Coelho, A.: A Study on the use of Stemming for Monolingual Ad-Hoc Portuguese. Information Retrieval (2006).

[6]

Azevedo Arcoverde, J.M., das Gracas Volpe Nunes, M.: NLP-Driven Constructive Learning for Filtering an IR Document Stream. {In this volume}.

[7]

Gonzalez, M., de Lima, V.L.S.: The PUCRS-PLN Group participation at CLEF 2006. {In this volume}.

[8]

Pingali, P., Tune, K.K., Varma, V.: Hindi, Telugu, Oromo, English CLIR Evaluation. {In this volume}.

[9]

Hayurani, H., Sari, S., Adriani, M.: Query and Document Translation for English-Indonesian Cross Language IR. {In this volume}.

[10]

Voorhees, E.M.: The TREC Robust Retrieval Track. SIGIR Forum 39, 11-20 (2005).

Digital Library

[11]

Savoy, J., Abdou, S.: Experiments with Monolingual, Bilingual, and Robust Retrieval. {In this volume}.

[12]

Voorhees, E.M.: Overview of the TREC 2005 Robust Retrieval Track. In: Voorhees, E.M., Buckland, L.P. (eds.): The Fourteenth Text REtrieval Conference Proceedings (TREC 2005) {last visited 2006, August 4} (2005), http://trec.nist.gov/pubs/trec14/t14_proceedings.html.

[13]

Martinez-Santiago, F., Montejo-Ráez, A., Garcia-Cumbreras, M., Ureña-Lopez, A.: SINAI at CLEF 2006 Ad-hoc Robust Multilingual Track: Query Expansion using the Google Search Engine. {In this volume} (2006).

[14]

Zazo, A., Berrocal, J., Figuerola, C.: Local Query Expansion Using Term Windows for Robust Retrieval. {In this volume}.

[15]

Tomlinson, S.: Comparing the Robustness of Expansion Techniques and Retrieval Measures. {In this volume}.

[16]

Goni-Menoyo, J., Gonzalez-Cristobal, J., Vilena-Román, J.: Report of the MIRACLE teach for the Ad-hoc track in CLEF 2006. In: Nardi, A., Peters, C., Vicedo, J.L. (eds.) Working Notes for the CLEF 2006 Workshop, Published Online (2006).

[17]

Hull, D.: Using Statistical Testing in the Evaluation of Retrieval Experiments. In: Korfhage, R., Rasmussen, E., Willett, P. (eds.) Proc. 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1993), pp. 329-338. ACM Press, New York (1993).

Digital Library

[18]

Conover, W.J.: Practical Nonparametric Statistics, 1st edn. John Wiley and Sons, New York (1971).

[19]

Judge, G.G., Hill, R.C., Griffiths, W.E., Lütkepohl, H., Lee, T.C.: Introduction to the Theory and Practice of Econometrics, 2nd edn. John Wiley and Sons, New York (1988).

[20]

Tague-Sutcliffe, J.: The Pragmatics of Information Retrieval Experimentation, Revisited. In: Sparck Jones, K., Willett, P. (eds.) Readings in Information Retrieval, pp. 205-216. Morgan Kaufmann Publisher, Inc, San Francisco, California (1997).

Digital Library

[21]

Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2006: Ad Hoc Track Overview. In: Nardi, A., Peters, C., Vicedo, J.L., eds.: Working Notes for the CLEF 2006 Workshop (2006) (last visited, March 23, 2007), http://www.clef-campaign. org/2006/working_notes/workingnotes2006/dinunzioOCLEF2006.pdf

Digital Library

Cited By

Vilares JAlonso MDoval YVilares M(2016)Studying the effect and treatment of misspelled queries in Cross-Language Information RetrievalInformation Processing and Management: an International Journal10.1016/j.ipm.2015.12.01052:4(646-657)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1016/j.ipm.2015.12.010
Ferro N(2014)CLEF 15th BirthdayACM SIGIR Forum10.1145/2701583.270158748:2(31-55)Online publication date: 23-Dec-2014
https://dl.acm.org/doi/10.1145/2701583.2701587
Leveling JJones GCrestani FMarchand-Maillet SChen HEfthimiadis ESavoy J(2010)Query recovery of short user queriesProceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval10.1145/1835449.1835589(733-734)Online publication date: 19-Jul-2010
https://dl.acm.org/doi/10.1145/1835449.1835589
Show More Cited By

Recommendations

JHU/APL ad hoc experiments at CLEF 2006
CLEF'06: Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval

JHU/APL continued its participation in multilingual retrieval at CLEF in 2006. We again applied our hallmark technique for combating language diversity and morphological complexity: character n-gram tokenization. This year we participated in the ad hoc ...
Dublin city university at CLEF 2004: experiments in monolingual, bilingual and multilingual retrieval
CLEF'04: Proceedings of the 5th conference on Cross-Language Evaluation Forum: multilingual Information Access for Text, Speech and Images

The Dublin City University group participated in the monolingual, bilingual and multilingual retrieval tasks. The main focus of our investigation for CLEF 2004 was extending our information retrieval system to document languages other than English, and ...
CLEF 2007: Ad Hoc Track Overview
Advances in Multilingual and Multimodal Information Retrieval

We describe the objectives and organization of the CLEF 2007 Ad Hoc track and discuss the main characteristics of the tasks offered to test monolingual and cross-language textual document retrieval systems. The track was divided into two streams. The ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

CLEF'06: Proceedings of the 7th international conference on Cross-Language Evaluation Forum: evaluation of multilingual and multi-modal information retrieval

September 2006

998 pages

ISBN:3540749985

Editors:
Carol Peters
ISTI-CNR, Pisa, Italy
,
Paul Clough
University of Sheffield, UK
,
Fredric C. Gey
University of California, Berkeley, CA
,
Jussi Karlgren
Swedish Institute of Computer Science, Kista, Sweden
,
Bernardo Magnini
FBK-Irst, Trento, Italy

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 20 September 2006

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vilares JAlonso MDoval YVilares M(2016)Studying the effect and treatment of misspelled queries in Cross-Language Information RetrievalInformation Processing and Management: an International Journal10.1016/j.ipm.2015.12.01052:4(646-657)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1016/j.ipm.2015.12.010
Ferro N(2014)CLEF 15th BirthdayACM SIGIR Forum10.1145/2701583.270158748:2(31-55)Online publication date: 23-Dec-2014
https://dl.acm.org/doi/10.1145/2701583.2701587
Leveling JJones GCrestani FMarchand-Maillet SChen HEfthimiadis ESavoy J(2010)Query recovery of short user queriesProceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval10.1145/1835449.1835589(733-734)Online publication date: 19-Jul-2010
https://dl.acm.org/doi/10.1145/1835449.1835589
Mandl TWomser-Hacker CDi Nunzio GFerro NWainwright RHaddad H(2008)How robust are multilingual information retrieval systems?Proceedings of the 2008 ACM symposium on Applied computing10.1145/1363686.1363949(1132-1136)Online publication date: 16-Mar-2008
https://dl.acm.org/doi/10.1145/1363686.1363949
Braschler MFerro N(2007)Adding multilingual information access to the European libraryProceedings of the 1st international conference on Digital libraries: research and development10.5555/1782334.1782361(218-227)Online publication date: 13-Feb-2007
https://dl.acm.org/doi/10.5555/1782334.1782361
Crivellari FDi Nunzio GFerro N(2007)How to compare bilingual to monolingual cross-language information retrievalProceedings of the 29th European conference on IR research10.5555/1763653.1763717(533-540)Online publication date: 2-Apr-2007
https://dl.acm.org/doi/10.5555/1763653.1763717

View Options

View options

Media

Figures

Other

Tables

View Table of Contents