Article

Seven years of INEX interactive retrieval experiments --- lessons and challenges

Authors:

Ragnar Nordlie,

Nils PharoAuthors Info & Claims

CLEF'12: Proceedings of the Third international conference on Information Access Evaluation: multilinguality, multimodality, and visual analytics

Pages 13 - 23

https://doi.org/10.1007/978-3-642-33247-0_2

Published: 17 September 2012 Publication History

Abstract

This paper summarizes a major effort in interactive search investigation, the INEX i-track, a collective effort run over a seven-year period. We present the experimental conditions, report some of the findings of the participating groups, and examine the challenges posed by this kind of collective experimental effort.

References

[1]

Gövert, N., Kazai, G.: Overview of the Initiative for the Evaluation of XML retrieval (INEX) 2002. Presented at the INEX Workshop (2002)

[2]

Kazai, G., Lalmas, M., Fuhr, N., Gövert, N.: A report on the first year of the INitiative for the evaluation of XML retrieval (INEX 2002). Journal of the American Society for Information Science and Technology 55, 551-556 (2004)

Digital Library

[3]

Tombros, A., Larsen, B., Malik, S.: The Interactive Track at INEX 2004. In: Fuhr, N., Lalmas, M., Malik, S., Szlávik, Z. (eds.) INEX 2004. LNCS, vol. 3493, pp. 410-423. Springer, Heidelberg (2005)

Digital Library

[4]

Fuhr, N., Gövert, N., Großjohann, K.: HyREX: Hyper-media Retrieval Engine for XML. University of Dortmund, Computer Science (2002)

[5]

Larsen, B., Malik, S., Tombros, A.: The Interactive Track at INEX 2005. In: Fuhr, N., Lalmas, M., Malik, S., Kazai, G. (eds.) INEX 2005. LNCS, vol. 3977, pp. 398-410. Springer, Heidelberg (2006)

Digital Library

[6]

Gövert, N., Fuhr, N., Klas, C.-P.: Daffodil: Distributed Agents for User-Friendly Access of Digital Libraries. In: Borbinha, J. L., Baker, T. (eds.) ECDL 2000. LNCS, vol. 1923, pp. 352-355. Springer, Heidelberg (2000)

Digital Library

[7]

Malik, S., Tombros, A., Larsen, B.: The Interactive Track at INEX 2006. In: Fuhr, N., Lalmas, M., Trotman, A. (eds.) INEX 2006. LNCS, vol. 4518, pp. 387-399. Springer, Heidelberg (2007)

[8]

Pharo, N., Nordlie, R., Fachry, K. N.: Overview of the INEX 2008 Interactive Track. In: Geva, S., Kamps, J., Trotman, A. (eds.) INEX 2008. LNCS, vol. 5631, pp. 300-313. Springer, Heidelberg (2009)

Digital Library

[9]

Pharo, N., Nordlie, R., Fuhr, N., Beckers, T., Fachry, K. N.: Overview of the INEX 2009 Interactive Track. In: Geva, S., Kamps, J., Trotman, A. (eds.) INEX 2009. LNCS, vol. 6203, pp. 303-311. Springer, Heidelberg (2010)

Digital Library

[10]

Pharo, N., Beckers, T., Nordlie, R., Fuhr, N.: Overview of the INEX 2010 Interactive Track. In: Geva, S., Kamps, J., Schenkel, R., Trotman, A. (eds.) INEX 2010. LNCS, vol. 6932, pp. 227-235. Springer, Heidelberg (2011)

Digital Library

[11]

Denoyer, L., Gallinari, P.: The Wikipedia XML corpus. SIGIR Forum 40, 64-69 (2006)

Digital Library

[12]

Borlund, P.: Evaluation of interactive information retrieval systems. Abo Akademis Forlag (2000)

[13]

Tombros, A., Ruthven, I., Jose, J. M.: How users assess Web pages for information seeking. Journal of the American Society for Information Science and Technology 56, 327-344 (2005)

Digital Library

[14]

Toms, E. G., O'Brien, H., Mackenzie, T., Jordan, C., Freund, L., Toze, S., Dawe, E., Mac-Nutt, A.: Task Effects on Interactive Search: The Query Factor. In: Fuhr, N., Kamps, J., Lalmas, M., Trotman, A. (eds.) INEX 2007. LNCS, vol. 4862, pp. 359-372. Springer, Heidelberg (2008)

Digital Library

[15]

Järvelin, K., Kekäläinen, J.: IR evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2000, Athens, Greece, pp. 41-48 (2000)

Digital Library

[16]

Pehcevski, J.: Relevance in XML Retrieval: The User Perspective. In: Trotman, A., Geva, S. (eds.) Proceedings of the SIGIR 2006 Workshop on XML Element Retrieval Methodology. University of Otago, Dunedin (2006)

[17]

Larsen, B., Tombros, A., Malik, S.: Is XML retrieval meaningful to users?: searcher preferences for full documents vs. elements. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 663-664. ACM, New York (2006)

Digital Library

[18]

Hammer-Aebi, B., Christensen, K. W., Lund, H., Larsen, B.: Users, structured documents and overlap: interactive searching of elements and the influence of context on search behaviour. In: Proceedings of the 1st International Conference on Information Interaction in Context, pp. 46-55. ACM, New York (2006)

Digital Library

[19]

Pharo, N.: The effect of granularity and order in XML element retrieval. Information Processing and Management 44, 1732-1740 (2008)

Digital Library

[20]

Pharo, N., Krahn, A.: The effect of task type on preferred element types in an XML-based retrieval system. Journal of the American Society for Information Science and Technology 62, 1717-1726 (2011)

Digital Library

[21]

Pharo, N., Nordlie, R.: Context Matters: An Analysis of Assessments of XML Documents. In: Crestani, F., Ruthven, I. (eds.) CoLIS 2005. LNCS, vol. 3507, pp. 238-248. Springer, Heidelberg (2005)

Digital Library

[22]

Kim, H., Son, H.: Users Interaction with the Hierarchically Structured Presentation in XML Document Retrieval. In: Fuhr, N., Lalmas, M., Malik, S., Kazai, G. (eds.) INEX 2005. LNCS, vol. 3977, pp. 422-431. Springer, Heidelberg (2006)

Digital Library

[23]

Ramírez, G., de Vries, A. P.: Relevant contextual features in XML retrieval. In: Proceedngs of the 1st International Conference on Information Interaction in Context, pp. 56-65. ACM, New York (2006)

Digital Library

Cited By

Gäde MKoolen MHall MBogers TPetras VScholer FThomas PElsweiler DJoho HKando NSmith C(2021)A Manifesto on Resource Re-Use in Interactive Information RetrievalProceedings of the 2021 Conference on Human Information Interaction and Retrieval10.1145/3406522.3446056(141-149)Online publication date: 14-Mar-2021
https://dl.acm.org/doi/10.1145/3406522.3446056
Bogers TGäde MMichael MFreund LKoolen MPetras VSkov M(2018)Report on the Workshop on Barriers to Interactive IR Resources Re-use (BIIRRR 2018)ACM SIGIR Forum10.1145/3274784.327479552:1(119-128)Online publication date: 31-Aug-2018
https://dl.acm.org/doi/10.1145/3274784.3274795
Ferro N(2014)CLEF 15th BirthdayACM SIGIR Forum10.1145/2701583.270158748:2(31-55)Online publication date: 23-Dec-2014
https://dl.acm.org/doi/10.1145/2701583.2701587

Seven years of INEX interactive retrieval experiments --- lessons and challenges
1. Information systems

Recommendations

Query length in interactive information retrieval
SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval

Query length in best-match information retrieval (IR) systems is well known to be positively related to effectiveness in the IR task, when measured in experimental, non-interactive environments. However, in operational, interactive IR systems, query ...
User Interactions with Search Systems
ICTIR '18: Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval

The proposed course on User Interactions with Search Systems provides students an overview on interactive information retrieval (IIR) evaluation studies that is specifically focused on the modelling of users, tasks and contexts, and methods for IIR ...
Toward an Interactive Patent Retrieval Framework based on Distributed Representations
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

We present a novel interactive framework for patent retrieval leveraging distributed representations of concepts and entities extracted from the patents text. We propose a simple and practical interactive relevance feedback mechanism where the user is ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

CLEF'12: Proceedings of the Third international conference on Information Access Evaluation: multilinguality, multimodality, and visual analytics

September 2012

143 pages

ISBN:9783642332463

Editors:
Tiziana Catarci
Department of Computer, Control and Management, Engenerring Antonio Ruberti, Sapienza University of Rome, Via Ariosto 25, Rome, Italy
,
Pamela Forner
Department of Computer, Control and Management, Engenerring Antonio Ruberti, Center for the Evaluation of Language and Communication Technologies (CELCT), Via alla Casata 56/c, Povo, TN, Italy
,
Djoerd Hiemstra
Department of Computer Science, Database Group, University of Twente, Via alla Casata 56/c, Enschede, TN, The Netherlands
,
Anselmo Peñas
Department of Computer Science, Database Group, UNED Natural Language Processing and Information Retrieval Research Group, E.T.S.I. Informática de la UNED, c/ Juan del Rosal 16, Madrid, TN, Spain
,
Giuseppe Santucci
Department of Computer, Control and Management, Engeneering Antonio Ruberti, Sapienza University of Rome, Via Ariosto 25, Rome, TN, Italy

Sponsors

PROMISE: PROMISE Network of Excellence
Sapienza: Sapienza University of Rome
ESF: European Science Foundation
ELIAS Research Network Programme: ELIAS Research Network Programme

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 17 September 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gäde MKoolen MHall MBogers TPetras VScholer FThomas PElsweiler DJoho HKando NSmith C(2021)A Manifesto on Resource Re-Use in Interactive Information RetrievalProceedings of the 2021 Conference on Human Information Interaction and Retrieval10.1145/3406522.3446056(141-149)Online publication date: 14-Mar-2021
https://dl.acm.org/doi/10.1145/3406522.3446056
Bogers TGäde MMichael MFreund LKoolen MPetras VSkov M(2018)Report on the Workshop on Barriers to Interactive IR Resources Re-use (BIIRRR 2018)ACM SIGIR Forum10.1145/3274784.327479552:1(119-128)Online publication date: 31-Aug-2018
https://dl.acm.org/doi/10.1145/3274784.3274795
Ferro N(2014)CLEF 15th BirthdayACM SIGIR Forum10.1145/2701583.270158748:2(31-55)Online publication date: 23-Dec-2014
https://dl.acm.org/doi/10.1145/2701583.2701587

View Options

View options

Media

Figures

Other

Tables

View Table of Contents