research-article

Deriving a test collection for clinical information retrieval from systematic reviews

Authors:

Florian Boudin,

Jian-Yun Nie,

Martin DawesAuthors Info & Claims

DTMBIO '10: Proceedings of the ACM fourth international workshop on Data and text mining in biomedical informatics

Pages 57 - 60

https://doi.org/10.1145/1871871.1871882

Published: 26 October 2010 Publication History

Get Access

DTMBIO '10: Proceedings of the ACM fourth international workshop on Data and text mining in biomedical informatics

Deriving a test collection for clinical information retrieval from systematic reviews

Pages 57 - 60

Abstract
References

Abstract

In this paper, we describe the construction of a test collection for evaluating clinical information retrieval. The purpose of this test collection is to provide a basis for researchers to experiment with PECO-structured queries. Systematic reviews are used as a starting point for generating queries and relevance judgments. We give some details on the difficulties encountered in building this resource and report the results achieved by current state-of-the-art approaches.

References

[1]

F. Boudin, J.-Y. Nie, and M. Dawes. Clinical Information Retrieval using Document and PICO Structure. In Proceedings of the NAACL-HLT conference, pages 822--830, Los Angeles, California, June 2010.

Digital Library

Google Scholar

[2]

G. Chung and E. Coiera. Are decision trees a feasible knowledge representation to guide extraction of critical information from randomized controlled trial reports? BMC Medical Informatics and Decision Making, 8(1):48, 2008.

Crossref

Google Scholar

[3]

D. Demner-Fushman and J. Lin. Answering clinical questions with knowledge-based and statistical techniques. Computational Linguistics, 33(1):63--103, 2007.

Digital Library

Google Scholar

[4]

K. Friberg Heppin. MedEval - A Swedish Medical Test Collection with Doctors and Patients User Groups. In Proceedings of the Second Louhi Workshop on Text and Data Mining of Health Documents, pages 1--7, Los Angeles, California, USA, June 2010.

Digital Library

Google Scholar

[5]

S. Green, J. P. T. Higgins, P. Alderson, M. Clarke, C. D. Mulrow, and A. D. Oxman. Cochrane Handbook for Systematic Reviews of Interventions, chapter 1. The Cochrane Collaboration, 2008.

Google Scholar

[6]

W. Hersh, C. Buckley, T. Leone, and D. Hickam. OHSUMED: an interactive retrieval evaluation and new large test collection for research. In Proceedings of the SIGIR conference, pages 192--201, 1994.

Digital Library

Google Scholar

[7]

X. Huang, J. Lin, and D. Demner-Fushman. Evaluation of PICO as a Knowledge Representation for Clinical Questions. In AMIA Annual Symposium Proceedings, pages 359--363, 2006.

Google Scholar

[8]

C. D. Manning, P. Raghavan, and H. Schütze. Introduction to Information Retrieval. Cambridge University Press, 2008.

Digital Library

Google Scholar

[9]

D. Metzler and W. B. Croft. A Markov random field model for term dependencies. In Proceedings of the SIGIR conference, pages 472--479, 2005.

Digital Library

Google Scholar

[10]

W. Richardson, M. Wilson, J. Nishikawa, and R. S. Hayward. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club, 123(3):A12, 1995.

Crossref

Google Scholar

[11]

T. Strohman, D. Metzler, H. Turtle, and W. B. Croft. Indri: A language model-based search engine for complex queries. In Proceedings of the International Conference on Intelligence Analysis, 2005.

Google Scholar

[12]

E. Voorhees. TREC: Continuing information retrieval's tradition of experimentation. Communications of the ACM, 50(11):51--54, 2007.

Digital Library

Google Scholar

Cited By

View all

Bouamrane MMacdonald COunis IMair F(2011)Protocol-driven searches for medical and health-sciences systematic reviewsProceedings of the Third international conference on Advances in information retrieval theory10.5555/2040317.2040342(188-200)Online publication date: 12-Sep-2011
https://dl.acm.org/doi/10.5555/2040317.2040342
Bouamrane MMacdonald COunis IMair F(2011)Protocol-Driven Searches for Medical and Health-Sciences Systematic ReviewsAdvances in Information Retrieval Theory10.1007/978-3-642-23318-0_18(188-200)Online publication date: 2011
https://doi.org/10.1007/978-3-642-23318-0_18

Index Terms

Deriving a test collection for clinical information retrieval from systematic reviews
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing
    2. Retrieval models and ranking

Recommendations

A Test Collection for Evaluating Retrieval of Studies for Inclusion in Systematic Reviews
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

This paper introduces a test collection for evaluating the effectiveness of different methods used to retrieve research studies for inclusion in systematic reviews. Systematic reviews appraise and synthesise studies that meet specific inclusion ...
2AIRTC: The Amharic Adhoc Information Retrieval Test Collection
Experimental IR Meets Multilinguality, Multimodality, and Interaction
Abstract
Evaluation is highly important for designing, developing, and maintaining information retrieval (IR) systems. The IR community has developed shared tasks where evaluation framework, evaluation measures and test collections have been developed for ...
A Test Collection for Ad-hoc Dataset Retrieval
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

This paper introduces a new test collection for ad-hoc dataset retrieval, which have been developed through a shared task called Data Search in the fifteenth NTCIR. This test collection consists of dataset collections derived from the US and Japanese ...

Comments

Information & Contributors

Information

Published In

DTMBIO '10: Proceedings of the ACM fourth international workshop on Data and text mining in biomedical informatics

October 2010

78 pages

ISBN:9781450303828

DOI:10.1145/1871871

General Chairs:
Doheon Kim
KAIST, Korea
,
Hagit Shatkay
Queen's University, Canada
,
Program Chairs:
Min Song
New Jersey Institute of Technology, USA
,
Shamkant Navathe
Georgia Tech, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '10

Sponsor:

CIKM '10: International Conference on Information and Knowledge Management

October 26, 2010

ON, Toronto, Canada

Acceptance Rates

Overall Acceptance Rate 41 of 247 submissions, 17%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
139
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bouamrane MMacdonald COunis IMair F(2011)Protocol-driven searches for medical and health-sciences systematic reviewsProceedings of the Third international conference on Advances in information retrieval theory10.5555/2040317.2040342(188-200)Online publication date: 12-Sep-2011
https://dl.acm.org/doi/10.5555/2040317.2040342
Bouamrane MMacdonald COunis IMair F(2011)Protocol-Driven Searches for Medical and Health-Sciences Systematic ReviewsAdvances in Information Retrieval Theory10.1007/978-3-642-23318-0_18(188-200)Online publication date: 2011
https://doi.org/10.1007/978-3-642-23318-0_18

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

A Test Collection for Evaluating Retrieval of Studies for Inclusion in Systematic Reviews

2AIRTC: The Amharic Adhoc Information Retrieval Test Collection

A Test Collection for Ad-hoc Dataset Retrieval

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations