short-paper

APS: An Active PubMed Search System for Technology Assisted Reviews

Authors:

SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2137 - 2140

https://doi.org/10.1145/3397271.3401401

Published: 25 July 2020 Publication History

Get Access

Abstract

Systematic reviews constitute the cornerstone of Evidence-based Medicine. They can provide guidance to medical policy-making by synthesizing all available studies regarding a certain topic. However, conducting systematic reviews has become a laborious and time-consuming task due to the large amount and rapid growth of published literature. The TAR approaches aim to accelerate the screening stage of systematic reviews by combining machine learning algorithms and human relevance feedback. In this work, we built an online active search system for systematic reviews, named APS, by applying an state-of-the-art TAR approach -- Continuous Active Learning. The system is built on the top of the PubMed collection, which is a widely used database of biomedical literature. It allows users to conduct the abstract screening for systematic reviews. We demonstrate the effectiveness and robustness of the APS in detecting relevant literature and reducing workload for systematic reviews using the CLEF TAR 2017 benchmark.

Supplementary Material

MP4 File (3397271.3401401.mp4)

Systematic reviews constitute the cornerstone of Evidence-based Medicine. They can provide guidance to medical policy-making by synthesizing all available studies regarding a certain topic. However, conducting systematic reviews has become a laborious and time-consuming task due to the large amount and rapid growth of published literature. The TAR approaches aim to accelerate the screening stage of systematic reviews by combining machine learn- ing algorithms and human relevance feedback. In this work, we built an online active search system for systematic reviews, named APS, by applying an state-of-the-art TAR approach Continuous Active Learning. The system is built on the top of the PubMed collection, which is a widely used database of biomedical literature. It allows users to conduct the abstract screening for systematic reviews. We demonstrate the effectiveness and robustness of the APS in detecting relevant literature and reducing workload for systematic reviews using the CLEF TAR 2017 benchmark.

Download
9.42 MB

References

[1]

Mustafa Abualsaud, Nimesh Ghelani, Haotian Zhang, Mark D. Smucker, Gordon V. Cormack, and Maura R. Grossman. 2018. A System for Efficient High-Recall Retrieval. In SIGIR '18. https://doi.org/10.1145/3209978.3210176

Crossref

Google Scholar

[2]

Elaine Beller, Justin Clark, Guy Tsafnat, Clive Adams, Heinz Diehl, Hans Lund, Mourad Ouzzani, Kristina Thayer, James Thomas, Tari Turner, et al. 2018. Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR). Systematic reviews, Vol. 7, 1 (2018), 77.

Google Scholar

[3]

Gordon V. Cormack and Maura R. Grossman. 2014. Evaluation of machine-learning protocols for technology-assisted review in electronic discovery. In SIGIR '14. https://doi.org/10.1145/2600428.2609601

Crossref

Google Scholar

[4]

Gordon V. Cormack and Maura R. Grossman. 2015. Autonomy and Reliability of Continuous Active Learning for Technology-Assisted Review. CoRR, Vol. abs/1504.06868 (2015). arxiv: 1504.06868

Google Scholar

[5]

Gordon V. Cormack and Maura R. Grossman. 2016. Scalability of Continuous Active Learning for Reliable High-Recall Text Classification. In CIKM '16. https://doi.org/10.1145/2983323.2983776

Crossref

Google Scholar

[6]

Gordon V. Cormack and Maura R. Grossman. 2018. Beyond Pooling. In SIGIR '18. https://doi.org/10.1145/3209978.3210119

Crossref

Google Scholar

[7]

Evangelos Kanoulas, Dan Li, Leif Azzopardi, and René Spijker. 2017. CLEF 2017 Technologically Assisted Reviews in Empirical Medicine Overview. In CLEF '17.

Google Scholar

[8]

Evangelos Kanoulas, Dan Li, Leif Azzopardi, and René Spijker. 2018. CLEF 2018 Technologically Assisted Reviews in Empirical Medicine Overview. In CLEF '18.

Google Scholar

[9]

Evangelos Kanoulas, Dan Li, Leif Azzopardi, and René Spijker. 2019. CLEF 2019 Technology Assisted Reviews in Empirical Medicine Overview. In CLEF '19.

Google Scholar

[10]

Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinform., Vol. 36, 4 (2020), 1234--1240. https://doi.org/10.1093/bioinformatics/btz682

Crossref

Google Scholar

[11]

Jimmy J. Lin, Matt Crane, Andrew Trotman, Jamie Callan, Ishan Chattopadhyaya, John Foley, Grant Ingersoll, Craig MacDonald, and Sebastiano Vigna. 2016. Toward Reproducible Baselines: The Open-Source IR Reproducibility Challenge. In ECIR '16. https://doi.org/10.1007/978--3--319--30671--1_30

Crossref

Google Scholar

[12]

Iain J Marshall and Byron C Wallace. 2019. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Systematic reviews, Vol. 8, 1 (2019), 163.

Google Scholar

[13]

Adam Roegiest, Gordon V. Cormack, Charles L. A. Clarke, and Maura R. Grossman. 2015. TREC 2015 Total Recall Track Overview. In TREC 2015 .

Google Scholar

Cited By

View all

Budau LEnsan F(2024)Fully Automated Scholarly Search for Biomedical Systematic Literature ReviewsIEEE Access10.1109/ACCESS.2024.340552912(83764-83773)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3405529
Santos Áda Silva ECouto LReis GBelo V(2023)The use of artificial intelligence for automating or semi-automating biomedical literature analyses: A scoping reviewJournal of Biomedical Informatics10.1016/j.jbi.2023.104389142(104389)Online publication date: Jun-2023
https://doi.org/10.1016/j.jbi.2023.104389
MacFarlane ARussell-Rose TShokraneh F(2022)Search strategy formulation for systematic reviews: Issues, challenges and opportunitiesIntelligent Systems with Applications10.1016/j.iswa.2022.20009115(200091)Online publication date: Sep-2022
https://doi.org/10.1016/j.iswa.2022.200091
Show More Cited By

Index Terms

APS: An Active PubMed Search System for Technology Assisted Reviews
1. Information systems
  1. Information retrieval
    1. Specialized information retrieval
      1. Structure and multilingual text search
        Chemical and biochemical retrieval
    2. Users and interactive retrieval
      1. Search interfaces

Recommendations

A review of author name disambiguation techniques for the PubMed bibliographic database

Author names in bibliographic databases often suffer from ambiguity owing to the same author appearing under different names and multiple authors possessing similar names. It creates difficulty in associating a scholarly work with the person who wrote it,...
Author Name Disambiguation in PubMed using Ensemble-Based Classification Algorithms
JCDL '20: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020

Author name ambiguity is a common problem in digital libraries. The problem occurs because multiple individuals may share the same name and the same individual may be represented by various names. Researchers have proposed various techniques for author ...
A comparison of systematic reviews and guideline-based systematic reviews in medical studies
Abstract
The question of how citation impact relates to academic quality accompanies every decade in bibliometric research. Although experts have employed more complex conceptions of research quality for responsible evaluation, detailed analyses of how ...

Comments

Information & Contributors

Information

Published In

SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2020

2548 pages

ISBN:9781450380164

DOI:10.1145/3397271

General Chairs:
Jimmy Huang
York University, Canada
,
Yi Chang
Jilin University, China
,
Xueqi Cheng
Chinese Academy of Sciences, China
,
Program Chairs:
Jaap Kamps
University of Amsterdam, Netherlands
,
Vanessa Murdock
Amazon, U.S.A.
,
Ji-Rong Wen
Renmin University of China, China
,
Yiqun Liu
Tsinghua University, China

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

the NWO Innovational Research Incentives Scheme Vidi
the H2020-EU.3.4. - SOCIETAL CHALLENGES - Smart Green And Integrated Transport
the NWO Smart Culture - Big Data / Digital Humanities

Conference

SIGIR '20

Sponsor:

SIGIR

SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval

July 25 - 30, 2020

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
197
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Budau LEnsan F(2024)Fully Automated Scholarly Search for Biomedical Systematic Literature ReviewsIEEE Access10.1109/ACCESS.2024.340552912(83764-83773)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3405529
Santos Áda Silva ECouto LReis GBelo V(2023)The use of artificial intelligence for automating or semi-automating biomedical literature analyses: A scoping reviewJournal of Biomedical Informatics10.1016/j.jbi.2023.104389142(104389)Online publication date: Jun-2023
https://doi.org/10.1016/j.jbi.2023.104389
MacFarlane ARussell-Rose TShokraneh F(2022)Search strategy formulation for systematic reviews: Issues, challenges and opportunitiesIntelligent Systems with Applications10.1016/j.iswa.2022.20009115(200091)Online publication date: Sep-2022
https://doi.org/10.1016/j.iswa.2022.200091
Rybinski MKarimi SKhoo ADiaz FShah CSuel TCastells PJones RSakai T(2021)Science2Cure: A Clinical Trial Search PrototypeProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3462797(2620-2624)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3462797

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A review of author name disambiguation techniques for the PubMed bibliographic database

Author Name Disambiguation in PubMed using Ensemble-Based Classification Algorithms

A comparison of systematic reviews and guideline-based systematic reviews in medical studies