Deep learning for detecting pulmonary tuberculosis via chest radiography: an international study across 10 countries
Authors:
Sahar Kazemzadeh,
Jin Yu,
Shahar Jamshy,
Rory Pilgrim,
Zaid Nabulsi,
Christina Chen,
Neeral Beladia,
Charles Lau,
Scott Mayer McKinney,
Thad Hughes,
Atilla Kiraly,
Sreenivasa Raju Kalidindi,
Monde Muyoyeta,
Jameson Malemela,
Ting Shih,
Greg S. Corrado,
Lily Peng,
Katherine Chou,
Po-Hsuan Cameron Chen,
Yun Liu,
Krish Eswaran,
Daniel Tse,
Shravya Shetty,
Shruthi Prabhakara
Abstract:
Tuberculosis (TB) is a top-10 cause of death worldwide. Though the WHO recommends chest radiographs (CXRs) for TB screening, the limited availability of CXR interpretation is a barrier. We trained a deep learning system (DLS) to detect active pulmonary TB using CXRs from 9 countries across Africa, Asia, and Europe, and utilized large-scale CXR pretraining, attention pooling, and noisy student semi…
▽ More
Tuberculosis (TB) is a top-10 cause of death worldwide. Though the WHO recommends chest radiographs (CXRs) for TB screening, the limited availability of CXR interpretation is a barrier. We trained a deep learning system (DLS) to detect active pulmonary TB using CXRs from 9 countries across Africa, Asia, and Europe, and utilized large-scale CXR pretraining, attention pooling, and noisy student semi-supervised learning. Evaluation was on (1) a combined test set spanning China, India, US, and Zambia, and (2) an independent mining population in South Africa. Given WHO targets of 90% sensitivity and 70% specificity, the DLS's operating point was prespecified to favor sensitivity over specificity. On the combined test set, the DLS's ROC curve was above all 9 India-based radiologists, with an AUC of 0.90 (95%CI 0.87-0.92). The DLS's sensitivity (88%) was higher than the India-based radiologists (75% mean sensitivity), p<0.001 for superiority; and its specificity (79%) was non-inferior to the radiologists (84% mean specificity), p=0.004. Similar trends were observed within HIV positive and sputum smear positive sub-groups, and in the South Africa test set. We found that 5 US-based radiologists (where TB isn't endemic) were more sensitive and less specific than the India-based radiologists (where TB is endemic). The DLS also remained non-inferior to the US-based radiologists. In simulations, using the DLS as a prioritization tool for confirmatory testing reduced the cost per positive case detected by 40-80% compared to using confirmatory testing alone. To conclude, our DLS generalized to 5 countries, and merits prospective evaluation to assist cost-effective screening efforts in radiologist-limited settings. Operating point flexibility may permit customization of the DLS to account for site-specific factors such as TB prevalence, demographics, clinical resources, and customary practice patterns.
△ Less
Submitted 29 October, 2021; v1 submitted 16 May, 2021;
originally announced May 2021.
Deep Learning for Distinguishing Normal versus Abnormal Chest Radiographs and Generalization to Unseen Diseases
Authors:
Zaid Nabulsi,
Andrew Sellergren,
Shahar Jamshy,
Charles Lau,
Edward Santos,
Atilla P. Kiraly,
Wenxing Ye,
Jie Yang,
Rory Pilgrim,
Sahar Kazemzadeh,
Jin Yu,
Sreenivasa Raju Kalidindi,
Mozziyar Etemadi,
Florencia Garcia-Vicente,
David Melnick,
Greg S. Corrado,
Lily Peng,
Krish Eswaran,
Daniel Tse,
Neeral Beladia,
Yun Liu,
Po-Hsuan Cameron Chen,
Shravya Shetty
Abstract:
Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to build specific systems to detect every possible conditi…
▽ More
Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to build specific systems to detect every possible condition. In this work, we developed and evaluated an AI system to classify CXRs as normal or abnormal. For development, we used a de-identified dataset of 248,445 patients from a multi-city hospital network in India. To assess generalizability, we evaluated our system using 6 international datasets from India, China, and the United States. Of these datasets, 4 focused on diseases that the AI was not trained to detect: 2 datasets with tuberculosis and 2 datasets with coronavirus disease 2019. Our results suggest that the AI system generalizes to new patient populations and abnormalities. In a simulated workflow where the AI system prioritized abnormal cases, the turnaround time for abnormal cases reduced by 7-28%. These results represent an important step towards evaluating whether AI can be safely used to flag cases in a general setting where previously unseen abnormalities exist.
△ Less
Submitted 29 October, 2021; v1 submitted 21 October, 2020;
originally announced October 2020.
