Deep Learning and Artificial Intelligence in Radiology: Current Applications and Future Directions
Deep Learning and Artificial Intelligence in Radiology: Current Applications and Future Directions
Deep Learning and Artificial Intelligence in Radiology: Current Applications and Future Directions
1 Department of Radiology, The Institute of Medical Science, The University of Tokyo, Tokyo, Japan,
2 Department of Radiology, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan
* koyasaka@gmail.com
a1111111111
a1111111111
a1111111111 Radiological imaging diagnosis plays important roles in clinical patient management. Deep
a1111111111 learning with convolutional neural networks (CNNs) is recently gaining wide attention for its
a1111111111
high performance in recognizing images. If CNNs realize their promise in the context of radi-
ology, they are anticipated to help radiologists achieve diagnostic excellence and to enhance
patient healthcare. Here, we discuss very recent developments in the field, including studies
published in the current PLOS Medicine Special Issue on Machine Learning in Health and Bio-
OPEN ACCESS medicine, with comment on expectations and planning for artificial intelligence (AI) in the
Citation: Yasaka K, Abe O (2018) Deep learning
radiology clinic.
and artificial intelligence in radiology: Current Chest radiographs are one of the most utilized radiological modalities in the world and
applications and future directions. PLoS Med have been collected into a number of large datasets currently available to machine learning
15(11): e1002707. https://doi.org/10.1371/journal. researchers. In this Special Issue, three groups of researchers applied deep learning to radiolog-
pmed.1002707 ical imaging diagnosis using this modality. In the first, Pranav Rajpurkar and colleagues found
Published: November 30, 2018 that deep learning models detected clinically important abnormalities (e.g., edema, fibrosis,
Copyright: © 2018 Yasaka, Abe. This is an open
mass, pneumonia, and pneumothorax) on chest radiography, at a performance level compara-
access article distributed under the terms of the ble to practicing radiologists [1]. In a similar study, Andrew Taylor and colleagues developed
Creative Commons Attribution License, which deep learning models that detected clinically significant pneumothoraces on chest radiography
permits unrestricted use, distribution, and with excellent performance on data from the same site—with areas under the receiver operat-
reproduction in any medium, provided the original ing characteristic curve (AUC) of 0.94–0.96 [2]. Meanwhile, Eric Oermann and colleagues
author and source are credited.
investigated how well deep learning models that detected pneumonia on chest radiography
Funding: This work was supported by the Japan generalized across different hospitals. They found that models trained on pooled data from
Radiological Society. The funder had no role in sites with different pneumonia prevalence performed well on new pooled data from these
writing, decision to publish, or preparation of the
same sites (AUC of 0.93–0.94) but significantly less well on external data (AUC 0.75–0.89);
manuscript.
additional analyses supported the interpretation that deep learning models diagnosing pneu-
Competing interests: I have read the journal’s monia on chest radiography are able to exploit confounding information that is associated
policy and the authors of this manuscript have the
with pneumonia prevalence [3]. Also in this Special Issue, Nicholas Bien and colleagues
following competing interests: KY receives a
research grant from Japan Radiological Society. applied deep learning techniques to detect knee abnormalities on magnetic resonance (MR)
OA has no competing interest regarding this work. imaging and found that the trained model showed near-human-level performance [4]. Taking
these four studies together, we can interpret that deep learning is currently able to diagnose a
Abbreviations: AI, artificial intelligence; AUC, area
under the receiver operating characteristic curve; number of conditions using radiological data, but such diagnostic models may not be robust
CNN, convolutional neural network; CT, computed to a change in location.
tomography; MR, magnetic resonance; PET, These Special Issue studies join a growing number of applications of deep learning to radio-
positron emission tomography. logical images from various modalities that can aid with detection, diagnosis, staging, and sub-
Provenance: Commissioned; not externally peer classification of conditions. Cerebral aneurysms can be detected on MR angiography with
reviewed.
References
1. Rajpurkar P, Irvin J, Ball RL, Zhu K, Yang B, Mehta H, et al. Deep learning for chest radiograph diagno-
sis: A retrospective comparison of CheXNeXt to practicing radiologists. PLoS Med. 2018;15(11):
e1002686. https://doi.org/10.1371/journal.pmed.1002686
2. Taylor AG, Mielke C, Mongan J. Automated detection of clinically-significant pneumothorax on frontal
chest X-rays using deep convolutional neural networks. PLoS Med. 2018;15(11):e1002697. https://doi.
org/10.1371/journal.pmed.1002697
3. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance
of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS
Med. 2018;15(11):e1002683. https://doi.org/10.1371/journal.pmed.1002683
4. Bien N, Rajpurkar P, Ball RL, Irvin J, Park AK, Jones E, et al. AI-assisted diagnosis for knee MR: Devel-
opment and retrospective validation. PLoS Med. 2018;15(11):e1002699. https://doi.org/10.1371/
journal.pmed.1002699
5. Nakao T, Hanaoka S, Nomura Y, Sato I, Nemoto M, Miki S, et al. Deep neural network-based com-
puter-assisted detection of cerebral aneurysms in MR angiography. J Magn Reson Imaging 2018; 47
(4):948–953. https://doi.org/10.1002/jmri.25842 PMID: 28836310
6. Yasaka K, Akai H, Abe O, Kiryu S. Deep Learning with Convolutional Neural Network for Differentiation
of Liver Masses at Dynamic Contrast-enhanced CT: A Preliminary Study. Radiology 2018; 286(3):887–
896. https://doi.org/10.1148/radiol.2017170706 PMID: 29059036
7. Yasaka K, Akai H, Kunimatsu A, Abe O, Kiryu S. Liver Fibrosis: Deep Convolutional Neural Network for
Staging by Using Gadoxetic Acid-enhanced Hepatobiliary Phase MR Images. Radiology 2018; 287
(1):146–155. https://doi.org/10.1148/radiol.2017171928 PMID: 29239710
8. Chang P, Grinband J, Weinberg BD, Bardis M, Khy M, Cadena G, et al. Deep-Learning Convolutional
Neural Networks Accurately Classify Genetic Mutations in Gliomas. AJNR Am J Neuroradiol 2018; 39
(7):1201–1207. https://doi.org/10.3174/ajnr.A5667 PMID: 29748206
9. Hosny A, Parmar C, Coroller T, Grossmann P, Zeleznik R, Kumar A, et al. Deep learning for lung cancer
prognostication: A retrospective multi-cohort radiomics study. PLoS Med. 2018;15(11):e1002711
https://doi.org/10.1371/journal.pmed.1002711
10. Grainger AT, Tustison NJ, Qing K, Roy R, Berr SS, Shi W. Deep learning-based quantification of
abdominal fat on magnetic resonance images. PLoS ONE 2018; 13(9):e0204071. https://doi.org/10.
1371/journal.pone.0204071 PMID: 30235253
11. Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep Learning MR Imaging-based Attenuation
Correction for PET/MR Imaging. Radiology 2018; 286(2):676–684. https://doi.org/10.1148/radiol.
2017170700 PMID: 28925823
12. Prevedello LM, Erdal BS, Ryu JL, Little KJ, Demirer M, Qian S, et al. Automated Critical Test Findings
Identification and Online Notification System Using Artificial Intelligence in Imaging. Radiology 2017;
285(3):923–931. https://doi.org/10.1148/radiol.2017162664 PMID: 28678669
13. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual explanations
from deep networks via gradient-based localization. arXiv: 1610.02391 [Preprint]. 2016 Oct 7 [cited
2018 Oct 10]. https://arxiv.org/abs/1610.02391.