Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

OPINION

Artificial intelligence in radiology

Nature Reviews Cancer volume 18, pages 500–510 (2018)Cite this article

Subjects

Abstract

Artificial intelligence (AI) algorithms, particularly deep learning, have demonstrated remarkable progress in image-recognition tasks. Methods ranging from convolutional neural networks to variational autoencoders have found myriad applications in the medical image analysis field, propelling it forward at a rapid pace. Historically, in radiology practice, trained physicians visually assessed medical images for the detection, characterization and monitoring of diseases. AI methods excel at automatically recognizing complex patterns in imaging data and providing quantitative, rather than qualitative, assessments of radiographic characteristics. In this Opinion article, we establish a general understanding of AI methods, particularly those pertaining to image-based tasks. We explore how these methods could impact multiple facets of radiology, with a general focus on applications in oncology, and demonstrate ways in which these methods are advancing the field. Finally, we discuss the challenges facing clinical implementation and provide our perspective on how the domain could be advanced.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Artificial versus human intelligence.
Fig. 2: Artificial intelligence methods in medical imaging.
Fig. 3: Artificial intelligence impact areas within oncology imaging.

Similar content being viewed by others

References

  1. Editors, N. Auspicious machine learning. Nat. Biomed. Engineer. 1, 0036 (2017).

    Article  Google Scholar 

  2. Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015).

    Article  PubMed  CAS  Google Scholar 

  3. Moravcík, M. et al. DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science 356, 508–513 (2017).

    Article  PubMed  CAS  Google Scholar 

  4. Xiong, W. et al. Toward human parity in conversational speech recognition. IEEE/ACM Trans. Audio Speech Language Process. 25, 2410–2423 (2017).

    Google Scholar 

  5. Pendleton, S. D. et al. Perception, planning, control, and coordination for autonomous vehicles. Machines 5, 6 (2017).

    Article  Google Scholar 

  6. Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).

    Article  PubMed  CAS  Google Scholar 

  7. Grace, K., Salvatier, J., Dafoe, A., Zhang, B. & Evans, O. When will AI exceed human performance? Evidence from AI experts. Preprint at arXiv, 1705.08807 (2017).

  8. Rusk, N. Deep learning. Nat. Methods 13, 35–35 (2015).

    Article  CAS  Google Scholar 

  9. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  PubMed  CAS  Google Scholar 

  10. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

    Article  PubMed  Google Scholar 

  11. Aerts, H. J. W. L. The potential of radiomic-based phenotyping in precision medicine: a review. JAMA Oncol. 2, 1636–1642 (2016).

    Article  PubMed  Google Scholar 

  12. Kumar, V. et al. Radiomics: the process and the challenges. Magn. Reson. Imag. 30, 1234–1248 (2012).

    Article  Google Scholar 

  13. Lambin, P. et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur. J. Cancer 48, 441–446 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017).

    Article  PubMed  Google Scholar 

  15. Kolossváry, M., Kellermayer, M., Merkely, B. & Maurovich-Horvat, P. Cardiac computed tomography radiomics: a comprehensive review on radiomic techniques. J. Thorac. Imag. 33, 26–34 (2018).

    Article  Google Scholar 

  16. Aerts, H. J. W. L. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat. Commun. 5, 4006 (2014).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Coroller, T. P. et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother. Oncol. 114, 345–350 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  18. Wu, W. et al. Exploratory study to identify radiomics classifiers for lung cancer histology. Front. Oncol. 6, 71 (2016).

    PubMed  PubMed Central  Google Scholar 

  19. Huynh, E. et al. Associations of radiomic data extracted from static and respiratory-gated CT scans with disease recurrence in lung cancer patients treated with SBRT. PLoS ONE 12, e0169172 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  20. Rios Velazquez, E. et al. Somatic mutations drive distinct imaging phenotypes in lung cancer. Cancer Res. 77, 3922–3930 (2017).

    Article  PubMed  CAS  Google Scholar 

  21. Grossmann, P. et al. Defining the biological basis of radiomic phenotypes in lung cancer. eLife 6, e23421 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  22. Parmar, C., Grossmann, P., Bussink, J., Lambin, P. & Aerts, H. J. W. L. Machine learning methods for quantitative radiomic biomarkers. Sci. Rep. 5, 13087 (2015).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  23. O’Connor, J. P. B. et al. Imaging biomarker roadmap for cancer studies. Nat. Rev. Clin. Oncol. 14, 169–186 (2017).

    Article  PubMed  CAS  Google Scholar 

  24. Boland, G. W. L., Guimaraes, A. S. & Mueller, P. R. The radiologist’s conundrum: benefits and costs of increasing CT capacity and utilization. Eur. Radiol. 19, 9–12 (2009).

    Article  PubMed  Google Scholar 

  25. McDonald, R. J. et al. The effects of changes in utilization and technological advancements of cross-sectional imaging on radiologist workload. Acad. Radiol. 22, 1191–1198 (2015).

    Article  PubMed  Google Scholar 

  26. Fitzgerald, R. Error in radiology. Clin. Radiol. 56, 938–946 (2001).

    Article  PubMed  CAS  Google Scholar 

  27. Ledley, R. S. & Lusted, L. B. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science 130, 9–21 (1959).

    Article  PubMed  CAS  Google Scholar 

  28. Lodwick, G. S., Keats, T. E. & Dorst, J. P. The coding of Roentgen images for computer analysis as applied to lung cancer. Radiology 81, 185–200 (1963).

    Article  PubMed  CAS  Google Scholar 

  29. Ambinder, E. P. A history of the shift toward full computerization of medicine. J. Oncol. Pract. 1, 54–56 (2005).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Haug, P. J. Uses of diagnostic expert systems in clinical care. Proc. Annu. Symp. Comput. Appl. Med. Care, 379–383 (1993).

  31. Castellino, R. A. Computer aided detection (CAD): an overview. Cancer Imag. 5, 17–19 (2005).

    Article  Google Scholar 

  32. Shen, D., Wu, G. & Suk, H.-I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  33. Veeraraghavan, H. MO-A-207B-01: Radiomics: Segmentation & feature extraction techniques. Med. Phys. 43, 3694–3694 (2016).

    Google Scholar 

  34. Paul, R. et al. Deep feature transfer learning in combination with traditional features predicts survival among patients with lung adenocarcinoma. Tomography 2, 388–395 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Cheng, J.-Z. et al. Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci. Rep. 6, 24454 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  36. Chen, H., Zheng, Y., Park, J.-H., Heng, P.-A. & Zhou, S. K. in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2016 487–495 (Athens, Greece, 2016).

  37. Ghafoorian, M. et al. Location sensitive deep convolutional neural networks for segmentation of white matter hyperintensities. Sci. Rep. 7, 5110 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  38. Wang, H. et al. Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18F-FDG PET/CT images. EJNMMI Res. 7, 11 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  39. van Ginneken, B., Schaefer-Prokop, C. M. & Prokop, M. Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology 261, 719–732 (2011).

    Article  PubMed  Google Scholar 

  40. Nagaraj, S., Rao, G. N. & Koteswararao, K. The role of pattern recognition in computer-aided diagnosis and computer-aided detection in medical imaging: a clinical validation. Int. J. Comput. Appl. 8, 18–22 (2010).

    Google Scholar 

  41. Cole, E. B. et al. Impact of computer-aided detection systems on radiologist accuracy with digital mammography. AJR Am. J. Roentgenol. 203, 909–916 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Lehman, C. D. et al. Diagnostic accuracy of digital screening mammography with and without computer-aided detection. JAMA Intern. Med. 175, 1828–1837 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  43. Huang, X., Shan, J. & Vaidya, V. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 379–383 (Melbourne, Australia, 2017).

  44. Tsehay, Y. K. et al. in Proceedings of SPIE https://doi.org/10.1117/12.2254423 (2017).

  45. Kooi, T. et al. Large scale deep learning for computer aided detection of mammographic lesions. Med. Image Anal. 35, 303–312 (2017).

    Article  PubMed  Google Scholar 

  46. Sharma, N. & Aggarwal, L. M. Automated medical image segmentation techniques. J. Med. Phys. 35, 3–14 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Haralick, R. M. & Shapiro, L. G. Image segmentation techniques. Computer Vision Graph. Image Process. 29, 100–132 (1985).

    Article  Google Scholar 

  48. Pham, D. L., Xu, C. & Prince, J. L. Current methods in medical image segmentation. Annu. Rev. Biomed. Eng. 2, 315–337 (2000).

    Article  PubMed  CAS  Google Scholar 

  49. Grau, V., Mewes, A. U. J., Alcañiz, M., Kikinis, R. & Warfield, S. K. Improved watershed transform for medical image segmentation using prior information. IEEE Trans. Med. Imag. 23, 447–458 (2004).

    Article  CAS  Google Scholar 

  50. Parisot, S. et al. A probabilistic atlas of diffuse WHO grade II glioma locations in the brain. PLoS ONE 11, e0144200 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  51. Ghose, S. et al. in 2012 19th IEEE International Conference on Image Processing 541–544 (Orlando, FL, USA, 2012).

  52. Han, X. et al. Atlas-based auto-segmentation of head and neck CT images. Med. Image Comput. Comput. Assist. Interv. 11, 434–441 (2008).

    PubMed  Google Scholar 

  53. Long, J., Shelhamer, E. & Darrell, T. in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3431–3440 (Boston, MA, USA, 2015).

  54. Ronneberger, O., Fischer, P. & Brox, T. U. in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015 234–241 (Munich, Germany, 2015).

  55. Moeskops, P. et al. in Medical Image Computing and Computer-Assisted Intervention — MICCAI 2016 478–486 (Athens, Greece, 2016).

  56. de Brebisson, A. & Montana, G. in 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 20–28 (Boston, MA, USA, 2015).

  57. Cioffi, U., Raveglia, F., De Simone, M. & Baisi, A. Ground-glass opacities: a curable disease but a big challenge for surgeons. J. Thorac. Cardiovasc. Surg. 154, 375–376 (2017).

    Article  PubMed  Google Scholar 

  58. Champaign, J. L. & Cederbom, G. J. Advances in breast cancer detection with screening mammography. Ochsner J. 2, 33–35 (2000).

    PubMed  PubMed Central  CAS  Google Scholar 

  59. Shiraishi, J., Li, Q., Appelbaum, D. & Doi, K. Computer-aided diagnosis and artificial intelligence in clinical imaging. Semin. Nucl. Med. 41, 449–462 (2011).

    Article  PubMed  Google Scholar 

  60. Ayer, T., Ayvaci, M. U., Liu, Z. X., Alagoz, O. & Burnside, E. S. Computer-aided diagnostic models in breast cancer screening. Imag. Med. 2, 313–323 (2010).

    Article  Google Scholar 

  61. Zhang, J., Wang, Y., Yu, B., Shi, X. & Zhang, Y. Application of computer-aided diagnosis to the sonographic evaluation of cervical lymph nodes. Ultrason. Imag. 38, 159–171 (2016).

    Article  Google Scholar 

  62. Giannini, V. et al. A fully automatic computer aided diagnosis system for peripheral zone prostate cancer detection using multi-parametric magnetic resonance imaging. Comput. Med. Imaging Graph. 46, 219–226 (2015).

    Article  PubMed  Google Scholar 

  63. El-Baz, A. et al. Computer-aided diagnosis systems for lung cancer: challenges and methodologies. Int. J. Biomed. Imag. 2013, 942353 (2013).

    Google Scholar 

  64. Edey, A. J. & Hansell, D. M. Incidentally detected small pulmonary nodules on CT. Clin. Radiol. 64, 872–884 (2009).

    Article  PubMed  CAS  Google Scholar 

  65. Mirsadraee, S., Oswal, D., Alizadeh, Y., Caulo, A. & van Beek, E. Jr. The7th lung cancer TNM classification and staging system: review of the changes and implications. World J. Radiol. 4, 128–134 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  66. Sohn, K., Shang, W. & Lee, H. in Advances in Neural Information Processing Systems 27 (NIPS 2014) (eds Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q.) 2141–2149 (Montreal, Canada, 2014).

  67. Litjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  68. Cruz-Roa, A. et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent. Sci. Rep. 7, 46450 (2017).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  69. Jaffe, C. C. Measures of response: RECIST, WHO, and new alternatives. J. Clin. Oncol. 24, 3245–3251 (2006).

    Article  PubMed  Google Scholar 

  70. Thiesse, P. et al. Response rate accuracy in oncology trials: reasons for interobserver variability. Groupe Français d’Immunothérapie of the Fédération Nationale des Centres de Lutte Contre le Cancer. J. Clin. Oncol. 15, 3507–3514 (1997).

    Article  PubMed  CAS  Google Scholar 

  71. Khorasani, R., Erickson, B. J. & Patriarche, J. New opportunities in computer-aided diagnosis: change detection and characterization. J. Am. Coll. Radiol. 3, 468–469 (2006).

    Article  PubMed  Google Scholar 

  72. Patriarche, J. W. & Erickson, B. J. Part 1. Automated change detection and characterization in serial MR studies of brain-tumor patients. J. Digit. Imag. 20, 203–222 (2007).

    Article  Google Scholar 

  73. Pan, X., Sidky, E. Y. & Vannier, M. Why do commercial CT scanners still employ traditional, filtered back-projection for image reconstruction? Inverse Probl. 25, 1230009 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Pipatsrisawat, T., Gacic, A., Franchetti, F., Puschel, M. & Moura, J. M. F. in Proceedings. (ICASSP ‘05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005 v/153–v/156 (Philadelphia, PA, USA, 2005).

  75. Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R. & Rosen, M. S. Image reconstruction by domain-transform manifold learning. Nature 555, 487–492 (2018).

    Article  PubMed  CAS  Google Scholar 

  76. Hammernik, K., Würfl, T., Pock, T. & Maier, A. A. in Bildverarbeitung für die Medizin 2017 (eds Maier-Hein, K., Deserno, T., Handels, H. & Tolxdorff, T.) 92–97 (Springer, Berlin, Heidelberg, 2017).

  77. Gjesteby, L. et al. in Developments in X-Ray Tomography XI 10391-31 (San Diego, CA, USA, 2017).

  78. El-Gamal, F. E.-Z. A., Elmogy, M. & Atwan, A. Current trends in medical image registration and fusion. Egypt. Informat. J. 17, 99–124 (2016).

    Article  Google Scholar 

  79. Yang, X., Kwitt, R., Styner, M. & Niethammer, M. Quicksilver: fast predictive image registration — a deep learning approach. Neuroimage 158, 378–396 (2017).

    Article  PubMed  Google Scholar 

  80. Ngiam, J. et al. in Proceedings of the 28th International Conference on Machine Learning 689–696 (Bellevue, WA, USA, 2011).

  81. Yankeelov, T. E., Abramson, R. G. & Quarles, C. C. Quantitative multimodality imaging in cancer research and therapy. Nat. Rev. Clin. Oncol. 11, 670–680 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  82. Johnson, A. J., Chen, M. Y. M., Zapadka, M. E., Lyders, E. M. & Littenberg, B. Radiology report clarity: a cohort study of structured reporting compared with conventional dictation. J. Am. Coll. Radiol. 7, 501–506 (2010).

    Article  PubMed  Google Scholar 

  83. Levy, M. A. & Rubin, D. L. Tool support to enable evaluation of the clinical response to treatment. AMIA Annu. Symp. Proc. 2008, 399–403 (2008).

    PubMed Central  Google Scholar 

  84. European Society of Radiology (ESR). Good practice for radiological reporting. Guidelines from the European Society of Radiology (ESR). Insights Imag. 2, 93–96 (2011).

    Article  Google Scholar 

  85. Folio, L. R. et al. Quantitative radiology reporting in oncology: survey of oncologists and radiologists. AJR Am. J. Roentgenol. 205, W233–W243 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  86. Karpathy, A. & Fei-Fei, L. Deep visual-semantic alignments for generating image descriptions. IEEE Trans. Pattern Anal. Mach. Intell. 39, 664–676 (2017).

    Article  PubMed  Google Scholar 

  87. Shin, H.-C. et al. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2497–2506 (Las Vegas, NV, USA, 2016).

  88. Lee, J.-G. et al. Deep learning in medical imaging: general overview. Kor. J. Radiol. 18, 570–584 (2017).

    Article  Google Scholar 

  89. OECD. Computed tomography (CT) exams. https://doi.org/10.1787/3c994537-en (2018).

  90. OECD. Magnetic resonance imaging (MRI) exams. https://doi.org/10.1787/1d89353f-en (2018).

  91. Bryan, S. et al. Radiology report times: impact of picture archiving and communication systems. AJR Am. J. Roentgenol. 170, 1153–1159 (1998).

    Article  PubMed  CAS  Google Scholar 

  92. Mansoori, B., Erhard, K. K. & Sunshine, J. L. Picture Archiving and Communication System (PACS) implementation, integration and benefits in an integrated health system. Acad. Radiol. 19, 229–235 (2012).

    Article  PubMed  Google Scholar 

  93. Lemke, H. U. PACS developments in Europe. Comput. Med. Imag. Graph. 27, 111–120 (2003).

    Article  Google Scholar 

  94. Mendel, J. B. & Schweitzer, A. L. PACS for the developing world. J. Global Radiol. 1, 5 (2015).

    Article  Google Scholar 

  95. Goodfellow, I. et al. in Advances in Neural Information Processing Systems 27 (NIPS 2014) (eds Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D. & Weinberger, K. Q.) 2672–2680 (Montreal, Canada, 2014).

  96. Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at arXiv, 1312.6114 (2013).

    Google Scholar 

  97. Kamnitsas, K. et al. in Information Processing in Medical Imaging 597–609 (Springer, Cham, 2017).

  98. Kallenberg, M. et al. Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring. IEEE Trans. Med. Imag. 35, 1322–1331 (2016).

    Article  Google Scholar 

  99. Zhang, P., Wang, F. & Zheng, Y. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 578–582 (Melbourne, Australia, 2017).

  100. Clark, K. et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imag. 26, 1045–1057 (2013).

    Article  Google Scholar 

  101. Wang, G. A. Perspective on deep imaging. IEEE Access 4, 8914–8924 (2016).

    Article  Google Scholar 

  102. Ford, R. A., Price, W. & Nicholson, I. I. Privacy and accountability in black-box medicine. Mich. Telecomm. Tech. L. Rev. 23, 1 (2016).

    Google Scholar 

  103. Selbst, A. D. & Powles, J. Meaningful information and the right to explanation. Int. Data Privacy Law 7, 233–242 (2017).

    Article  Google Scholar 

  104. Imming, P., Sinning, C. & Meyer, A. Drugs, their targets and the nature and number of drug targets. Nat. Rev. Drug Discov. 5, 821–834 (2006).

    Article  PubMed  CAS  Google Scholar 

  105. Mehlhorn, H. et al. in Encyclopedia of Parasitology 3rd edn (ed. Mehlhorn, H.) 400–402 (Springer, Berlin, Heidelberg, 2008).

  106. Shokri, R. & Shmatikov, V. in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security 1310–1321 (Denver, CO, USA, 2015).

  107. Phong, L. T., Aono, Y., Hayashi, T., Wang, L. & Moriai, S. in Applications and Techniques in Information Security. 8th International Conference, ATIS 2017 (eds Batten, L., Kim, D. S., Zhang, X. & Li, G.) 719, 100–110 (Auckland, New Zealand, 2017).

  108. McMahan, H. B., Moore, E., Ramage, D., Hampson, S. & y Arcas, B. A. in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) 1273–1282 (Fort Lauderdale, FL, USA, 2017).

  109. Gilad-Bachrach, R. et al. in Proceedings of the 33rd International Conference on Machine Learning 201–210 (New York, NY, USA, 2016).

  110. Cahan, A. & Cimino, J. J. A. Learning health care system using computer-aided diagnosis. J. Med. Internet Res. 19, e54 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  111. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  PubMed  CAS  Google Scholar 

  112. Miotto, R., Wang, F., Wang, S., Jiang, X. & Dudley, J. T. Deep learning for healthcare: review, opportunities and challenges. Brief. Bioinform. https://doi.org/10.1093/bib/bbx044 (2017).

  113. Kevin Zhou, S., Greenspan, H. & Shen, D. Deep Learning for Medical Image Analysis. (Academic Press, 2017).

  114. Shin, H.-C. et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imag. 35, 1285–1298 (2016).

    Article  Google Scholar 

  115. Shin, Y. & Balasingham, I. in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 3277–3280 (Jeju Island, Korea, 2017).

  116. Orringer, D. A. et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat. Biomed. Eng. 1, 0027 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  117. Albarqouni, S. et al. AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imag. 35, 1313–1321 (2016).

    Article  Google Scholar 

  118. Djuric, U., Zadeh, G., Aldape, K. & Diamandis, P. Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care. Precision Oncol. 1, 22 (2017).

    Article  Google Scholar 

  119. Janowczyk, A. & Madabhushi, A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J. Pathol. Inform. 7, 29 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  120. Bejnordi, B. E. et al. in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) 929–932 (Melbourne, Australia, 2017).

  121. Yuan, Y. et al. DeepGene: an advanced cancer type classifier based on deep learning and somatic point mutations. BMC Bioinform. 17, 476 (2016).

    Article  CAS  Google Scholar 

  122. Alipanahi, B., Delong, A., Weirauch, M. T. & Frey, B. J. Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nat. Biotechnol. 33, 831–838 (2015).

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgements

The authors acknowledge financial support from the US National Institutes of Health (NIH-USA U24CA194354 and NIH-USA U01CA190234).

Author information

Authors and Affiliations

  1. Department of Radiation Oncology, Dana-Farber Cancer Institute, Harvard Medical School, Boston, MA, USA

    Ahmed Hosny, Chintan Parmar & Hugo J. W. L. Aerts

  2. Department of Biostatistics & Computational Biology, Dana-Farber Cancer Institute, Boston, MA, USA

    John Quackenbush

  3. Department of Cancer Biology, Dana-Farber Cancer Institute, Boston, MA, USA

    John Quackenbush

  4. Department of Radiology, Columbia University College of Physicians and Surgeons, New York, NY, USA

    Lawrence H. Schwartz

  5. Department of Radiology, New York Presbyterian Hospital, New York, NY, USA

    Lawrence H. Schwartz

  6. Department of Radiology, Dana-Farber Cancer Institute, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA

    Hugo J. W. L. Aerts

Authors
  1. Ahmed Hosny
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chintan Parmar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John Quackenbush
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lawrence H. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hugo J. W. L. Aerts
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.H., C.P. and H.J.W.L.A. performed the literature survey, curated the content and general direction and wrote the manuscript. J.Q. and L.H.S. provided substantial contributions to discussions of the content. All authors contributed to reviewing and editing the manuscript before submission.

Corresponding author

Correspondence to Hugo J. W. L. Aerts.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary

Area under receiver operating characteristic curve

(AUC). A sensitivity versus specificity metric for measuring the performance of binary classifiers that can be extended to multi-class problems. The area under the curve is equal to the probability that a randomly chosen positive sample ranks above a randomly chosen negative one or is regarded to have a higher probability of being positive.

Artificial intelligence

(AI). A branch of computer science involved with the development of machines that are able to perform cognitive tasks that would normally require human intelligence.

Caption generation

The often automated generation of qualitative text describing an illustration or image and its contents.

Ground-glass opacity

(GGO). A visual feature of some subsolid pulmonary nodules that is characterized by focal areas of slightly increased attenuation on computed tomography. Underlying bronchial structures and vessels are often visually preserved (being even more recognizable owing to increased contrast), thus making the detection and diagnosis of such nodules somewhat challenging.

Health Insurance Portability and Accountability Act

(HIPAA). A US act that sets provisions for protecting and securing sensitive patient medical data.

Image registration

A process that involves aligning medical images either in terms of spatial or temporal characteristics, mostly intramodality and occasionally intermodality.

Imaging modalities

A multitude of imaging methods that are used to non-invasively generate visualizations of the human anatomy. Examples of these include computed tomography (CT), computed tomography angiography (CTA), magnetic resonance imaging (MRI), mammography, ultrasonography (echocardiography) and positron emission tomography (PET).

Initializations

Within optimization problems, constantly adjusted parameters during run time need to be initialized to some value before the start of the process. Good initialization techniques aid models in converging faster and hence speed up the iteration process.

Machine learning

A branch of artificial intelligence and computer science that enables computers to learn without being explicitly programmed.

Multiparametric imaging

Medical imaging in which two or more parameters are used to visualize differences between healthy and diseased tissue. In multiparametric magnetic resonance imaging (MRI), these parameters include T2-weighted MRI, diffusion-weighted MRI and dynamic contrast-enhanced MRI, among others.

Predefined engineered features

A set of context-based human-crafted features designed to represent knowledge regarding a specific data space.

Probabilistic atlas

A single composite image formed by combining and registering pre-segmented images of multiple patients that thus contains knowledge on population variability.

Radiomics

A data-centric field investigating the clinical relevance of radiographic tissue characteristics automatically quantified from medical images.

Report generation

The communication of assessments and findings in both image and text formats among medical professionals.

Segmentation

The partitioning of images to produce boundary delineations of objects of interest. Such a boundary is defined by pixels and voxels (3D pixels) when performed in 2D and 3D, respectively.

Self-supervised learning

A type of supervised learning where labels are determined by the input data as opposed to being explicitly provided.

Supervised learning

A type of machine learning where functions are inferred from labelled training data. Example data pairs consist of the input together with its desired output or label.

Unsupervised learning

A type of machine learning where functions are inferred from training data without corresponding labels.

Wearables

A collective term describing health-monitoring devices, smartwatches and fitness trackers that have recently been integrated into the health-care ecosystem as a means to remotely track vitals and adhere to treatment plans.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hosny, A., Parmar, C., Quackenbush, J. et al. Artificial intelligence in radiology. Nat Rev Cancer 18, 500–510 (2018). https://doi.org/10.1038/s41568-018-0016-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41568-018-0016-5

This article is cited by

Search

Advanced search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer