Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Detection of anaemia from retinal fundus images via deep learning

Nature Biomedical Engineering volume 4, pages 18–27 (2020)Cite this article

Subjects

An Author Correction to this article was published on 12 February 2020

This article has been updated

Abstract

Owing to the invasiveness of diagnostic tests for anaemia and the costs associated with screening for it, the condition is often undetected. Here, we show that anaemia can be detected via machine-learning algorithms trained using retinal fundus images, study participant metadata (including race or ethnicity, age, sex and blood pressure) or the combination of both data types (images and study participant metadata). In a validation dataset of 11,388 study participants from the UK Biobank, the metadata-only, fundus-image-only and combined models predicted haemoglobin concentration (in g dl–1) with mean absolute error values of 0.73 (95% confidence interval: 0.72–0.74), 0.67 (0.66–0.68) and 0.63 (0.62–0.64), respectively, and with areas under the receiver operating characteristic curve (AUC) values of 0.74 (0.71–0.76), 0.87 (0.85–0.89) and 0.88 (0.86–0.89), respectively. For 539 study participants with self-reported diabetes, the combined model predicted haemoglobin concentration with a mean absolute error of 0.73 (0.68–0.78) and anaemia an AUC of 0.89 (0.85–0.93). Automated anaemia screening on the basis of fundus images could particularly aid patients with diabetes undergoing regular retinal imaging and for whom anaemia can increase morbidity and mortality risks.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Bland–Altman plot for predicted and measured Hb.
Fig. 2: Prediction of anaemia classifications.
Fig. 3: Effects of masking parts of the image on the prediction of anaemia and moderate anaemia.
Fig. 4: Effects of removing high-frequency information using Gaussian blur on the prediction of anaemia and moderate anaemia.
Fig. 5: Examples applying different explanation techniques to generate saliency maps highlighting the regions the model focuses on when predicting anaemia.

Similar content being viewed by others

Data availability

The data supporting the findings of this study are available, with restrictions, from the UK Biobank24.

Code availability

The machine-learning models were developed by using standard model libraries and scripts in TensorFlow56. Custom code was specific to our computing infrastructure and mainly used for data input/output and parallelization across computers.

Change history

References

  1. McLean, E., Cogswell, M., Egli, I., Wojdyla, D. & de Benoist, B. Worldwide prevalence of anaemia, WHO Vitamin and Mineral Nutrition Information System, 1993–2005. Public Health Nutr. 12, 444–454 (2008).

    Article  PubMed  Google Scholar 

  2. Stevens, G. A. et al. Global, regional, and national trends in haemoglobin concentration and prevalence of total and severe anaemia in children and pregnant and non-pregnant women for 1995–2011: a systematic analysis of population-representative data. Lancet Glob. Health 1, e16–e25 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  3. Stoltzfus, R. J. Iron-deficiency anemia: reexamining the nature and magnitude of the public health problem. Summary: implications for research and programs. J. Nutr. 131, 697S–701S (2001).

    Article  CAS  PubMed  Google Scholar 

  4. Milman, N. Anemia—still a major health problem in many parts of the world! Ann. Hematol. 90, 369–377 (2011).

    Article  PubMed  Google Scholar 

  5. Smith, R. E. Jr. The clinical and economic burden of anemia. Am. J. Manag. Care 16 (Suppl.), S59–S66 (2010).

    PubMed  Google Scholar 

  6. Shah, N., Osea, E. A. & Martinez, G. J. Accuracy of noninvasive hemoglobin and invasive point-of-care hemoglobin testing compared with a laboratory analyzer. Int. J. Lab. Hematol. 36, 56–61 (2014).

    Article  CAS  PubMed  Google Scholar 

  7. Kalantri, A., Karambelkar, M., Joshi, R., Kalantri, S. & Jajoo, U. Accuracy and reliability of pallor for detecting anaemia: a hospital-based diagnostic accuracy study. PLoS ONE 5, e8545 (2010).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  8. Kasper, D. L. et al. Harrison’s Principles of Internal Medicine (McGraw Hill Professional, 2006).

  9. Mannino, R. G. et al. Smartphone app for non-invasive detection of anemia using only patient-sourced photos. Nat. Commun. 9, 4924 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  10. Barker, S. J. & Badal, J. J. The measurement of dyshemoglobins and total hemoglobin by pulse oximetry. Curr. Opin. Anaesthesiol. 21, 805–810 (2008).

    Article  PubMed  Google Scholar 

  11. Pinto, M. et al. The new noninvasive occlusion spectroscopy hemoglobin measurement method: a reliable and easy anemia screening test for blood donors. Transfusion 53, 766–769 (2013).

    Article  CAS  PubMed  Google Scholar 

  12. Wittenmeier, E. et al. Comparison of the gold standard of hemoglobin measurement with the clinical standard (BGA) and noninvasive hemoglobin measurement (SpHb) in small children: a prospective diagnostic observational study. Paediatr. Anaesth. 25, 1046–1053 (2015).

    Article  PubMed  Google Scholar 

  13. Posey, W. M. C. The ocular manifestations of anemia. JAMA XXIX, 169–171 (1897).

    Article  Google Scholar 

  14. Aisen, M. L., Bacon, B. R., Goodman, A. M. & Chester, E. M. Retinal abnormalities associated with anemia. Arch. Ophthalmol. 101, 1049–1052 (1983).

    Article  CAS  PubMed  Google Scholar 

  15. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  CAS  PubMed  Google Scholar 

  16. Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

    Article  PubMed  Google Scholar 

  17. Krause, J. et al. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology 125, 1264–1272 (2018).

    Article  PubMed  Google Scholar 

  18. Ting, D. S. W. et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 318, 2211–2223 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Liu, S. et al. A deep learning-based algorithm identifies glaucomatous discs using monoscopic fundus photographs. Ophthalmol. Glaucoma 1, 15–22 (2018).

    Article  PubMed  Google Scholar 

  20. Christopher, M. et al. Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Sci. Rep. 8, 16685 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  21. Li, Z. et al. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 125, 1199–1206 (2018).

    Article  PubMed  Google Scholar 

  22. Varadarajan, A. V. et al. Deep learning for predicting refractive error from retinal fundus images. Invest. Ophthalmol. Vis. Sci. 59, 2861–2868 (2018).

    Article  PubMed  Google Scholar 

  23. Poplin, R. et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat. Biomed. Eng. 2, 158–164 (2018).

    Article  PubMed  Google Scholar 

  24. Sudlow, C. et al. UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 12, e1001779 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Szegedy, C., Ioffe, S., Vanhoucke, V. & Alemi, A. A. Inception-v4, inception-ResNet and the impact of residual connections on learning. In Proc. of the 31st AAAI Conference on Artificial Intelligence 4278–4284 (AAAI Press, 2017).

  26. Bland, J. M. & Altman, D. G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1, 307–310 (1986).

    Article  CAS  PubMed  Google Scholar 

  27. Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization.International Journal of Computer Vision https://doi.org/10.1007/s11263-019-01228-7 (2019).

  28. Sundararajan, M., Taly, A. & Yan, Q. Axiomatic attribution for deep networks. In Proc. of the 34th International Conference on Machine Learning 3319–3328 (Microtome Publishing, 2017).

  29. Smilkov, D., Thorat, N., Kim, B., Viégas, F. & Wattenberg, M. SmoothGrad: removing noise by adding noise. Preprint at https://arxiv.org/abs/1706.03825 (2017).

  30. Springenberg, J. T., Dosovitskiy, A., Brox, T. & Riedmiller, M. Striving for simplicity: the all convolutional net. Preprint at https://arxiv.org/abs/1412.6806 (2014).

  31. Bland, J. M. & Altman, D. G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 8, 135–160 (1999).

    Article  CAS  PubMed  Google Scholar 

  32. Barker, S. J., Shander, A. & Ramsay, M. A. Continuous noninvasive hemoglobin monitoring: a measured response to a critical review. Anesth. Analg. 122, 565–572 (2016).

    Article  PubMed  Google Scholar 

  33. Elliott, P. & Peakman, T. C. The UK Biobank sample handling and storage protocol for the collection, processing and archiving of human blood and urine. Int. J. Epidemiol. 37, 234–244 (2008).

    Article  PubMed  Google Scholar 

  34. Gehring, H. et al. Accuracy of point-of-care-testing (POCT) for determining hemoglobin concentrations. Acta Anaesthesiol. Scand. 46, 980–986 (2002).

    Article  CAS  PubMed  Google Scholar 

  35. Hiscock, R., Kumar, D. & Simmons, S. W. Systematic review and meta-analysis of method comparison studies of Masimo pulse co-oximeters (Radical-7TM or Pronto-7TM) and HemoCue® absorption spectrometers (B-Hemoglobin or 201+) with laboratory haemoglobin estimation. Anaesth. Intensive Care 43, 341–350 (2015).

    Article  CAS  PubMed  Google Scholar 

  36. Kim, S.-H. et al. Accuracy of continuous noninvasive hemoglobin monitoring: a systematic review and meta-analysis. Anesth. Analg. 119, 332–346 (2014).

    Article  CAS  PubMed  Google Scholar 

  37. Tsan, G. L. et al. Assessment of diabetic teleretinal imaging program at the Portland Department of Veterans Affairs Medical Center. J. Rehabil. Res. Dev. 52, 193–200 (2015).

    Article  PubMed  Google Scholar 

  38. Conlin, P. R. et al. Nonmydriatic teleretinal imaging improves adherence to annual eye examinations in patients with diabetes. J. Rehabil. Res. Dev. 43, 733–740 (2006).

    Article  PubMed  Google Scholar 

  39. Garg, S., Jani, P. D., Kshirsagar, A. V., King, B. & Chaum, E. Telemedicine and retinal imaging for improving diabetic retinopathy evaluation. Arch. Intern. Med. 172, 1677–1678 (2012).

    Article  PubMed  Google Scholar 

  40. Jones, S. C. et al. Prevalence and nature of anaemia in a prospective, population-based sample of people with diabetes: Teesside anaemia in diabetes (TAD) study. Diabet. Med. 27, 655–659 (2010).

    Article  CAS  PubMed  Google Scholar 

  41. Thomas, M. C., MacIsaac, R. J., Tsalamandris, C., Power, D. & Jerums, G. Unrecognized anemia in patients with diabetes: a cross-sectional survey. Diabetes Care 26, 1164–1169 (2003).

    Article  PubMed  Google Scholar 

  42. Wright, J. A., Oddy, M. J. & Richards, T. Presence and characterisation of anaemia in diabetic foot ulceration. Anemia 2014, 104214 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. AlDallal, S. M. & Jena, N. Prevalence of anemia in type 2 diabetic patients. J. Hematol. 7, 57–61 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Mehdi, U. & Toto, R. D. Anemia, diabetes, and chronic kidney disease. Diabetes Care 32, 1320–1326 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  45. Kliger, A. S. et al. KDOQI US commentary on the 2012 KDIGO clinical practice guideline for anemia in CKD. Am. J. Kidney Dis. 62, 849–859 (2013).

    Article  PubMed  Google Scholar 

  46. Davis, M. D. et al. Risk factors for high-risk proliferative diabetic retinopathy and severe visual loss: early treatment diabetic retinopathy study report #18. Invest. Ophthalmol. Vis. Sci. 39, 233–252 (1998).

    CAS  PubMed  Google Scholar 

  47. Taylor-Phillips, S. et al. Extending the diabetic retinopathy screening interval beyond 1 year: systematic review. Br. J. Ophthalmol. 100, 105–114 (2016).

    Article  PubMed  Google Scholar 

  48. Owsley, C. et al. Diabetes eye screening in urban settings serving minority populations: detection of diabetic retinopathy and other ocular findings using telemedicine. JAMA Ophthalmol. 133, 174–181 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Scanlon, P. H. The English national screening programme for diabetic retinopathy 2003–2016. Acta Diabetol. 54, 515–525 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  50. Das, T. & Pappuru, R. R. Telemedicine in diabetic retinopathy: access to rural India. Indian J. Ophthalmol. 64, 84–86 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  51. American Diabetes Association. Standards of medical care in diabetes—2018 Abridged for primary care providers. Clin. Diabetes 36, 14–37 (2018).

  52. Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N. & Folk, J. C. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit. Med. 1, 39 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Tran, K., Mendel, T. A., Holbrook, K. L. & Yates, P. A. Construction of an inexpensive, hand-held fundus camera through modification of a consumer ‘point-and-shoot’ camera. Invest. Opthalmol. Vis. Sci. 53, 7600–7607 (2012).

    Article  Google Scholar 

  54. Firat, P. G., Demirel, E. E., Dikci, S., Kuku, I. & Genc, O. Evaluation of iron deficiency anemia frequency as a risk factor in glaucoma. Anemia 2018, 1456323 (2018).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  55. WHO. Haemoglobin concentrations for the diagnosis of anaemia and assessment of severity. WHO https://www.who.int/vmnis/indicators/haemoglobin.pdf (2011).

  56. Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In Proc. of the 12th USENIX Symposium on Operating Systems Design and Implementation 265–283 (USENIX Association, 2016).

  57. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Proc. of the 25th Conference on Advances in Neural Information Processing Systems 1097–1105 (Curran Associates, 2012).

  58. Sutskever, I., Martens, J., Dahl, G. & Hinton, G. On the importance of initialization and momentum in deep learning. In Proc. of the 30th International Conference on Machine Learning 1139–1147 (Microtome Publishing, 2013).

  59. Priya, G. et al. Accurate, Large Minibatch SGD: training ImageNet in 1 hour. Preprint at https://arxiv.org/abs/1706.02677 (2017).

  60. Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In Proc. of the 44th Annual International Symposium on Computer Architecture 1–12 (ACM New York, 2017).

  61. Caruana, R., Lawrence, S. & Giles, L. Overftting in neural nets: backpropagation, conjugate gradient, and early stopping. In Proc. of the 13th Conference on Advances in Neural Information Processing Systems 381–387 (MIT Press, 2001).

  62. Opitz, D. & Maclin, R. Popular ensemble methods: an empirical study. J. Artif. Intell. Res. 11, 169–198 (1999).

    Article  Google Scholar 

Download references

Acknowledgements

This research was conducted using the UK Biobank Resource under application number 17643. The work of A.H. was done via Advanced Clinical, San Francisco, USA. The authors thank C. Angermueller from Google Research for his engineering contributions and A. Zaidi, A. Narayanaswamy, C. Chen, J. Krause and R. Sayres from Google Research for their advice and assistance with reviewing the manuscript.

Author information

Author notes

  1. These authors contributed equally: A. V. Varadarajan and Y. Liu.

Authors and Affiliations

  1. Google Health, Google, Mountain View, CA, USA

    Akinori Mitani, Abigail Huang, Greg S. Corrado, Lily Peng, Dale R. Webster, Naama Hammel, Yun Liu & Avinash V. Varadarajan

  2. Google Research, Google, Mountain View, CA, USA

    Subhashini Venugopalan

Authors
  1. Akinori Mitani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abigail Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Subhashini Venugopalan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Greg S. Corrado
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lily Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dale R. Webster
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Naama Hammel
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Avinash V. Varadarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.M., G.S.C., L.P., D.R.W., N.H. and A.V.V. designed the research. A.M., L.P. and A.V.V. acquired data from the UK Biobank. A.M. executed the research and analysed the data. S.V. conducted the model explanation analysis. A.M., Y.L. and A.V.V. interpreted the results. A.M., A.H., Y.L. and N.H. prepared the manuscript. All authors contributed to manuscript revision and approved the submitted version.

Corresponding author

Correspondence to Akinori Mitani.

Ethics declarations

Competing interests

The authors are employees of Google and own Alphabet stock or are working at Google. A.M., A.V.V., L.P. and D.R.W. are inventors on a patent applied by Google related to this work (current status: pending).

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Table 1 and Supplementary Figs. 1–8.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mitani, A., Huang, A., Venugopalan, S. et al. Detection of anaemia from retinal fundus images via deep learning. Nat Biomed Eng 4, 18–27 (2020). https://doi.org/10.1038/s41551-019-0487-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41551-019-0487-z

This article is cited by

Search

Advanced search

Quick links

Nature Briefing: Translational Research

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Get what matters in translational research, free to your inbox weekly. Sign up for Nature Briefing: Translational Research