Abstract
Fine-grained classification and counting of bone marrow erythroid cells are vital for evaluating the health status and formulating therapeutic schedules for leukemia or hematopathy. Due to the subtle visual differences between different types of erythroid cells, it is challenging to apply existing image-based deep learning models for fine-grained erythroid cell classification. Moreover, there is no large open-source datasets on erythroid cells to support the model training. In this paper, we introduce BMEC (Bone Morrow Erythroid Cells), the first large fine-grained image dataset of erythroid cells, to facilitate more deep learning research on erythroid cells. BMEC contains 5,666 images of individual erythroid cells, each of which is extracted from the bone marrow erythroid cell smears and professionally annotated to one of the four types of erythroid cells. To distinguish the erythroid cells, one key indicator is the cell shape which is closely related to the cell growth and maturation. Therefore, we design a novel shape-aware image classification network for fine-grained erythroid cell classification. The shape feature is extracted from the shape mask image and aggregated to the raw image feature with a shape attention module. With the shape-attended image feature, our network achieved superior classification performance (81.12% top-1 accuracy) on the BMEC dataset comparing to the baseline methods. Ablation studies also demonstrate the effectiveness of incorporating the shape information for the fine-grained cell classification. To further verify the generalizability of our method, we tested our network on two additional public white blood cells (WBC) datasets and the results show our shape-aware method can generally outperform recent state-of-the-art works on classifying the WBC. The code and BMEC dataset can be found on https://github.com/wangye8899/BMEC.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The code and dataset have been published on github (https://github.com/wangye8899/BMEC).
Code Availability
References
Chitra P, Jebarani M, Kavipriya P, Srilatha K, Sumathi M, Lakshmi S (2019) Detection of aml in blood microscopic images using local binary pattern and supervised classifier. Res J Pharm Technol 12(4):1717–1720
Alomari YM, Sheikh Abdullah SNH, Zaharatul Azma R, Omar K (2014) Automatic detection and quantification of wbcs and rbcs using iterative structured circle detection algorithm. Comput Math. Meth Med
Lippeveld M, Knill C, Ladlow E, Fuller A, Michaelis LJ, Saeys Y, Filby A, Peralta D (2020) Classification of human white blood cells using machine learning for stain-free imaging flow cytometry. Cytometry A 97(3):308–319
Petrović N , Moyà-alcover G, Jaume-i-capó A, González-Hidalgo M (2020) Sickle-cell disease diagnosis support selecting the most appropriate machine learning method : towards a general and interpretable approach for cell morphology analysis from microscopy images. Comput Biol Med 126:104027
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer : hierarchical vision transformer using shifted windows. arXiv:2103.14030
Xu M, Zhang Z, Hu H, Wang J, Wang L, Wei F, Bai X, Liu Z (2021) End-to-end semi-supervised object detection with soft teacher. Proc ICCV
Strudel R, Garcia R, Laptev I, Schmid C (2021) Segmenter: transformer for semantic segmentation
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE CVPR pp 770–778
Huang G, Liu Z, Van Der Maaten L , Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE CVPR pp 4700–4708
Zhang H, Wu C, Zhang Z, Zhu Y, Zhang Z, Lin H, Sun Y, He T, Mueller J, Manmatha R, Li M, Smola A J (2020) Resnest : split-attention networks. arxiv: Comput Vis Pattern Recognit
Rezatofighi SH, Soltanian-Zadeh H (2011) Automatic recognition of five types of white blood cells in peripheral blood. Comput Med Imaging Graph 35(4):333–343
Kouzehkanan ZM, et al., Saghari S, Tavakoli E, Rostami P, Abaszadeh M, Mirzadeh F, Satlsar ES, Gheidishahran M, Gorgi F, Mohammadi S (2021) Raabin-wbc: a large free access dataset of white blood cells from normal peripheral blood. bioRxiv
Alam MM, Islam MT (2019) Machine learning approach of automatic identification and counting of blood cells. Healthc Technol Lett 6(4):103–108
Gonzalez-Hidalgo M, Guerrero-Pena F, Herold-garcía S, Jaume-i-capó A, Marrero-Fernández PD (2014) Red blood cell cluster separation from digital images for use in sickle cell disease. IEEE J Biomed Health Inform 19(4):1514–1525
Labati RD, Piuri V, Scotti F (2011) All-idb : the acute lymphoblastic leukemia image database for image processing. In: 2011 18th IEEE international conference on image processing, IEEE, pp 2045–2048
Zheng X, Wang Y, Wang G, Liu J (2018) Fast and robust segmentation of white blood cell images by self-supervised learning. Micron 107:55–71
Naruenatthanaset K, Chalidabhongse TH, Palasuwan D, Anantrasirichai N, Palasuwan A (2020) Red blood cell segmentation with overlapping cell separation and classification on imbalanced dataset. arXiv:2012.01321
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38(4):367–378
Tavakoli E, Ghaffari A, Kouzehkanan ZM, Hosseini R (2021) New segmentation and feature extraction algorithm for classification of white blood cells in peripheral smear images. bioRxiv
Liu C, Huang L, Wei Z, Zhang W (2021) Subtler mixed attention network on fine-grained image classification. Appl Intell 51(11):7903–7916
Wang L, He K, Feng X, Ma X (2022) Multilayer feature fusion with parallel convolutional block for fine-grained image classification. Appl Intell 52(3):2872–2883
Sun C, Ai Y, Wang S, Zhang W (2021) Mask-guided ssd for small-object detection. Appl Intell 51(6):3311–3322
Zhou Y, Wang Y, Wu J, Hassan M, Pang W, Lv L, Wang L, Cui H (2022) Erythroidcounter : an automatic pipeline for erythroid cell detection, identification and counting based on deep learning. Multimed Tools Appl, 1–16
Toğaçar M, Ergen B, Cömert Z (2020) Classification of white blood cells using deep features obtained from convolutional neural network models based on the combination of feature selection methods. Appl Soft Comput 97:106810
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf. Process Syst 25:1097–1105
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D , Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE CVPR, pp 1–9
Sahlol AT, Kollmannsberger P, Ewees AA (2020) Efficient classification of white blood cell leukemia with improved swarm optimization of deep features. Sci Rep 10(1):1–11
Mirjalili S, Gandomi AH, Mirjalili SZ, Saremi S, Faris H, Mirjalili SM (2017) Salp swarm algorithm : a bio-inspired optimizer for engineering design problems. Adv Eng Softw 114:163–191
Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Duan Y (2020) Deep learning models for classification of red blood cells in microscopy images to aid in sickle cell anemia diagnosis. Electronics 9(3):427
Pasupa K, Vatathanavaro S, Tungjitnob S (2020) Convolutional neural networks based focal loss for class imbalance problem : a case study of canine red blood cells morphology classification. J Ambient Intell Humanized Comput, 1–17
Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: ECCV
Tavakoli S, Ghaffari A, Kouzehkanan ZM (2021) Generalizability in white blood cells’ classification problem. bioRxiv
Gregory TR (2001) The bigger the c-value, the larger the cell : genome size and red blood cell size in vertebrates. Blood Cells Mol Dis 27(5):830–843
Otsu N (1979) A threshold selection method from gray level histograms. IEEE Trans Syst Man Cybern 9:62–66
Hu J, Shen L, Albanie S, Sun G, Wu E (2018) Squeeze-and-excitation networks. In: Computer vision and pattern recognition
Wightman R (2019) Pytorch Image Models GitHub. https://doi.org/10.5281/zenodo.4414861
Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch. In: NIPS-W
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers : surpassing human-level performance on imagenet classification. In: International conference on computer vision
Loshchilov I, Hutter F (2016) Sgdr: stochastic gradient descent with warm restarts. arXiv:1608.03983
Jung C, Abuhamad M, Alikhanov J, Mohaisen A, Han K, Nyang D (2019) W-net: a cnn-based architecture for white blood cells image classification. arXiv:1910.01091
Baydilli YY, Atila Ü (2020) Classification of white blood cells using capsule networks. Comput Med Imaging Graph 80:101699
Harshanand B, Sangaiah AK (2020) Comprehensive analysis of deep learning methodology in classification of leukocytes and enhancement using swish activation units. Mobile networks and applications 25 (6):2302–2320
Khan MA, Qasim M, Lodhi HMJ, Nazir M, Javed K, Rubab S, Din A, Habib U (2021) Automated design for recognition of blood cells diseases from hematopathology using classical features selection and elm. Microsc Res Tech 84(2):202–216
Funding
This research is supported by the National Natural Science Foundation of China (Grants No. 61772227, 61972174, 61972175, 62202199), Science and Technology Development Foundation of Jilin Province (No. 20180201045GX, 20200201300JC, 20200401083GX, 20200201163JC), the Jilin Development and Reform Commission Fund (No. 2020C020-2).
Author information
Authors and Affiliations
Contributions
Conceptualization, Y.W. and R.M.; methodology, Y.W., R.M. and Y.Z.; validation, X.M., Y.X. and X.W.; formal analysis, Y.W. and R.M.; investigation, H.C.; resources, Y.Z.; data curation, H.C. and X.M.; writing—original draft preparation, Y.W.; writing—review and editing, R.M.; visualization, Y.W. and R.M.; supervision, Y.Z.; project administration, Y.Z.; All authors have read and agreed to the published version of the manuscript.
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Ye Wang and Rui Ma contributed equally to this work.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, Y., Ma, R., Ma, X. et al. Shape-aware fine-grained classification of erythroid cells. Appl Intell 53, 19115–19127 (2023). https://doi.org/10.1007/s10489-023-04465-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04465-z