Abstract
Oral cancer is a major health issue among low- and middle-income countries due to the late diagnosis. Automated algorithms and tools have the potential to identify oral lesions for early detection of oral cancer. In this paper, we aim to develop a novel deep learning framework named D’OraCa to classify oral lesions using photographic images. We are the first to develop a mouth landmark detection model for the oral images and incorporate it into the oral lesion classification model as a guidance to improve the classification accuracy. We evaluate the performance of five different deep convolutional neural networks and MobileNetV2 was chosen as the feature extractor for our proposed mouth landmark detection model. Quantitative and qualitative results demonstrate the effectiveness of the mouth landmark detection model in guiding the classification model to classify the oral lesions into four different referral decision classes. We train our proposed mouth landmark model on a combination of five datasets, containing 221,565 images. Then, we train and evaluate our proposed classification model with mouth landmark guidance using 2,455 oral images. The results are consistent with clinicians and the \(F_1\) score of the classification model is improved to 61.68%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amarasinghe, H., Johnson, N., Lalloo, R., Kumaraarachchi, M., Warnakulasuriya, S.: Derivation and validation of a risk-factor model for detection of oral potentially malignant disorders in populations with high prevalence. Br. J. Cancer 103(3), 303–309 (2010)
Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3444–3451 (2013)
Aubreville, M., et al.: Automatic classification of cancerous tissue in laserendomicroscopy images of the oral cavity using deep learning. Sci. Rep. 7(1), 1–10 (2017)
Ayan, E., Ünver, H.M.: Diagnosis of pneumonia from chest x-ray images using deep learning. In: 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT), pp. 1–5. IEEE (2019)
Bao, P.T., Nguyen, H., Nhan, D.: A new approach to mouth detection using neural network. In: 2009 IITA International Conference on Control, Automation and Systems Engineering (case 2009), pp. 616–619. IEEE (2009)
Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2930–2940 (2013)
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R.L., Torre, L.A., Jemal, A.: Global cancer statistics 2018: globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: Cancer J. Clin. 68(6), 394–424 (2018)
Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)
Chandran, P., Bradley, D., Gross, M., Beeler, T.: Attention-driven cropping for very high resolution facial landmark detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5861–5870 (2020)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Dong, X., Yan, Y., Ouyang, W., Yang, Y.: Style aggregated network for facial landmark detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 379–388 (2018)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)
Folmsbee, J., Liu, X., Brandwein-Weber, M., Doyle, S.: Active deep learning: Improved training efficiency of convolutional neural networks for tissue classification in oral cavity cancer. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 770–773. IEEE (2018)
Fu, Q., et al.: A deep learning algorithm for detection of oral cavity squamous cell carcinoma from photographic images: a retrospective study. EClinicalMedicine 27, 100558 (2020)
Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316(22), 2402–2410 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Howard, A., et al.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019)
Howard, A.G., et al.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Jeyaraj, P.R., Nadar, E.R.S.: Computer-assisted medical image classification for early diagnosis of oral cancer employing deep learning algorithm. J. Cancer Res. Clin. Oncol. 145(4), 829–837 (2019)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
Kowalski, M., Naruniec, J., Trzcinski, T.: Deep alignment network: a convolutional neural network for robust face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 88–97 (2017)
Krishna, M.M.R., et al.: Automated oral cancer identification using histopathological images: a hybrid feature extraction paradigm. Micron 43(2–3), 352–364 (2012)
Laukamp, K.R., et al.: Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI. Eur. Radiol. 29(1), 124–132 (2019)
Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: Interactive facial feature localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7574, pp. 679–692. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33712-3_49
Li, R., et al.: Deep learning based imaging data completion for improved brain disease diagnosis. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8675, pp. 305–312. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10443-0_39
Llewellyn, C.D., Linklater, K., Bell, J., Johnson, N.W., Warnakulasuriya, S.: An analysis of risk factors for oral cancer in young people: a case-control study. Oral Oncol. 40(3), 304–313 (2004)
Lv, J., Shao, X., Xing, J., Cheng, C., Zhou, X.: A deep regression architecture with two-stage re-initialization for high performance facial landmark detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3317–3326 (2017)
Mintz, Y., Brodie, R.: Introduction to artificial intelligence in medicine. Minim. Invasive Ther. Allied Technol. 28(2), 73–81 (2019)
Nagao, T., Warnakulasuriya, S.: Screening for oral cancer: future prospects, research and policy development for Asia. Oral Oncol. 105, 104632 (2020)
Pantic, M., Tomc, M., Rothkrantz, L.J.: A hybrid approach to mouth features detection. In: 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat. No. 01CH37236), vol. 2, pp. 1188–1193. IEEE (2001)
Rajpurkar, P., et al.: Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)
Rimal, J., Shrestha, A., Maharjan, I.K., Shrestha, S., Shah, P.: Risk assessment of smokeless tobacco among oral precancer and cancer patients in eastern developmental region of Nepal. Asian Pac. J. Cancer Prev.: APJCP 20(2), 411 (2019)
Saba, T., Khan, M.A., Rehman, A., Marie-Sainte, S.L.: Region extraction and classification of skin cancer: A heterogeneous framework of deep CNN features fusion and reduction. J. Med. Syst. 43(9), 1–19 (2019)
Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: database and results. Image Vis. Comput. 47, 3–18 (2016)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: The first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2013)
Song, B., et al.: Automatic classification of dual-modalilty, smartphone-based oral dysplasia and malignancy images using deep learning. Biomed. Opt. Express 9(11), 5318–5329 (2018)
Tzimiropoulos, G.: Project-out cascaded regression with an application to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3659–3667 (2015)
Uthoff, R.D., et al.: Point-of-care, smartphone-based, dual-modality, dual-view, oral cancer screening device with neural network classification for low-resource communities. PloS ONE 13(12), e0207493 (2018)
Van der Waal, I., de Bree, R., Brakenhoff, R., Coebegh, J.: Early diagnosis in primary oral cancer: is it possible? Medicina oral, patologia oral y cirugia bucal 16(3), e300–e305 (2011)
Welikala, R.A., et al.: Automated detection and classification of oral lesions using deep learning for early detection of oral cancer. IEEE Access 8, 132677–132693 (2020)
Welikala, R.A., et al.: Fine-tuning deep learning architectures for early detection of oral cancer. In: Bebis, G., Alekseyev, M., Cho, H., Gevertz, J., Rodriguez Martinez, M. (eds.) ISMCO 2020. LNCS, vol. 12508, pp. 25–31. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64511-3_3
Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)
Xu, S., et al.: An early diagnosis of oral cancer based on three-dimensional convolutional neural networks. IEEE Access 7, 158603–158611 (2019)
Yu, X., Huang, J., Zhang, S., Yan, W., Metaxas, D.N.: Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1944–1951 (2013)
Yu, X., Zhou, F., Chandraker, M.: Deep deformation network for object landmark localization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 52–70. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_4
Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 94–108. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_7
Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886. IEEE (2012)
Acknowledgements
This work was supported by the Medical Research Council under grant MR/S013865/1.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lim, J.H. et al. (2021). D’OraCa: Deep Learning-Based Classification of Oral Lesions with Mouth Landmark Guidance for Early Detection of Oral Cancer. In: Papież, B.W., Yaqub, M., Jiao, J., Namburete, A.I.L., Noble, J.A. (eds) Medical Image Understanding and Analysis. MIUA 2021. Lecture Notes in Computer Science(), vol 12722. Springer, Cham. https://doi.org/10.1007/978-3-030-80432-9_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-80432-9_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80431-2
Online ISBN: 978-3-030-80432-9
eBook Packages: Computer ScienceComputer Science (R0)