research-article

Scribble-Supervised Meibomian Glands Segmentation in Infrared Images

Authors:

Quan YuanAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 18, Issue 3

Article No.: 88, Pages 1 - 23

https://doi.org/10.1145/3497747

Published: 04 March 2022 Publication History

Abstract

Infrared imaging is currently the most effective clinical method to evaluate the morphology of the meibomian glands (MGs) in patients. As an important indicator for monitoring the development of MG dysfunction, it is necessary to accurately measure gland-drop and gland morphology. Although there are existing methods for automatic segmentation of MGs using deep learning frameworks, they require fully annotated ground-truth labels for training, which is time-consuming and laborious. In this article, we proposed a new scribble-supervised deep learning framework for segmenting the MGs, which only requires easily attainable scribble annotations for training. To cope with the shortage of supervision and regularize the network, a transformation consistent strategy is incorporated, which requires the prediction to follow the same transformation if a transform is performed on an input image of the network. The proposed segmentation method consists of two stages. In the first stage, a U-Net network is used to obtain the meibomian region segmentation map. In the second stage, we concentrate on segmenting glands in the meibomian region. We utilize the gradient prior information of the original image at the decoder part of the segmentation network, which can coarsely locate the target contour. We automatically generate reliable labels using the exponential moving average of the predictions during training and filter out the unreliable pseudo-label by uncertainty threshold. Experimental results on a local MG dataset and two other public medical image datasets demonstrate the effectiveness of the proposed segmentation framework.

References

[1]

P. J. Driver and M. A. Lemp. 1996. Meibomian gland dysfunction. 40, 5 (1996), 343–367.

[2]

Tony H. Ko, James G. Fujimoto, Jay S. Duker, Lelia A. Paunescu, Wolfgang Drexler, Caroline R. Baumal, Carmen A. Puliafito, Elias Reichel, Adam H. Rogers, and Joel S. Schuman. 2005. Comparison of ultrahigh- and standard-resolution optical coherence tomography for imaging macular pathology. Ophthalmology 111, 11 (2005), 2033–2043.

[3]

S. Hassanzadeh, M. Varmaghani, S. Zarei-Ghanavati, J. Heravian Shandiz, and A. Azimi Khorasani. 2021. Global prevalence of meibomian gland dysfunction: A systematic review and meta-analysis. Ocular Immunology and Inflammation 29, 1 (2021), 66–75.

[4]

K. Erich, K. Nadja, M. Thomas, O. Hiroto, and D. A. Sullivan. 2011. The international workshop on meibomian gland dysfunction: Report of the subcommittee on anatomy, physiology, and pathophysiology of the meibomian gland. Investigative Ophthalmology & Visual Science 52, 4 (2011), 1938–78.

[5]

X. Liu, Q. Yuan, Y. Gao, K. He, S. Wang, X. Tang, J. Tang, and D. Shen. 2022. Weakly supervised segmentation of COVID19 infection with scribble annotation on CT images. Pattern Recognition 122 (2022), 108341.

Digital Library

[6]

J. He, Q. Zhu, K. Zhang, P. Yu, and J. Tang. 2021. An evolvable adversarial network with gradient penalty for COVID-19 infection segmentation. Applied Soft Computing 113 (2021), 107947.

Digital Library

[7]

M. M. Marcet, R. M. Shtein, E. A. Bradley, S. X. Deng, D. R. Meyer, J. R. Bilyk, M. T. Yen, W. B. Lee, and L. A. Mawn. 2015. Safety and efficacy of lacrimal drainage system plugs for dry eye syndrome a report by the American academy of ophthalmology. Ophthalmology 122, 8 (2015), 1681–1687.

[8]

Xiaoming Liu, Tianyu Fu, Zhifang Pan, Dong Liu, Wei Hu, Jun Liu, and Kai Zhang. 2019. Automated layer segmentation of retinal optical coherence tomography images using a deep feature enhanced structured random forests classifier. IEEE Journal of Biomedical and Health Informatics 23, 4 (2019), 1404–1416.

[9]

R. Arita, K. Itoh, K. Inoue, and S. Amano. 2008. Noncontact Infrared meibography to document age-related changes of the meibomian glands in a normal population. Ophthalmology 115, 5 (2008), 911–915.

[10]

Stephanie Wong, Sruthi Srinivasan, Paul J. Murphy, and Lyndon Jones. 2019. Comparison of meibomian gland dropout using two infrared imaging devices. Contact Lens and Anterior Eye 42, 3 (2019), 311–317.

[11]

G. N. Foulks and A. J. Bron. 2003. Meibomian gland dysfunction: A clinical scheme for description, diagnosis, classification, and grading. Ocular Surface 1, 3 (2003), 107–126.

[12]

Y. W. Koh, T. Celik, H. K. Lee, A. Petznick, and L. H. Tong. 2012. Detection of meibomian glands and classification of meibography images. Journal of Biomedical Optics 17, 8 (2012), 086008.

[13]

R. Arita, J. Suehiro, T. Haraguchi, R. Shirakawa, H. Tokoro, and S. Amano. 2014. Objective image analysis of the meibomian gland area. British Journal of Ophthalmology 98, 6 (2014), 746–755.

[14]

X. Liu, S. Wang, Y. Zhang, D. Liu, and W. Hu. 2021. Automatic fluid segmentation in retinal optical coherence tomography images using attention based deep learning. Neurocomputing 452 (2021), 576–591.

[15]

X. Liu, A. Yu, X. Wei, Z. Pan, and J. Tang. 2020. Multimodal MR image synthesis using gradient prior and adversarial learning. IEEE Journal of Selected Topics in Signal Processing 14, 6 (2020), 1–1.

[16]

S. M. Prabhu, A. Chakiat, S. Shashank, K. P. Vunnava, and R. Shetty. 2020. Deep learning segmentation and quantification of Meibomian glands. Biomedical Signal Processing and Control 57 (2020), 101776.

[17]

H. Lee and W. K. Jeong. 2020. Scribble2Label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 14–23.

Digital Library

[18]

Y. Wang, J. Zhang, M. Kan, S. Shan, and X. Chen. 2020. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20).

[19]

A. Tarvainen and H. Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17).

[20]

D. T. Nguyen, C. K. Mummadi, T. Ngo, T. Nguyen, L. Beggel, and T. Brox. 2020. SELF: Learning to Filter Noisy Labels with Self-Ensembling. In 2020 International Conference on Learning Representations (ICLR).

[21]

O. Ronneberger, P. Fischer, and T. Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 234–241.

[22]

X. Li, L. Yu, H. Chen, C. W. Fu, and P. A. Heng. 2021. Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Transactions on Neural Networks and Learning Systems 32, 2 (2021), 523–534.

[23]

X. Liu, T. Zhu, L. Zhai, and J. Liu. 2017. Mass classification of benign and malignant with a new twin support vector machine joint l2,1-norm. International Journal of Machine Learning & Cybernetics 10, 5 (2017), 1–17.

[24]

V. Badrinarayanan, A. Kendall, and R. Cipolla. 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 12 (2017), 2481–2495.

[25]

X. Liu, Z. Guo, J. Cao, and J. Tang. 2021. MDC-Net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information. Computers in Biology and Medicine 135 (2021), 104543.

Digital Library

[26]

X. Liu, J. Cao, S. Wang, Y. Zhang, and M. Wang. 2021. Confidence-guided topology-preserving layer segmentation for optical coherence tomography images with focus-column module. IEEE Transactions on Instrumentation and Measurement 70 (2021), 5005612.

[27]

Z. Yang, P. Xu, Y. Yang, and B.-K. Bao. 2021. A densely connected network based on U-Net for medical image segmentation. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 3 (2021), 1–14.

Digital Library

[28]

T. Schlegl, P. Seebck, S. M. Waldstein, G. Langs, and U. Schmidt-Erfurth. 2019. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis 54 (2019), 30–44.

[29]

F. Lin, W. Zhou, J. Deng, B. Li, Y. Lu, and H. Li. 2021. Residual refinement network with attribute guidance for precise saliency detection. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 3 (2021), 1–19.

Digital Library

[30]

J. Wang, T. N. Yeh, R. Chakraborty, S. X. Yu, and M. C. Lin. 2019. A deep learning approach for meibomian gland atrophy evaluation in meibography images. Translational Vision Science Technology 8, 6 (2019), 37.

[31]

H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. 2017. Pyramid scene parsing network. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 2881–2890.

[32]

Z. K. Khan, A. I. Umar, S. H. Shirazi, A. Rasheed, and S. Gul. 2021. Image based analysis of meibomian gland dysfunction using conditional generative adversarial neural network. BMJ Open Ophthalmology 6, 1 (2021), e000436.

[33]

Abhijit Guha Roy, Sailesh Conjeti, Sri Phani Krishna Karri, Debdoot Sheet, Amin Katouzian, Christian Wachinger, and Nassir Navab. 2017. ReLayNet: Retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomedical Optics Express 8, 8 (2017), 3627.

[34]

S. K. Devalla, P. K. Renukanand, B. K. Sreedhar, G. Subramanian, L. Zhang, S. Perera, J. M. Mari, K. S. Chin, T. A. Tun, N. G. Strouthidis, T. Aung, A. H. Thiéry, and M. J. A. Girard. 2018. DRUNET: A dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images. Biomedical Optics Express 9, 7 (2018), 3244–3265.

[35]

P. O. Pinheiro and R. Collobert. 2015. From image-level to pixel-level labeling with convolutional networks. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15).

[36]

F. Saleh, M. S. Aliakbarian, M. Salzmann, L. Petersson, S. Gould, and J. M. Alvarez. 2016. Built-in foreground/ background prior for weakly-supervised semantic segmentation. In European Conference on Computer Vision. Springer, 413–432.

[37]

S. Lee, M. Lee, J. Lee, and H. Shim. 2021. Railroad is not a train: Saliency as pseudo-pixel supervision for weakly supervised semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5495–5505.

[38]

A. Khoreva, R. Benenson, J. Hosang, M. Hein, and B. Schiele. 2016. Simple does It: Weakly supervised instance and semantic segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).

[39]

X. Chen, Y. Yuan, G. Zeng, and J. Wang. 2021. Semi-supervised semantic segmentation with cross pseudo supervision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2613–2622.

[40]

A. Bearman, O. Russakovsky, V. Ferrari, and F. F. Li. 2016. What's the point: Semantic segmentation with point supervision. In European Conference on Computer Vision.

[41]

I. H. Laradji, N. Rostamzadeh, P. O. Pinheiro, D. Vazquez, and M. Schmidt. 2018. Where are the blobs: Counting by localization with point supervision. In European Conference on Computer Vision (ECCV’18). 547–562.

[42]

L. Di, J. Dai, J. Jia, K. He, and S. Jian. 2016. ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).

[43]

B. Wang, G. Qi, S. Tang, T. Zhang, and Y. Zhang. 2019. Boundary perception guidance: A scribble-supervised semantic segmentation approach. In 28th International Joint Conference on Artificial Intelligence (IJCAI’19).

[44]

Daisuke Harada, Ryoma Bise, Hiroki Tokunaga, Wataru Ohyama, Sanae Oka, Toshihiko Fujimori, and Seiichi Uchida. 2019. Scribbles for metric learning in histological image segmentation. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 1026–1030.

[45]

Mihir Sahasrabudhe, Stergios Christodoulidis, Roberto Salgado, Stefan Michiels, Sherene Loi, Fabrice André, Nikos Paragios, and Maria Vakalopoulou. 2020. Self-supervised nuclei segmentation in histopathological images using attention. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 393–402.

Digital Library

[46]

L. Yu, S. Wang, X. Li, C.-W. Fu, and P.-A. Heng. 2019. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 605–613.

Digital Library

[47]

P. A. Yushkevich, Y. Gao, and G. Gerig. 2016. ITK-SNAP: An interactive tool for semi-automatic segmentation of multi-modality biomedical images. In Engineering in Medicine & Biology Society.

[48]

S. C. Pflugfelder, S. C. Tseng, O. Sanabria, H. Kell, C. G. Garcia, C. Felix, W. Feuer, and B. L. Reis. 1998. Evaluation of subjective assessments and objective diagnostic tests for diagnosing tear-film disorders known to cause ocular irritation. Cornea, 17, 1 (1998), 38–56.

[49]

J. J. Nichols, D. A. Berntsen, G. L. Mitchell, and K. K. Nichols. 2005. An assessment of grading scales for meibography images. Cornea 24, 4 (2005), 382–388.

[50]

H. Pult and B. Riede-Pult. 2012. Comparison of subjective grading and objective assessment in meibography. Contact Lens & Anterior Eye: The Journal of the British Contact Lens Association 36, 1 (2012), 22–27.

[51]

V. Ljosa, K. L. Sokolnicki, and A. E. Carpenter. 2012. Annotated high-throughput microscopy image sets for validation. Nature Methods 9, 7 (2012), 637–637.

[52]

E. Villani, L. Marelli, A. Dellavalle, M. Serafino, and P. Nucci. 2020. Latest evidences on meibomian gland dysfunction diagnosis and management. The Ocular Surface 18, 4 (2020), 871–892.

[53]

D. Kingma and J. Ba, 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR'15).

[54]

M. Tang, F. Perazzi, A. Djelouah, I. Ben Ayed, C. Schroers, and Y. Boykov. 2018. On regularized losses for weakly-supervised CNN segmentation. In European Conference on Computer Vision (ECCV’18). 507–522.

[55]

J. Zhang, X. Yu, A. Li, P. Song, and Y. Dai. 2020. Weakly-supervised salient object detection via scribble annotations. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20).

[56]

N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane, and A. Sethi. 2017. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Transactions on Medical Imaging 36, 7 (2017), 1550–1560.

Cited By

Peng BSun LLei JLiu BShen HLi WHuang Q(2024)Self-Supervised Monocular Depth Estimation via Binocular Geometric Correlation LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366357020:8(1-19)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3663570
Huang QLi PHuang YShuang FCai Y(2024)Region-Focused Network for Dense CaptioningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364837020:6(1-20)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3648370
He WLi ZWang HXu TWang ZHuai BYuan NChen E(2024)Multimodal Dialogue Systems via Capturing Context-aware Dependencies and Ordinal Information of Semantic ElementsACM Transactions on Intelligent Systems and Technology10.1145/364509915:3(1-25)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3645099
Show More Cited By

Index Terms

Scribble-Supervised Meibomian Glands Segmentation in Infrared Images
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation

Recommendations

Weakly Supervised Segmentation of COVID19 Infection with Scribble Annotation on CT Images
Highlights
- We propose a weakly supervised COVID-19 infection segmentation method with scribble supervision. To the best of our knowledge, it is the first work adopting ...
Abstract
Segmentation of infections from CT scans is important for accurate diagnosis and follow-up in tackling the COVID-19. Although the convolutional neural network has great potential to automate the segmentation task, most existing deep ...
Scribble-Based Hierarchical Weakly Supervised Learning for Brain Tumor Segmentation
Medical Image Computing and Computer Assisted Intervention – MICCAI 2019
Abstract
The recent state-of-the-art deep learning methods have significantly improved brain tumor segmentation. However, fully supervised training requires a large amount of manually labeled masks, which is highly time-consuming and needs domain ...
Automatic Meibomian Gland Segmentation and Assessment Based on TransUnet with Data Augmentation
Advanced Intelligent Computing Technology and Applications
Abstract
Meibomian glands (MG) are important for eyes and meibomian gland dysfunction (MGD) would lead to dry eye disease. With the development of eye imaging technology, non-contact infrared imaging named meibography has become mainstream. Physicians ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 18, Issue 3

August 2022

478 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3505208

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2022

Accepted: 01 November 2021

Revised: 01 November 2021

Received: 01 August 2021

Published in TOMM Volume 18, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China
Research Project of Hubei Health Committee

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
560
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)6

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Peng BSun LLei JLiu BShen HLi WHuang Q(2024)Self-Supervised Monocular Depth Estimation via Binocular Geometric Correlation LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366357020:8(1-19)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3663570
Huang QLi PHuang YShuang FCai Y(2024)Region-Focused Network for Dense CaptioningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364837020:6(1-20)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3648370
He WLi ZWang HXu TWang ZHuai BYuan NChen E(2024)Multimodal Dialogue Systems via Capturing Context-aware Dependencies and Ordinal Information of Semantic ElementsACM Transactions on Intelligent Systems and Technology10.1145/364509915:3(1-25)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3645099
Qiu HLi HWu QShi HWang LMeng FXu L(2024)Learning Offset Probability Distribution for Accurate Object DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363721420:5(1-24)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3637214
Wu XFeng X(2024)Size Invariant Visual Cryptography Schemes With Evolving Threshold Access StructuresIEEE Transactions on Multimedia10.1109/TMM.2023.328257326(1488-1503)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3282573
Lv CZhang DGeng SWu ZHuang H(2023)Color Transfer for Images: A SurveyACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363515220:8(1-29)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3635152
Yang CWu XChung M(2023)Enhancement of Information Carrying and Decoding for Visual Cryptography with Error CorrectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/361292720:1(1-24)Online publication date: 18-Sep-2023
https://dl.acm.org/doi/10.1145/3612927
Zhang HLi PLiu XYang XAn L(2023)An Iterative Semi-supervised Approach with Pixel-wise Contrastive Loss for Road Extraction in Aerial ImagesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/360637420:3(1-21)Online publication date: 10-Nov-2023
https://dl.acm.org/doi/10.1145/3606374
Liu BLei JPeng BYu CLi WLing N(2023)Novel View Synthesis from a Single Unposed Image via Unsupervised LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/358746719:6(1-23)Online publication date: 31-May-2023
https://dl.acm.org/doi/10.1145/3587467
Xu YYang ZChen TLi KQing C(2023)Progressive Transformer Machine for Natural Character ReenactmentACM Transactions on Multimedia Computing, Communications, and Applications10.1145/355910719:2s(1-22)Online publication date: 17-Feb-2023
https://dl.acm.org/doi/10.1145/3559107
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents