Abstract
The placenta is a valuable organ that can aid in understanding adverse events during pregnancy and predicting issues post-birth. Manual pathological examination and report generation, however, are laborious and resource-intensive. Limitations in diagnostic accuracy and model efficiency have impeded previous attempts to automate placenta analysis. This study presents a novel framework for the automatic analysis of placenta images that aims to improve accuracy and efficiency. Building on previous vision-language contrastive learning (VLC) methods, we propose two enhancements, namely Pathology Report Feature Recomposition and Distributional Feature Recomposition, which increase representation robustness and mitigate feature suppression. In addition, we employ efficient neural networks as image encoders to achieve model compression and inference acceleration. Experiments validate that the proposed approach outperforms prior work in both performance and efficiency by significant margins. The benefits of our method, including enhanced efficacy and deployability, may have significant implications for reproductive healthcare, particularly in rural areas or low- and middle-income countries.
Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (NIH) under award number R01EB030130 and the College of Information Sciences and Technology of The Pennsylvania State University. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. This work used computing resources at the Pittsburgh Supercomputer Center through allocation IRI180002 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants Nos. 2138259, 2138286, 2138307, 2137603, and 2138296.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Asadpour, V., Puttock, E.J., Getahun, D., Fassett, M.J., Xie, F.: Automated placental abruption identification using semantic segmentation, quantitative features, SVM, ensemble and multi-path CNN. Heliyon 9(2), e13577:1–13 (2023)
Bakkali, S., Ming, Z., Coustaty, M., Rusiñol, M., Terrades, O.R.: VLCDoC: vision-language contrastive pre-training model for cross-modal document classification. Pattern Recogn. 139(109419), 1–11 (2023)
Chen, Y., Wu, C., Zhang, Z., Goldstein, J.A., Gernand, A.D., Wang, J.Z.: PlacentaNet: automatic morphological characterization of placenta photos with deep learning. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 487–495. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_54
Chen, Y., et al.: AI-PLAX: AI-based placental assessment and examination using photos. Comput. Med. Imaging Graph. 84(101744), 1–15 (2020)
Cui, Q., et al.: Contrastive vision-language pre-training with limited resources. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision - ECCV 2022. ECCV 2022. LNCS, vol. 13696, pp. 236–253. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20059-5_14
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dong, X., et al.: MaskCLIP: masked self-distillation advances contrastive language-image pretraining. arXiv preprint arXiv:2208.12262 (2022)
Dormer, J.D., et al.: CascadeNet for hysterectomy prediction in pregnant women due to placenta accreta spectrum. In: Proceedings of SPIE-the International Society for Optical Engineering, vol. 12032, pp. 156–164. SPIE (2022)
Goldstein, J.A., Gallagher, K., Beck, C., Kumar, R., Gernand, A.D.: Maternal-fetal inflammation in the placenta and the developmental origins of health and disease. Front. Immunol. 11(531543), 1–14 (2020)
Gupta, K., Balyan, K., Lamba, B., Puri, M., Sengupta, D., Kumar, M.: Ultrasound placental image texture analysis using artificial intelligence to predict hypertension in pregnancy. J. Matern.-Fetal Neonatal. Med. 35(25), 5587–5594 (2022)
Howard, A., et al.: Searching for MobileNetV3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019)
Jia, C., et al.: Scaling up visual and vision-language representation learning with noisy text supervision. In: Proceedings of the International Conference on Machine Learning, pp. 4904–4916. PMLR (2021)
Khodaee, A., Grynspan, D., Bainbridge, S., Ukwatta, E., Chan, A.D.: Automatic placental distal villous hypoplasia scoring using a deep convolutional neural network regression model. In: Proceedings of the IEEE International Instrumentation and Measurement Technology Conference (I2MTC), pp. 1–5. IEEE (2022)
Li, T., et al.: Addressing feature suppression in unsupervised visual representations. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1411–1420 (2023)
Mobadersany, P., Cooper, L.A., Goldstein, J.A.: GestAltNet: aggregation and attention to improve deep learning of gestational age from placental whole-slide images. Lab. Invest. 101(7), 942–951 (2021)
Pan, Y., Gernand, A.D., Goldstein, J.A., Mithal, L., Mwinyelle, D., Wang, J.Z.: Vision-language contrastive learning approach to robust automatic placenta analysis using photographic images. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol. 13433, pp 707–716. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16437-8_68
Pietsch, M., et al.: APPLAUSE: automatic prediction of PLAcental health via U-net segmentation and statistical evaluation. Med. Image Anal. 72(102145), 1–11 (2021)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: Proceedings of the International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)
Roberts, D.J.: Placental pathology, a survival guide. Arch. Pathol. Labor. Med. 132(4), 641–651 (2008)
Specktor-Fadida, B., et al.: A bootstrap self-training method for sequence transfer: state-of-the-art placenta segmentation in fetal MRI. In: Sudre, C.H., et al. (eds.) UNSURE/PIPPI -2021. LNCS, vol. 12959, pp. 189–199. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87735-4_18
Sun, H., Jiao, J., Ren, Y., Guo, Y., Wang, Y.: Multimodal fusion model for classifying placenta ultrasound imaging in pregnancies with hypertension disorders. Pregnancy Hypertension 31, 46–53 (2023)
Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: Proceedings of the International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Wang, Y., Li, Y.Z., Lai, Q.Q., Li, S.T., Huang, J.: RU-net: an improved U-Net placenta segmentation network based on ResNet. Comput. Methods Program. Biomed. 227(107206), 1–7 (2022)
Wen, K., Xia, J., Huang, Y., Li, L., Xu, J., Shao, J.: COOKIE: contrastive cross-modal knowledge sharing pre-training for vision-language representation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2208–2217 (2021)
Yang, Y., et al.: A comparative analysis of eleven neural networks architectures for small datasets of lung images of COVID-19 patients toward improved clinical decisions. Comput. Biol. Med. 139(104887), 1–26 (2021)
Ye, Z., Xuan, R., Ouyang, M., Wang, Y., Xu, J., Jin, W.: Prediction of placenta accreta spectrum by combining deep learning and radiomics using T2WI: A multicenter study. Abdom. Radiol. 47(12), 4205–4218 (2022)
Zhang, P., et al.: Vinvl: revisiting visual representations in vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5579–5588 (2021)
Zhang, Y., Jiang, H., Miura, Y., Manning, C.D., Langlotz, C.P.: Contrastive learning of medical visual representations from paired images and text. In: Proceedings of the Machine Learning for Healthcare Conference, pp. 2–25. PMLR (2022)
Zhang, Z., Davaasuren, D., Wu, C., Goldstein, J.A., Gernand, A.D., Wang, J.Z.: Multi-region saliency-aware learning for cross-domain placenta image segmentation. Pattern Recogn. Lett. 140, 165–171 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, Y. et al. (2023). Enhancing Automatic Placenta Analysis Through Distributional Feature Recomposition in Vision-Language Contrastive Learning. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14225. Springer, Cham. https://doi.org/10.1007/978-3-031-43987-2_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-43987-2_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43986-5
Online ISBN: 978-3-031-43987-2
eBook Packages: Computer ScienceComputer Science (R0)
