Abstract
Current image understanding methods on dental data are often trained end-to-end on inputs and labels, with focus on using state-of-the-art neural architectures. Such approaches, however, typically ignore domain specific peculiarities and lack the ability to generalize outside their training dataset. We observe that, in RGB images, teeth display a weak or unremarkable texture while exhibiting strong boundaries; similarly, in panoramic radiographs root tips and crowns are generally visible, while other parts of the teeth appear blurry. In their respective image type, these features are robust to the domain shift given by different sources or acquisition tools. Therefore, we formulate a method which we call MultiVT, able to leverage these low level image features to achieve results with good domain generalization properties. We demonstrate with experiments that, by focusing on such domain-robust features, we can achieve better segmentation and detection results. Additionally, MultiVT improves generalization capabilities without applying domain adaptive techniques - a characteristic which renders our method suitable for use in real-world applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Burgner-Kahrs, J., Rucker, D.C., Choset, H.: Continuum robots for medical applications: a survey. IEEE Trans. Robot. 31, 1261–1280 (2015)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
Cheng, B., et al.: Panoptic-DeepLab: a simple, strong, and fast baseline for bottom-up panoptic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (2020)
Collobert, R., Weston, J.: A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning (2008)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (2019)
Schwendicke, F., Samek, W., Krois, J.: Artificial intelligence in dentistry: chances and challenges. J. Dent. Res. 99, 769–774 (2020)
Finlayson, S.G., Bowers, J.D., Ito, J., Zittrain, J.L., Beam, A.L., Kohane, I.S.: Adversarial attacks on medical machine learning. Science 363(6433), 1287–1289 (2019)
Frid-Adar, M., Diamant, I., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Gan-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321, 321–331 (2018)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning (2015)
Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63, 139–144 (2020)
Guan, H., Liu, M.: Domain adaptation for medical image analysis: a survey. IEEE Trans. Biomed. Eng. 69, 1173–1185 (2022)
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR (2015)
Koprinska, I., Carrato, S.: Temporal video segmentation: a survey. In: Signal Processing: Image Communication (2001)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012)
Li, D., Yang, Y., Song, Y.Z., Hospedales, T.M.: Deeper, broader and artier domain generalization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) (2017)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. In: 4th International Conference on Learning Representations, ICLR (2016)
Liu, Q., Chen, C., Qin, J., Dou, Q., Heng, P.A.: FEDDG: federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 99–106 (2021)
Pooch, E.H.P., Ballester, P., Barros, R.C.: Can we trust deep learning based diagnosis? The impact of domain shift in chest radiograph classification (2020)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91
Rella, E.M., Chhatkuli, A., Liu, Y., Konukoglu, E., Gool, L.V.: Zero pixel directional boundary by vector transform. In: International Conference on Learning Representations (2022)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems (2015)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schwendicke, F., et al.: Artificial intelligence in dental research: checklist for authors, reviewers, readers. J. Dent. 10, 103610 (2021)
Wang, J., et al.: Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intelli. 43, 3349–3364 (2020)
Yadav, S.S., Jadhav, S.M.: Deep convolutional neural network based medical image classification for disease diagnosis. J. Big Data 6, 1–18 (2019)
Acknowledgements
This research was funded by Align Technology Switzerland GmbH (project AlignTech-ETH). Research was also partially funded by VIVO Collaboration Project on Real-time scene reconstruction.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mello Rella, E., Chhatkuli, A., Konukoglu, E., Van Gool, L. (2024). MultiVT: Multiple-Task Framework for Dentistry. In: Koch, L., et al. Domain Adaptation and Representation Transfer. DART 2023. Lecture Notes in Computer Science, vol 14293. Springer, Cham. https://doi.org/10.1007/978-3-031-45857-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-45857-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45856-9
Online ISBN: 978-3-031-45857-6
eBook Packages: Computer ScienceComputer Science (R0)
