Physics-Informed Knowledge Transfer for Underwater Monocular Depth Estimation
Authors: 
Jinghe Yang, Mingming Gong, Ye PuAuthors Info & Claims
Jinghe Yang, Mingming Gong, Ye PuAuthors Info & ClaimsPages 449 - 465
Abstract
Compared to the in-air case, underwater depth estimation has its own challenges. For instance, acquiring high-quality training datasets with groundtruth poses difficulties due to sensor limitations in aquatic environments. Additionally, the physics characteristics of underwater imaging diverge significantly from the in-air case, the methods developed for in-air depth estimation underperform when applied underwater, due to the domain gap. To address these challenges, our paper introduces a novel transfer-learning-based method - Physics-informed Underwater Depth Estimation (PUDE). The key idea is to transfer the knowledge of a pre-trained in-air depth estimation model to underwater settings utilizing a small underwater image set without groundtruth measurement, guided by a physical underwater imaging formation model. We propose novel bound losses based on the physical model to rectify the depth estimations to align with actual underwater physical properties. Finally, in the evaluations across multiple datasets, we compare PUDE model with other existing in-air and underwater methods. The results reveal that the PUDE model excels in both quantitative and qualitative comparisons.
References
[1]
Akkaynak, D., Treibitz, T.: Sea-thru: a method for removing water from underwater images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1682–1691 (2019)
[2]
Amitai, S., Klein, I., Treibitz, T.: Self-supervised monocular depth underwater. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 1098–1104. IEEE (2023)
[3]
Ancuti, C.O., Ancuti, C., De Vleeschouwer, C., Neumann, L., Garcia, R.: Color transfer for underwater dehazing and depth estimation. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 695–699. IEEE (2017)
[4]
Berman, D., Levy, D., Avidan, S., Treibitz, T.: Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
[5]
Bhat, S.F., Alhashim, I., Wonka, P.: AdaBins: depth estimation using adaptive bins. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4009–4018 (2021)
[6]
Bhat, S.F., Birkl, R., Wofk, D., Wonka, P., Müller, M.: ZoeDepth: zero-shot transfer by combining relative and metric depth. arXiv preprint arXiv:2302.12288 (2023)
[7]
Chang HH, Cheng CY, and Sung CC Single underwater image restoration based on depth estimation and transmission compensation IEEE J. Oceanic Eng. 2018 44 4 1130-1149
[8]
Chen, W., Qian, S., Deng, J.: Learning single-image depth from videos using quality assessment networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5604–5613 (2019)
[9]
Chiang JY and Chen YC Underwater image enhancement by wavelength compensation and dehazing IEEE Trans. Image Process. 2011 21 4 1756-1769
[10]
Drews PL, Nascimento ER, Botelho SS, and Campos MFM Underwater depth estimation and image restoration based on single images IEEE Comput. Graph. Appl. 2016 36 2 24-35
[11]
Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: Advances in Neural Information Processing Systems, vol. 27 (2014)
[12]
Fu, H., Gong, M., Wang, C., Batmanghelich, K., Tao, D.: Deep ordinal regression network for monocular depth estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2002–2011 (2018)
[13]
Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 270–279 (2017)
[14]
Godard, C., Mac Aodha, O., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3828–3838 (2019)
[15]
Gupta, H., Mitra, K.: Unsupervised single image underwater depth estimation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 624–628. IEEE (2019)
[16]
Hambarde P, Murala S, and Dhall A UW-GAN: single-image depth estimation and image enhancement for underwater images IEEE Trans. Instrum. Meas. 2021 70 1-12
[17]
He K, Sun J, and Tang X Single image haze removal using dark channel prior IEEE Trans. Pattern Anal. Mach. Intell. 2010 33 12 2341-2353
[18]
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N.: Deeper depth prediction with fully convolutional residual networks. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 239–248. IEEE (2016)
[19]
Levy, D., et al.: Seathru-NeRF: neural radiance fields in scattering media. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 56–65 (2023)
[20]
Li C et al. An underwater image enhancement benchmark dataset and beyond IEEE Trans. Image Process. 2019 29 4376-4389
[21]
Li, Z., Snavely, N.: MegaDepth: learning single-view depth prediction from internet photos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2041–2050 (2018)
[22]
Luo, Y., et al.: Single view stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 155–163 (2018)
[23]
Ming Y, Meng X, Fan C, and Yu H Deep learning for monocular depth estimation: a review Neurocomputing 2021 438 14-33
[24]
Peng YT, Cao K, and Cosman PC Generalization of the dark channel prior for single image restoration IEEE Trans. Image Process. 2018 27 6 2856-2868
[25]
Peng YT and Cosman PC Underwater image restoration based on image blurriness and light absorption IEEE Trans. Image Process. 2017 26 4 1579-1594
[26]
Peng, Y.T., Zhao, X., Cosman, P.C.: Single underwater image enhancement using depth estimation based on blurriness. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4952–4956. IEEE (2015)
[27]
Ramamonjisoa, M., Lepetit, V.: SharpNet: fast and accurate recovery of occluding contours in monocular depth estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)
[28]
Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12179–12188 (2021)
[29]
Ranftl R, Lasinger K, Hafner D, Schindler K, and Koltun V Towards robust monocular depth estimation: mixing datasets for zero-shot cross-dataset transfer IEEE Trans. Pattern Anal. Mach. Intell. 2020 44 3 1623-1637
[30]
Varghese, N., Kumar, A., Rajagopalan, A.: Self-supervised monocular underwater depth recovery, image restoration, and a real-sea video dataset. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12248–12258 (2023)
[31]
Vasiljevic, I., et al.: DIODE: A Dense Indoor and Outdoor DEpth Dataset. CoRR abs/1908.00463 (2019). http://arxiv.org/abs/1908.00463
[32]
Wu, C.Y., Wang, J., Hall, M., Neumann, U., Su, S.: Toward practical monocular indoor depth estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3814–3824 (2022)
[33]
Wu Y, Zhou Y, Chen S, Ma Y, and Li Q Defect inspection for underwater structures based on line-structured light and binocular vision Appl. Opt. 2021 60 25 7754-7764
[34]
Xian, K., et al.: Monocular relative depth perception with web stereo data supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 311–320 (2018)
[35]
Yang, J., Gong, M., Nair, G., Lee, J.H., Monty, J., Pu, Y.: Knowledge distillation for feature extraction in underwater VSLAM. arXiv preprint arXiv:2303.17981 (2023)
[36]
Ye, X., Zhang, J., Yuan, Y., Xu, R., Wang, Z., Li, H.: Underwater depth estimation via stereo adaptation networks. IEEE Trans. Circuits Syst. Video Technol. (2023)
[37]
Yu, B., Wu, J., Islam, M.J.: Udepth: fast monocular depth estimation for visually-guided underwater robots. In: 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 3116–3123. IEEE (2023)
Index Terms
- Physics-Informed Knowledge Transfer for Underwater Monocular Depth Estimation
Index terms have been assigned to the content through auto-classification.
Recommendations
Transferring knowledge from monocular completion for self-supervised monocular depth estimation
AbstractMonocular depth estimation is a very challenging task in computer vision, with the goal to predict per-pixel depth from a single RGB image. Supervised learning methods require large amounts of depth measurement data, which are time-consuming and ...
Underwater Monocular Image Depth Estimation using Single-beam Echosounder
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)This paper proposes a methodology for real-time depth estimation of underwater monocular camera images, fusing measurements from a single-beam echosounder. Our system exploits the echosounder’s detection cone to match its measurements with the ...
Underwater self-supervised depth estimation
Highlights- We proposed a self-supervised monocular depth estimation framework for underwater.
AbstractAccurate underwater depth estimation is a cornerstone of reaching autonomous underwater exploration. However, it is incredibly tricky due to the inherent attenuation character and heavy noise. Fortunately, the depth-changing trend and ...
Comments
Information & Contributors
Information
Published In
Sep 2024
551 pages
ISBN:978-3-031-73208-9
DOI:10.1007/978-3-031-73209-6
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 01 November 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 26 Dec 2024