Abstract
Nowadays underwater vision systems are being widely applied in ocean research. However, the largest portion of the ocean - the deep sea - still remains mostly unexplored. Only relatively few image sets have been taken from the deep sea due to the physical limitations caused by technical challenges and enormous costs. Deep sea images are very different from the images taken in shallow waters and this area did not get much attention from the community. The shortage of deep sea images and the corresponding ground truth data for evaluation and training is becoming a bottleneck for the development of underwater computer vision methods. Thus, this paper presents a physical model-based image simulation solution, which uses an in-air texture and depth information as inputs, to generate underwater image sequences taken by robots in deep ocean scenarios. Different from shallow water conditions, artificial illumination plays a vital role in deep sea image formation as it strongly affects the scene appearance. Our radiometric image formation model considers both attenuation and scattering effects with co-moving spotlights in the dark. By detailed analysis and evaluation of the underwater image formation model, we propose a 3D lookup table structure in combination with a novel rendering strategy to improve simulation performance. This enables us to integrate an interactive deep sea robotic vision simulation in the Unmanned Underwater Vehicles simulator. To inspire further deep sea vision research by the community, we release the source code of our deep sea image converter to the public (https://www.geomar.de/en/omv-research/robotic-imaging-simulator).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akkaynak, D., Treibitz, T.: A revised underwater image formation model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6723–6732 (2018)
Akkaynak, D., Treibitz, T., Shlesinger, T., Loya, Y., Tamir, R., Iluz, D.: What is the space of attenuation coefficients in underwater computer vision? In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 568–577. IEEE (2017)
Allais, A., et al.: Sofi: a 3D simulator for the generation of underwater optical images. In: OCEANS 2011, Spain, pp. 1–6. IEEE (2011)
Álvarez-Tuñón, O., Jardón, A., Balaguer, C.: Generation and processing of simulated underwater images for infrastructure visual inspection with UUVs. Sensors 19(24), 5497 (2019)
Bryson, M., Johnson-Roberson, M., Pizarro, O., Williams, S.B.: True color correction of autonomous underwater vehicle imagery. J. Field Robot. 33(6), 853–874 (2016)
Cozman, F., Krotkov, E.: Depth from scattering. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 801–806. IEEE (1997)
Jaffe, J.S.: Computer modeling and the design of optimal underwater imaging systems. IEEE J. Oceanic Eng. 15(2), 101–111 (1990)
Jerlov, N.: Irradiance optical classification. Opt. Oceanogr. 118–120 (1968)
Johnson, L.J.: The underwater optical channel. Dept. Eng., Univ. Warwick, Coventry, UK, Technical report (2012)
Koenig, N., Howard, A.: Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No. 04CH37566), vol. 3, pp. 2149–2154. IEEE (2004)
Li, J., Skinner, K.A., Eustice, R.M., Johnson-Roberson, M.: WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 3(1), 387–394 (2017)
Manhães, M.M.M., Scherer, S.A., Voss, M., Douat, L.R., Rauschenbach, T.: UUV simulator: a gazebo-based package for underwater intervention and multi-robot simulation. In: OCEANS 2016 MTS/IEEE Monterey. IEEE, September 2016. https://doi.org/10.1109/oceans.2016.7761080
McGlamery, B.: A computer model for underwater camera systems. In: Ocean Optics VI, vol. 208, pp. 221–231. International Society for Optics and Photonics (1980)
Mobley, C.D.: Light and Water: Radiative Transfer in Natural Waters. Academic Press, San Diego (1994)
Petzold, T.J.: Volume scattering functions for selected ocean waters. Technical report, Scripps Institution of Oceanography La Jolla Ca Visibility Lab (1972)
Prats, M., Perez, J., Fernández, J.J., Sanz, P.J.: An open source tool for simulation and supervision of underwater intervention missions. In: 2012 IEEE/RSJ international conference on Intelligent Robots and Systems, pp. 2577–2582. IEEE (2012)
Schechner, Y.Y., Karpel, N.: Clear underwater vision. In: 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. I. IEEE (2004)
Sedlazeck, A., Koch, R.: Simulating deep sea underwater images using physical models for light attenuation, scattering, and refraction (2011)
She, M., Song, Y., Mohrmann, J., Köser, K.: Adjustment and calibration of dome port camera systems for underwater vision. In: Fink, G.A., Frintrop, S., Jiang, X. (eds.) DAGM GCPR 2019. LNCS, vol. 11824, pp. 79–92. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33676-9_6
Song, Y., Köser, K., Kwasnitschka, T., Koch, R.: Iterative refinement for underwater 3d reconstruction: application to disposed underwater munitions in the baltic sea. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W10, pp. 181–187 (2019)
Song, Y., Sticklus, J., Nakath, D., Wenzlaff, E., Koch, R., Köser, K.: Optimization of multi-led setups for underwater robotic vision systems. In: Proceedings of the Computer Vision for Automated Analysis of Underwater Imagery Workshop (CVAUI). Springer (2020)
Stephan, T., Beyerer, J.: Computer graphical model for underwater image simulation and restoration. In: 2014 ICPR Workshop on Computer Vision for Analysis of Underwater Imagery, pp. 73–79. IEEE (2014)
Ueda, T., Yamada, K., Tanaka, Y.: Underwater image synthesis from RGB-D images and its application to deep underwater image restoration. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 2115–2119. IEEE (2019)
Acknowledgements
This publication has been funded by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) Projektnummer 396311425, through the Emmy Noether Programme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Song, Y., Nakath, D., She, M., Elibol, F., Köser, K. (2021). Deep Sea Robotic Imaging Simulator. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12662. Springer, Cham. https://doi.org/10.1007/978-3-030-68790-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-68790-8_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68789-2
Online ISBN: 978-3-030-68790-8
eBook Packages: Computer ScienceComputer Science (R0)