Abstract
Underwater images taken from autonomous underwater vehicles (AUVs) often suffer from low light, high turbidity, poor contrast, motion-blur, and excessive light scattering and hence require image enhancement techniques for object recognition. Machine learning methods are being increasingly used for object recognition under such adverse conditions. These enhanced object recognition methods of images taken from AUVs have potential applications in underwater pipeline and optical fibre surveillance, ocean bed resource extraction, ocean floor mapping, underwater species exploration, etc. While the classical machine learning methods are very efficient in terms of accuracy, they require large datasets and high computational time for image classification. In the current work, we use quantum-classical hybrid machine learning methods for real-time underwater object recognition on-board an AUV for the first time. We use real-time motion-blurred and low-light images taken from an on-board camera of AUV built in-house and apply existing hybrid machine learning methods for object recognition. Our hybrid methods consist of quantum encoding and flattening of classical images using quantum circuits and sending them to classical neural networks for image classification. The results of hybrid methods carried out using Pennylane-based quantum simulators both on GPU and using pre-trained models on an on-board NVIDIA GPU chipset are compared with results from corresponding classical machine learning methods. We observe that the hybrid quantum machine learning methods show an efficiency greater than 65% and reduction in runtime by one-thirds and require 50% smaller dataset sizes for training the models compared to classical machine learning methods. We hope that our work opens up further possibilities in quantum enhanced real-time computer vision in autonomous vehicles.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
No datasets were generated or analyzed during the current study.
References
Aguirre-Castro OA, Inzunza-González E, García-Guerrero EE, Tlelo-Cuautle E, López-Bonilla OR, Olguín-Tiznado JE, Cárdenas-Valdez JR (2019) Design and construction of an ROV for underwater exploration. Sensors 19(24):5387. https://doi.org/10.3390/s19245387
Atlas L, Homma T, Marks R (1987) An artificial neural network for spatio-temporal bipolar patterns: application to phoneme classification. In: Neural information processing systems, vol 0, pp 31–40
Bergholm V et al (2018) PennyLane: automatic differentiation of hybrid quantum-classical computations. arXiv preprint. https://arxiv.org/abs/1811.04968
Bergler C, Schröter H, Cheng RX, Barth V, Weber M, Nöth E, Hofer H, Maier A (2019) ORCA-SPOT: an automatic killer whale sound detection toolkit using deep learning. Sci Rep 9:10997. https://doi.org/10.1038/s41598-019-47335-w
Bicknell AW, Godley BJ, Sheehan EV, Votier SC, Witt MJ (2016) Camera technology for monitoring marine biodiversity and human impact. Front Ecol Environ 14(8):424–432. https://doi.org/10.1002/fee.1322
Bokhan D, Mastiukova AS, Boev AS, Trubnikov DN, Fedorov AK (2022) Multiclass classification using quantum convolutional neural networks with hybrid quantum-classical learning. Front Phys 10. https://doi.org/10.3389/fphy.2022.1069985
Cameron F, Jahidul IM, Junaed S (2017) Enhancing underwater imagery using generative adversarial networks. In: 2018 IEEE International conference on robotics and automation (ICRA), pp 183–192. https://doi.org/10.1007/978-3-319-59773-7_19
Capocci R, Dooly G, Omerdić E, Coleman J, Newe T, Toal D (2017) Inspection-class remotely operated vehicles—a review. J Mar Sci Eng 5(1). https://doi.org/10.3390/jmse5010013
Chao L, Wang M (2010) Removal of water scattering. In: 2010 2nd International conference on computer engineering and technology, vol 2, pp 2–35239. https://doi.org/10.1109/ICCET.2010.5485339
Chemisky B, Menna F, Nocerino E, Drap P (2021) Underwater survey for oil and gas industry: a review of close range optical methods. Remote Sensing 13(14):2789. https://doi.org/10.3390/rs13142789
Chen X, Yu J, Kong S, Wu Z, Fang X, Wen L (2019) Towards real-time advancement of underwater visual quality with GAN. IEEE Trans Industr Electron 66(12):9350–9359. https://doi.org/10.1109/TIE.2019.2893840
Chiang JY, Ying-Ching C (2012) Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans Image Process 21(4):1756–1769. https://doi.org/10.1109/TIP.2011.2179666
Codruta A, Cosmin A, Christophe DV, Philippe B (2018) Color balance and fusion for underwater image enhancement. IEEE Trans Image Process 27(1):379–393. https://doi.org/10.1109/TIP.2017.2759252
Cong I, Choi S, Lukin MD (2019) Quantum convolutional neural networks. Nat Phys 15:1273–1278. https://doi.org/10.1038/s41567-019-0648-8
Dang Y, Jiang N, Hu H, Ji Z, Zhang W (2018) Image classification based on quantum k-nearest-neighbor algorithm. Quantum Inf Process 17(9):239. https://doi.org/10.1007/s11128-018-2004-9
Danovaro R, Fanelli E, Aguzzi J, Billett D, Carugati L, Corinaldesi C, Dell’Anno A, Gjerde K, Jamieson AJ, Kark S, McClain C, Levin L, Levin N, Ramirez-Llodra E, Ruhl H, Smith CR, Snelgrove PVR, Thomsen L, Dover CLV, Yasuhara M (2020) Ecological variables for developing a global deep-ocean monitoring and conservation strategy. Nat Ecol Evol 4:181–192. https://doi.org/10.1038/s41559-019-1091-z
Das M, Bolisetti T (2023) Variational quantum neural networks (VQNNS) in image classification. arXiv preprint. https://arxiv.org/pdf/2303.05860
Enrico S, Giuseppe C (2017) Manipulation and transportation with cooperative underwater vehicle manipulator systems. IEEE J Oceanic Eng 42(4):782–799. https://doi.org/10.1109/JOE.2016.2618182
Fan J, Wang X, Zhou C, Ou Y, Jing F, Hou Z (2023) Development, calibration, and image processing of underwater structured light vision system: a survey. IEEE Trans Instrum Meas 72:1–18. https://doi.org/10.1109/TIM.2023.3235420
Fukushima K (1969) Visual feature extraction by a multilayered network of analog threshold elements. IEEE Trans Syst Sci Cybern 5(4):322–333. https://doi.org/10.1109/TSSC.1969.300225
Ghani ASA, Isa NAM (2015) Enhancement of low quality underwater image through integrated global and local contrast correction. Appl Soft Comput 37. https://doi.org/10.1016/j.asoc.2015.08.033
Guan X, Jian S, Hongda P, Zhiguo Z, Haibin G (2009) An image enhancement method based on gamma correction. In: 2009 Second international symposium on computational intelligence and design, vol 1, pp 60–63. https://doi.org/10.1109/ISCID.2009.22
Guo Y, Li H, Zhuang P (2019) Underwater image enhancement using a multiscale dense generative adversarial network. IEEE J Oceanic Eng 45(3):862–870. https://doi.org/10.1109/JOE.2019.2911447
Henderson M, Shakya S, Pradhan S, Cook T (2020) Quanvolutional neural networks: powering image recognition with quantum circuits. Quantum Mach Intell 2. https://doi.org/10.1007/s42484-020-00012-y
Huang G, Liu Z, Van Der Maaten L, Kilian QW (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708. https://doi.org/10.1109/CVPR.2017.243
Huimin L, Yujie L, Seiichi S (2017) Computer vision for ocean observing. Artif Intell Comput Vi, pp 1–16. Springer. https://doi.org/10.1007/978-3-319-46245-5_1
Jacobi M (2015) Autonomous inspection of underwater structures. Robot Auton Syst 67:80–86. https://doi.org/10.1016/j.robot.2014.10.006
Joshi KR, Kamathe RS (2008) Quantification of retinex in enhancement of weather degraded images. In: 2008 International conference on audio, language and image processing, pp 1229–1233. https://doi.org/10.1109/ICALIP.2008.4590120
Kanaev AV, Smith LN, Hou WW, Woods S (2012) Restoration of turbulence degraded underwater images. Opt Eng 51. https://doi.org/10.1117/1.OE.51.5.057007
Katzschmann RK, DelPreto J, MacCurdy R, Rus D (2018) Exploration of underwater life with an acoustically controlled soft robotic fish. Sci Robot 3(16). https://doi.org/10.1126/scirobotics.aar3449
Khan A, Ali SSA, Anwer A, Adil SH, Mériaudeau F (2018) Subsea pipeline corrosion estimation by restoring and enhancing degraded underwater images. IEEE Access 6:40585–40601. https://doi.org/10.1109/ACCESS.2018.2855725
Kirchner F (2020) A survey of challenges and potentials for ai technologies. In: Kirchner F, Straube S, Kühn D, Hoyer N (eds) AI Technology for underwater robots. Intelligent systems, control and automation: science and engineering, vol 96, pp 3–17. Springer, Cham. https://doi.org/10.1007/978-3-030-30683-0_1
Le PQ, Dong F, Hirota K (2011) A flexible representation of quantum images for polynomial preparation, image compression, and processing operations. Quantum Inf Process 10:63–84. https://doi.org/10.1007/s11128-010-0177-y
Li W, Chu P-C, Liu G-Z, Tian Y-B, Qiu T-H, Wang S-M (2022) An image classification algorithm based on hybrid quantum classical convolutional neural network. Quantum Eng 2022(1):5701479. https://doi.org/10.1155/2022/5701479
Marionette: sea animals image dataset (2024). Accessed 23 Jan 2024. https://www.kaggle.com/datasets/vencerlanz09/sea-animals-image-dataste
Matthew L, Qin R, Mao X (2022) A review on machine learning, artificial intelligence, and smart technology in water treatment and monitoring. Water 14. https://doi.org/10.3390/w14091384
Mohammad J, Wei X, Hanzo L, Mostafa RA (2021) Internet of underwater things and big marine data analytics—a comprehensive survey. IEEE Communications Surveys & Tutorials 23(2):904–956. https://doi.org/10.1109/COMST.2021.3053118
Myungjin C, Bahram J (2010) Three-dimensional visualization of objects in turbid water using integral imaging. J Disp Technol 6:544–547. https://doi.org/10.1109/JDT.2010.2066546
Oh J, Hong M-C (2022) Low-light image enhancement using hybrid deep-learning and mixed-norm loss functions. Sensors 22(18):6904. https://doi.org/10.3390/s22186904
Perez J, Attanasio AC, Nechyporenko N, Sanz PJ (2018) A deep learning approach for underwater image enhancement. In: Biomedical applications based on natural and artificial computing. IWINAC 2017. Lect Notes Comput Sci pp 7159–7165. https://doi.org/10.1109/ICRA.2018.8460552
Plutino A, Barricelli BR, Casiraghi E, Rizzi A (2021) Scoping review on automatic color equalization algorithm. J Electron Imaging 30:020901. https://doi.org/10.1117/1.JEI.30.2.020901
Pravin SC, Rohith G, Kiruthika V, Manikandan E, Methelesh S, Manoj A (2023) Underwater animal identification and classification using a hybrid classical-quantum algorithm. IEEE Access 11:141902–141914. https://doi.org/10.1109/ACCESS.2023.3343120
Raihan AJ, Abas PE, De Silva LC (2019) Review of underwater image restoration algorithms. IET Image Process 13(10):1587–1596. https://doi.org/10.1049/iet-ipr.2019.0117
Schechner YY, Karpel N (2004) Clear underwater vision. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, CVPR 2004., vol 1, p 2004. https://doi.org/10.1109/CVPR.2004.1315078
Sedlazeck A, Koch R (2011) Perspective and non-perspective camera models in underwater imaging - overview and error analysis. In: Outdoor and large-scale real-world scene analysis. Lect Notes Comput Sci vol 7474, pp 212–242. https://doi.org/10.1007/978-3-642-34091-8_10
Shi S, Wang Z, Shang R, Li Y, Li J, Zhong G, Gu Y (2023) Hybrid quantum-classical convolutional neural network for phytoplankton classification. Frontiers Mar Sci 10. https://doi.org/10.3389/fmars.2023.1158548
Thakur RS, Chatterjee S, Yadav RN, Gupta L (2021) Image de-noising with machine learning: a review. IEEE Access 9:93338–93363. https://doi.org/10.1109/ACCESS.2021.3092425
Weng J, Ahuja N, Huang T (1992) Cresceptron: a self-organizing neural network which grows adaptively. In: Proceedings - 1992 international joint conference on neural networks, IJCNN 1992. Proceedings of the international joint conference on neural networks, pp 576–581. https://doi.org/10.1109/IJCNN.1992.287150
Xavier V, Fenglei H, Jingzheng Y, Haitao Z, Chunhui W (2020) Underwater image processing and object detection based on Deep CNN method. Journal of Sensors 2020. https://doi.org/10.1155/2020/6707328
Yuh J, West M (2001) Underwater robotics. Adv Robot 2001:609–639. https://doi.org/10.1163/156855301317033595
Yushan S, Junhan C, Guocheng Z, Hao X (2019) Mapless motion planning system for an autonomous underwater vehicle using policy gradient-based deep reinforcement learning. J Intell Robot Syst 96:74–80. https://doi.org/10.1007/s10846-019-01004-2
Zhang Y, Lu K, Gao Y, Wang M (2013) NEQR: a novel enhanced quantum representation of digital images. Quantum Inf Process 12:2833–2860. https://doi.org/10.1007/s11128-013-0567-z
Acknowledgements
All the authors acknowledge Mahindra University for funding and support.
Author information
Authors and Affiliations
Contributions
S.W and H.D implemented hybrid quantum algorithms both on surface and on-board. S.W implemented classical algorithms. R.P, S.W, and S.B collected and cleaned the images. S.W, S.U, and J.D had written and reviewed the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Warrier, S.R., Reddy, D.S.H., Bada, S. et al. On-board classification of underwater images using hybrid classical-quantum CNN-based method. Quantum Mach. Intell. 6, 70 (2024). https://doi.org/10.1007/s42484-024-00206-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s42484-024-00206-8