Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

A coastal obstacle detection framework of dual USVs based on dual-view color fusion

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Distinguishing waves from obstacles in challenging coastal environments with backlight and low light conditions is crucial for obstacle detection of unmanned surface vehicles (USVs). We propose a dual-view color fusion framework utilizes cooperative dual USVs for obstacle detection. The framework leverages the saliency of the object in dual views relative to the background to determine whether the object is an obstacle. To achieve this, we first perform spatiotemporal calibration of multi-sensors for dual USVs to accurately extract images of the Lidar-detected objects and background in dual-view. Then, we design a feature representation method that fuses dual-view information to capture the saliency of the object in dual-view and the difference in illumination between the dual views. Finally, we train a capped L1-norm twin support vector machine using experimental data to classify the object. Experimental results in two sea areas demonstrate that the proposed framework can enhance obstacle detection performance by effectively distinguishing waves from obstacles, and is generalizable to different types, sizes, and colors of obstacles in various sea areas. Moreover, the proposed framework achieves a high accuracy rate while maintaining a good balance between missed alarms and false alarms, outperforming other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Madeo, D., Pozzebon, A., Mocenni, C., Bertoni, D.: A low-cost unmanned surface vehicle for pervasive water quality monitoring. IEEE Trans. Instrum. Meas. 69, 1433–1444 (2020)

    Article  Google Scholar 

  2. Jimenez, J.F., Giron-Sierra, J.M.: USV based automatic deployment of booms along quayside mooring ships: scaled experiments and simulations. Ocean Eng. 207, 107438 (2020)

    Article  Google Scholar 

  3. Gu, Y.W., Goez, J.C., Guajardo, M., Wallace, S.W.: Autonomous vessels: state of the art and potential opportunities in logistics. Int. Trans. Oper. Res. 28, 1706–1739 (2021)

    Article  MathSciNet  Google Scholar 

  4. Bovcon, B., Muhovic, J., Vranac, D., Mozetic, D., Pers, J., Kristan, M.: MODS-A USV-oriented object detection and obstacle segmentation benchmark. IEEE Trans. Intell. Transp. Syst. 23, 13403–13418 (2022)

    Article  Google Scholar 

  5. Yang, P., Dong, L.L., Xu, W.H.: Detecting small infrared maritime targets overwhelmed in heavy waves by weighted multidirectional gradient measure. IEEE Geosci. Remote Sens. Lett. 19, 702005 (2022)

    Google Scholar 

  6. Sravanthi, R., Sarma, A.S.V.: Efficient image-based object detection for floating weed collection with low cost unmanned floating vehicles. Soft Comput. 25, 13093–13101 (2021)

    Article  Google Scholar 

  7. Paccaud, P., Barry, D.A.: Obstacle detection for lake-deployed autonomous surface vehicles using RGB imagery. PloS ONE 13, e0205319 (2018)

    Article  Google Scholar 

  8. Jin, J.C., Zhang, J., Liu, D.Q., Shi, J.N., Wang, D., Li, F.X.: Vision-based target tracking for unmanned surface vehicle considering its motion features. IEEE Access 8, 132655–132664 (2020)

    Article  Google Scholar 

  9. Shi, J.N., Jin, J.C., Zhang, J.: Object detection based on saliency and sea-sky line for USV vision. In: IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 1581–1586 (2018)

  10. Lin, C., Chen, W., Zhou, H.F.: Multi-visual feature saliency detection for sea-surface targets through improved sea-sky-line detection. J. Mar. Sci. Eng. 8, 799 (2020)

    Article  MathSciNet  Google Scholar 

  11. Liu, J.Y., Li, H.Y., Liu, J., Xie, S.R., Luo, J.: Real-time monocular obstacle detection based on horizon line and saliency estimation for unmanned surface vehicles. Mob. Netw. Appl. 26, 1372–1385 (2021)

    Article  Google Scholar 

  12. Zhang, W., Jiang, F., Yang, C.F., Wang, Z.P., Zhao, T.J.: Research on unmanned surface vehicles environment perception based on the fusion of vision and Lidar. IEEE Access 9, 63107–63121 (2021)

    Article  Google Scholar 

  13. Zhang, W., Gao, X.Z., Yang, C.F., Jiang, F., Chen, Z.Y.: A object detection and tracking method for security in intelligence of unmanned surface vehicles. J Ambient Intell. Humaniz. Comput. 13, 1279–1291 (2020)

    Article  Google Scholar 

  14. Li, Y., Guo, J.H., Guo, X.M., Liu, K.Z., Zhao, W.T., Luo, Y.T., Wang, Z.Y.: A novel target detection method of the unmanned surface vehicle under all-weather conditions with an improved YOLOV3. Sensors 20, 4885 (2020)

    Article  Google Scholar 

  15. Fu, H.X., Song, G.Q., Wang, Y.C.: Improved YOLOv4 marine target detection combined with CBAM. Symmetry 13, 623 (2021)

    Article  Google Scholar 

  16. Liu, T., Pang, B., Zhang, L., Yang, W., Sun, X.Q.: Sea surface object detection algorithm based on YOLO v4 fused with reverse depthwise separable convolution (RDSC) for USV. J. Mar. Sci. Eng. 9, 753 (2021)

    Article  Google Scholar 

  17. Sun, X.Q., Liu, T., Yu, X.P., Pang, B.: Unmanned surface vessel visual object detection under all-weather conditions with optimized feature fusion network in YOLOv4. J. Intell. Robot. Syst. 103, 55 (2021)

    Article  Google Scholar 

  18. Muhovic, J., Mandeljc, R., Bovcon, B., Kristan, M., Pers, J.: Obstacle tracking for unmanned surface vessels using 3-D point cloud. IEEE J. Ocean. Eng. 45, 786–798 (2020)

    Article  Google Scholar 

  19. Lee, S.J., Moon, Y.S., Ko, N.Y., Choi, H.T., Lee, J.M.: A method for object detection using point cloud measurement in the sea environment. In: IEEE OES International Symposium on Underwater Technology (UT), pp.1–4 (2017).

  20. Zhang, X.L., Wang, H.S., Chen, W.D.: Vessel detection and classification fusing radar and vision data. In: 7th International Conference on Information Science and Technology (ICIST), pp.474–479 (2017).

  21. Wang, S.W., Wu, C., Xie, L., Xu, K., Ma, F.: Research of obstacle recognition method for USV based on laser radar. In: 4th International Conference on Transportation Information and Safety (ICTIS), pp. 343–348 (2017)

  22. Zhang, W., Yang, C.F., Jiang, F., Gao, X.Z., Yang, K.: A water surface moving target detection based on information fusion using deep learning. In: 2020 International Conference on 5G Mobile Communications and Information Science, pp. 1–7 (2020)

  23. Chen, J.Y., Wang, H.: An obstacle detection method for USV by fusing of radar and motion stereo. In: 16th IEEE International Conference on Control and Automation (ICCA), pp. 159–164 (2020).

  24. Haghbayan, M.H., Farahnakian, F., Poikonen, J., Laurinen, M., Nevalainen, P., Plosila, J., Heikkonen, J.: An efficient multi-sensor fusion approach for object detection in maritime environments. In: 21st IEEE International Conference on Intelligent Transportation System (ITSC), pp. 2163–2170 (2018)

  25. Liu, D.Q., Zhang, J., Jin, J.C., Dai, Y.S., Li, L.G.: A new approach of obstacle fusion detection for unmanned surface vehicle using Dempster-Shafer evidence theory. Appl. Ocean Res. 119, 103016 (2022)

    Article  Google Scholar 

  26. Wu, Y.Y., Qin, H.C., Liu, T., Liu, H., Wei, Z.Q.: A 3D object detection based on multi-modality sensors of USV. Appl. Sci. Basel 9, 535 (2019)

    Article  Google Scholar 

  27. Liu, H., Nie, J., Liu, Y.J., Wu, Y.Y., Wang, H.X., Qu, F.C., Liu, W., Li, Y.Y.: A multi-modality sensor system for unmanned surface vehicle. Neural Process. Lett. 52, 977–992 (2020)

    Article  Google Scholar 

  28. Wang, C.Y., Ye, Q.L., Luo, P., Ye, N., Fu, L.Y.: Robust capped L1-norm twin support vector machine. Neural Netw. 114, 47–59 (2019)

    Article  MATH  Google Scholar 

  29. Nunes, D., Fortuna, J., Damas, B., Ventura, R.: Real-time vision based obstacle detection in maritime environments. In: IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 243–248 (2022)

  30. Gonzalez-Garcia, A., Collado-Gonzalez, I., Cuan-Urquizo, R., Sotelo, C., Sotelo, D., Castaneda, H.: Path-following and LiDAR-based obstacle avoidance via NMPC for an autonomous surface vehicle. Ocean Eng. 266, 112900 (2022)

    Article  Google Scholar 

Download references

Funding

National Natural Science Foundation of China (42274159); Independent Innovation Technology Special Project of China University of Petroleum (22CX01004A-2); National Key Research and Development Program of China (2017YFC1405203).

Author information

Authors and Affiliations

  1. College of Control Science and Engineering, China University of Petroleum (East China), Qingdao, 266580, China

    Zehao He

  2. College of Oceanography and Space Informatics, China University of Petroleum (East China), Qingdao, 266580, China

    Yongshou Dai, Ligang Li & Hongbin Xu

  3. First Institute of Oceanography, Ministry of Natural Resources, Qingdao, 266061, China

    Jiucai Jin & Deqing Liu

Authors
  1. Zehao He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongshou Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ligang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongbin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiucai Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Deqing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HE: Conceptualization, Methodology, Writing–original draft. DAI: Supervision, Project administration. LI: Investigation, Software, Writing—review and editing. XU: Data curation, JIN: Validation. LIU: Resources.

Corresponding author

Correspondence to Yongshou Dai.

Ethics declarations

Conflict of interest

The authors declare that there are no conflicts of interest related to this article.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, Z., Dai, Y., Li, L. et al. A coastal obstacle detection framework of dual USVs based on dual-view color fusion. SIViP 17, 3883–3892 (2023). https://doi.org/10.1007/s11760-023-02617-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02617-9

Keywords

Search

Navigation