Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Computer Vision Models in Intelligent Aquaculture with Emphasis on Fish Detection and Behavior Analysis: A Review

  • Original Paper
  • Published:
Archives of Computational Methods in Engineering Aims and scope Submit manuscript

Abstract

Intelligence technologies play an important role in increasing product quality and production efficiency in digital aquaculture. Automatic fish detection will contribute to achieving intelligent production and scientific management in precision farming. Due to the availability and ubiquity of modern information technology, such as the internet of things, big data, and camera devices, computer vision techniques, as an essential branch of artificial intelligence, have emerged as a powerful tool for achieving automatic fish detection. At present, it has been widely used in fish species identification, counting, and behavior analysis. Nevertheless, computer vision modeling used for fish detection is riddled with many challenges, such as varies in illumination, low contrast, high noise, fish deformation, frequent occlusion, and dynamic background. Hence, this paper provides a comprehensive review of the computer vision model for fish detection under unique application scenarios. Firstly, the image acquisition system based on 2D and 3D is discussed. Further, many fish detection techniques are categorized as appearance-based, motion-based, and deep learning. In addition, applications of fish detection and public open-source datasets are also presented in the literature. Finally, the prominent findings and the directions of future research are addressed toward the advancement in the aquaculture field throughout the discussion and conclusion section.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Shvets AA, Rakhlin A, Kalinin AA, Iglovikov VI (2019) Automatic instrument segmentation in robot-assisted surgery using deep learning. In: Proceedings of 17th IEEE international conference mach learning application ICMLA 2018, pp 624–628. https://doi.org/10.1109/ICMLA.2018.00100

  2. Wäldchen J, Mäder P (2018) Plant species identification using computer vision techniques: a systematic literature review. Springer, Dordrecht

    MATH  Google Scholar 

  3. Sur C (2019) Survey of deep learning and architectures for visual captioning—transitioning between media and natural languages. Multimed Tools Appl 78:32187–32237. https://doi.org/10.1007/s11042-019-08021-1

    Article  Google Scholar 

  4. Spänig S, Emberger-Klein A, Sowa JP et al (2019) The virtual doctor: an interactive clinical-decision-support system based on deep learning for non-invasive prediction of diabetes. Artif Intell Med 100:101706. https://doi.org/10.1016/j.artmed.2019.101706

    Article  Google Scholar 

  5. Zion B (2012) The use of computer vision technologies in aquaculture—a review. Comput Electron Agric 88:125–132. https://doi.org/10.1016/j.compag.2012.07.010

    Article  Google Scholar 

  6. Saberioon M, Gholizadeh A, Cisar P et al (2017) Application of machine vision systems in aquaculture with emphasis on fish: state-of-the-art and key issues. Rev Aquac 9(4):369–387. https://doi.org/10.1111/raq.12143

    Article  Google Scholar 

  7. Xia C, Fu L, Liu Z et al (2018) Aquatic toxic analysis by monitoring fish behavior using computer vision: a recent progress. J Toxicol. https://doi.org/10.1155/2018/2591924

    Article  Google Scholar 

  8. Zhou C, Xu D, Lin K et al (2018) Intelligent feeding control methods in aquaculture with an emphasis on fish: a review. Rev Aquac 10:975–993. https://doi.org/10.1111/raq.12218

    Article  Google Scholar 

  9. Li D, Hao Y, Duan Y (2019) Nonintrusive methods for biomass estimation in aquaculture with emphasis on fish: a review. Rev Aquac. https://doi.org/10.1111/raq.12388

    Article  Google Scholar 

  10. Spampinato C, Palazzo S, Boom B et al (2014) Understanding fish behavior during typhoon events in real-life underwater environments. Multimed Tools Appl 70:199–236. https://doi.org/10.1007/s11042-012-1101-5

    Article  Google Scholar 

  11. Labao AB, Naval PC (2019) Cascaded deep network systems with linked ensemble components for underwater fish detection in the wild. Ecol Inform 52:103–121. https://doi.org/10.1016/j.ecoinf.2019.05.004

    Article  Google Scholar 

  12. Boom BJ, Huang PX, He J, Fisher RB (2012) Supporting ground-truth annotation of image datasets using clustering. 2012 21st International conference on pattern recognition (ICPR), pp 1542–1545

  13. Cutter G, Stierhoff K, Zeng J (2015) Automated detection of rockfish in unconstrained underwater videos using haar cascades and a new image dataset: labeled fishes in the wild. In: Proceedings of 2015 IEEE winter conference on applied mathematics and computation WACVW 2015, pp 57–62. https://doi.org/10.1109/WACVW.2015.11

  14. Shevchenko V, Eerola T, Kaarna A (2018) Fish detection from low visibility underwater videos. In: 2018 24th 25th International Conference on Pattern Recognition, pp 1971–1976

  15. Meng L, Hirayama T, Oyanagi S (2018) Underwater-drone with panoramic camera for automatic fish recognition based on deep learning. IEEE Access 6:17880–17886. https://doi.org/10.1109/ACCESS.2018.2820326

    Article  Google Scholar 

  16. Mao J, Xiao G, Sheng W et al (2016) Research on realizing the 3D occlusion tracking location method of fish’s school target. Neurocomputing 214:61–79. https://doi.org/10.1016/j.neucom.2016.05.067

    Article  Google Scholar 

  17. Xiao G, Fan WK, Mao JF et al (2017) Research of the fish tracking method with occlusion based on monocular stereo vision. In: Proceedings of 2016 international conference on information system and artificial intelligence ISAI 2016, pp 581–589. https://doi.org/10.1109/ISAI.2016.0129

  18. Mao J, Xiao G, Sheng W et al (2017) A theoretical 2D image model for locating 3D targets. Int J Adv Robot Syst 94:1430–1450. https://doi.org/10.1080/00207160.2016.1199861

    Article  MathSciNet  MATH  Google Scholar 

  19. Butail S, Paley DA (2012) Three-dimensional reconstruction of the fast-start swimming kinematics of densely schooling fish. J R Soc Interface 9:77–88. https://doi.org/10.1098/rsif.2011.0113

    Article  Google Scholar 

  20. Qian Z, Chen YQ (2017) Feature point based 3D tracking of multiple fish from multi-view images. PLoS ONE 12:1–18. https://doi.org/10.1371/journal.pone.0180254

    Article  Google Scholar 

  21. Cheng XE, Du SS, Li HY et al (2018) Obtaining three-dimensional trajectory of multiple fish in water tank via video tracking. Multimed Tools Appl 77:24499–24519. https://doi.org/10.1007/s11042-018-5755-5

    Article  Google Scholar 

  22. Wang SH, Zhao J, Liu X, et al (2017) 3D tracking swimming fish school with learned kinematic model using LSTM network. In: IEEE international conference on acoustics, speech, and signal processing (ICASSP), pp 1068–1072

  23. Pautsina A, Císař P, Štys D et al (2015) Infrared reflection system for indoor 3D tracking of fish. Aquac Eng 69:7–17. https://doi.org/10.1016/j.aquaeng.2015.09.002

    Article  Google Scholar 

  24. Zhou C, Zhang B, Lin K et al (2017) Near-infrared imaging to quantify the feeding behavior of fish in aquaculture. Comput Electron Agric 135:233–241. https://doi.org/10.1016/j.compag.2017.02.013

    Article  Google Scholar 

  25. Lin K, Zhou C, Xu D et al (2018) Three-dimensional location of target fish by monocular infrared imaging sensor based on a L–z correlation model. Infrared Phys Technol 88:106–113. https://doi.org/10.1016/j.infrared.2017.11.002

    Article  Google Scholar 

  26. Duan Y, Stien LH, Thorsen A et al (2015) An automatic counting system for transparent pelagic fish eggs based on computer vision. Aquac Eng 67:8–13. https://doi.org/10.1016/j.aquaeng.2015.05.001

    Article  Google Scholar 

  27. Abe S, Takagi T, Takehara K et al (2017) How many fish in a tank? Constructing an automated fish counting system by using PTV analysis. Proc SPIE 10328:3–7. https://doi.org/10.1117/12.2270627

    Article  Google Scholar 

  28. Salman A, Jalal A, Shafait F et al (2016) Fish species classification in unconstrained underwater environments based on deep learning. Limnol Oceanogr Methods 14:570–585. https://doi.org/10.1002/lom3.10113

    Article  Google Scholar 

  29. Jin L, Liang H (2017) Deep learning for underwater image recognition in small sample size situations. In: Ocean 2017, Aberdeen, pp 1–4. https://doi.org/10.1109/OCEANSE.2017.8084645

  30. Fier R, Albu AB, Hoeberechts M (2015) Automatic fish counting system for noisy deep-sea videos. In: 2014 Ocean, St John’s. https://doi.org/10.1109/OCEANS.2014.7003118

  31. Sharif MH, Galip F, Guler A, Uyaver S (2015) A simple approach to count and track underwater fishes from videos. In: 2015 18th international conference on computer and information technology (ICCIT), pp 347–352

  32. Fouad MMM, Zawbaa HM, El-Bendary N, Hassanien AE (2013) Automatic Nile Tilapia fish classification approach using machine learning techniques. In: 13th International conference on hybrid intelligent systems, HIS 2013, pp 173–178. https://doi.org/10.1109/HIS.2013.6920477

  33. Dhawal RS, Chen L (2016) A copula based method for fish species classification. In: 2016 IEEE international conference on cognitive informatics and cognitive computing, pp 471–478. https://doi.org/10.1109/ICCI-CC.2016.7862079

  34. Joo D, Kwan Y, Song J et al (2013) Identification of Cichlid Fishes from Lake Malawi using computer vision. PLoS ONE 8:e77686. https://doi.org/10.1371/journal.pone.0077686

    Article  Google Scholar 

  35. Kartika DSY, Herumurti D (2017) Koi fish classification based on HSV color space. In: Proceedings of 2016 international conference on information and communications technology ICTS 2016, pp 96–100. https://doi.org/10.1109/ICTS.2016.7910280

  36. Tharwat A, Hemedan AA, Hassanien AE, Gabel T (2018) A biometric-based model for fish species classification. Fish Res 204:324–336. https://doi.org/10.1016/j.fishres.2018.03.008

    Article  Google Scholar 

  37. Chuang MC, Hwang JN, Kuo FF et al (2014) Recognizing live fish species by hierarchical partial classification based on the exponential benefit. IEEE Int Conf Image Process ICIP 2014:5232–5236. https://doi.org/10.1109/ICIP.2014.7026059

    Article  Google Scholar 

  38. Hernández-Serna A, Jiménez-Segura LF (2014) Automatic identification of species with neural networks. PeerJ 2:e563. https://doi.org/10.7717/peerj.563

    Article  Google Scholar 

  39. Alsmadi MK, Omar KB, Noah SA, Almarashdeh I (2010) Fish recognition based on robust features extraction from color texture measurements using back-propagation classifier. J Theor Appl Inf Technol 18:11–18

    Google Scholar 

  40. Hu J, Li D, Duan Q et al (2012) Fish species classification by color, texture and multi-class support vector machine using computer vision. Comput Electron Agric 88:133–140. https://doi.org/10.1016/j.compag.2012.07.008

    Article  Google Scholar 

  41. Huang PX, Boom BJ (2013) Fisher RBBT-AC on CV underwater live fish recognition using a balance-guaranteed optimized tree, pp 422–433

  42. Li L, Hong J (2014) Identification of fish species based on image processing and statistical analysis research. IEEE Int Conf Mechatron Autom 2014:1155–1160. https://doi.org/10.1109/ICMA.2014.6885861

    Article  Google Scholar 

  43. Spampinato C, ChenBurger Y-H, Nadarajan G, Fisher RB (2008) Detecting, tracking and counting fish in low quality unconstrained underwater videos. Image Process

  44. Palazzo S, Kavasidis I, Spampinato C (2013) Covariance based modeling of underwater scenes for fish detection. In: Proceedings of 2013 IEEE international conference on image processing, ICIP 2013, pp 1481–1485. https://doi.org/10.1109/ICIP.2013.6738304

  45. Huang PX (2016) Hierarchical classification system with reject option for live fish recognition. Intell Syst Ref Libr 104:141–159. https://doi.org/10.1007/978-3-319-30208-9_11

    Article  Google Scholar 

  46. Spampinato C, Palazzo S, Giordano D et al (2012) Covariance based fish tracking in real-life underwater environment. In: Proceedings of the international conference on computer vision theory and applications, vol 2, pp 409–414. https://doi.org/10.5220/0003866604090414

  47. Cutter G, Stierhoff K, Zeng J (2015) Automated detection of rockfish in unconstrained underwater videos using haar cascades and a new image dataset: labeled fishes in the wild. In: Proceedings of 2015 IEEE winter conference on applications of computer vision workshops, WACVW 2015, pp 57–62

  48. Ravanbakhsh M, Shortis MR, Shafait F et al (2015) Automated fish detection in underwater images using shape-based level sets. Photogramm Rec 30:46–62. https://doi.org/10.1111/phor.12091

    Article  Google Scholar 

  49. Chuang MC, Hwang JN, Williams K, Towler R (2013) Multiple fish tracking via Viterbi data association for low-frame-rate underwater camera systems. Proc IEEE Int Symp Circuits Syst. https://doi.org/10.1109/ISCAS.2013.6572362

    Article  Google Scholar 

  50. Chuang MC, Hwang JN, Williams K, Towler R (2015) Tracking live fish from low-contrast and low-frame-rate stereo videos. IEEE Trans Circuits Syst Video Technol 25:167–179. https://doi.org/10.1109/TCSVT.2014.2357093

    Article  Google Scholar 

  51. Pinkiewicz TH, Purser GJ, Williams RN (2011) A computer vision system to analyse the swimming behaviour of farmed fish in commercial aquaculture facilities: a case study using cage-held atlantic salmon. Aquac Eng 45:20–27. https://doi.org/10.1016/j.aquaeng.2011.05.002

    Article  Google Scholar 

  52. Fan L, Liu Y (2013) Automate fry counting using computer vision and multi-class least squares support vector machine. Aquaculture 380–383:91–98. https://doi.org/10.1016/j.aquaculture.2012.10.016

    Article  Google Scholar 

  53. Hernández-Ontiveros JM, Inzunza-González E, García-Guerrero EE et al (2018) Development and implementation of a fish counter by using an embedded system. Comput Electron Agric 145:53–62. https://doi.org/10.1016/j.compag.2017.12.023

    Article  Google Scholar 

  54. Li X, Hao J, Qin H, Chen L (2015) Real-time fish localization with binarized normed gradients. In: MTS/IEEE, Ocean 2015, Washington, pp 1–5. https://doi.org/10.23919/OCEANS.2015.7404465

  55. Bogdan A, Thomas D, Vittorio F (2012) Measuring the objectness of image windows. IEEE Trans Pattern Anal Mach Intell 34:2189–2202. https://doi.org/10.1109/TPAMI.2012.28

    Article  Google Scholar 

  56. Uijlings JRR, Van De Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104:154–171. https://doi.org/10.1007/s11263-013-0620-5

    Article  Google Scholar 

  57. Loh BCS, Raman V, Then PHH (2011) First prototype of aquatic tool kit: towards low-cost intelligent larval fish counting in hatcheries. In: Proceedings of IEEE 9th international conference on dependable, autonomic and secure computing DASC 2011, pp 192–195. https://doi.org/10.1109/DASC.2011.53

  58. Spampinato C, Paris S, Lingrand D (2016) Fine-grained object recognition in underwater visual data. Multimed Tools Appl 75:1701–1720. https://doi.org/10.1007/s11042-015-2601-x

    Article  Google Scholar 

  59. Chuang MC, Hwang JN, Ye JH et al (2017) Underwater fish tracking for moving cameras based on deformable multiple kernels. IEEE Trans Syst Man Cybern Syst 47:2467–2477. https://doi.org/10.1109/TSMC.2016.2523943

    Article  Google Scholar 

  60. Kumar S (2015) Simulation based algorithm for tracking fish population in unconstrained underwater. In: International conference on microwave, optical and communication engineering (ICMOCE), pp 361–364

  61. Wang G, Hwang JN, Williams K, Cutter G (2017) Closed-loop tracking-by-detection for ROV-based multiple fish tracking. In: 016 ICPR 2nd workshop on computer vision for analysis of underwater imagery (CVAUI), pp 7–12. https://doi.org/10.1109/CVAUI.2016.17

  62. Li X, Wei Z, Huang L, et al (2018) Real-time underwater fish tracking based on adaptive multi-appearance model. In: 2018 25th IEEE international conference on image processing (ICIP), pp 2710–2714. https://doi.org/10.1109/ICIP.2018.8451469

  63. Wang J, Zhao M, Zou L et al (2019) Fish tracking based on improved TLD algorithm in real-world underwater environment. Mar Technol Soc J 53:80–89

    Article  Google Scholar 

  64. Qian ZM, Cheng XE, Chen YQ (2014) Automatically detect and track multiple fish swimming in shallow water with frequent occlusion. PLoS ONE. https://doi.org/10.1371/journal.pone.0106506

    Article  Google Scholar 

  65. Xu Z, Cheng XE (2017) Zebrafish tracking using convolutional neural networks. Sci Rep 7:1–11. https://doi.org/10.1038/srep42815

    Article  Google Scholar 

  66. Wang SH, Zhao JW, Chen YQ (2017) Robust tracking of fish schools using CNN for head identification. Multimed Tools Appl 76:23679–23697. https://doi.org/10.1007/s11042-016-4045-3

    Article  Google Scholar 

  67. Qian ZM, Wang SH, Cheng XE, Chen YQ (2016) An effective and robust method for tracking multiple fish in video image based on fish head detection. BMC Bioinform 17:1–11. https://doi.org/10.1186/s12859-016-1138-y

    Article  Google Scholar 

  68. Wang SH, Cheng XE, Qian ZM et al (2016) Automated planar tracking the waving bodies of multiple zebrafish swimming in shallow water. PLoS ONE 11:1–24. https://doi.org/10.1371/journal.pone.0154714

    Article  Google Scholar 

  69. Boom BJ, Beauxis-Aussalet E, Hardman L, Fisher RB (2015) Uncertainty-aware estimation of population abundance using machine learning. Multimed Syst. https://doi.org/10.1007/s00530-015-0479-0

    Article  Google Scholar 

  70. Costa C, Scardi M, Vitalini V, Cataudella S (2009) A dual camera system for counting and sizing Northern Blue fin Tuna (Thunnus thynnus; Linnaeus, 1758) stock, during transfer to aquaculture cages, with a semi automatic. Artif Neural Netw Tool Aquac 291:161–167. https://doi.org/10.1016/j.aquaculture.2009.02.013

    Article  Google Scholar 

  71. Zheng X, Zhang Y (2010) A fish population counting method using fuzzy artificial neural network. In: 2010 IEEE international conference on progress in informatics and computing (PIC), IEEE, pp 225–228

  72. Hossain E, Alam SMS, Ali AA, Amin MA (2016) Fish activity tracking and species identification in underwater video. In: 2016 5th International conference on informatics, electronics and vision (ICIEV), pp 62–66. https://doi.org/10.1109/ICIEV.2016.7760189

  73. Jaeger J, Simon M, Denzler J et al (2015) Croatian fish dataset: fine-grained classification of fish species in their natural habitat. Croat Fish Dataset 2015:1–7

    Google Scholar 

  74. Siddiqui SA, Salman A, Malik MI et al (2018) Automatic fish species classification in underwater videos: exploiting pre-trained deep neural network models to compensate for limited labelled data. ICES J Mar Sci 75:374–389. https://doi.org/10.1093/icesjms/fsx109

    Article  Google Scholar 

  75. Sun X, Shi J, Dong J, Wang X (2017) Fish recognition from low-resolution underwater images. In: Proceedings of 2016 9th international congress on image and signal processing, biomedical engineering and informatics, CISP-BMEI 2016, pp 471–476. https://doi.org/10.1109/CISP-BMEI.2016.7852757

  76. Tamou A Ben, Benzinou A, Nasreddine K, Ballihi L (2018) Underwater live fish recognition by deep learning. In: International conference on image and signal processing, Springer, pp 275–283

  77. Sun X, Shi J, Liu L et al (2018) Transferring deep knowledge for object recognition in Low-quality underwater videos. Neurocomputing 275:897–908. https://doi.org/10.1016/j.neucom.2017.09.044

    Article  Google Scholar 

  78. Ben Tamou A, Benzinou A, Nasreddine K, Ballihi L (2018) Transfer learning with deep convolutional neural network for underwater live fish recognition. In: IEEE 3rd international image processing applications and systems, IPAS 2018, pp 204–209. https://doi.org/10.1109/IPAS.2018.8708871

  79. Alsmadi MK, Bin Omar K, Noah SA, Almarashdeh I (2010) Fish recognition based on robust features extraction from size and shape measurements using neural network. J Comput Sci 6:1088–1094. https://doi.org/10.3844/jcssp.2010.1088.1094

    Article  Google Scholar 

  80. Luo S, Li X, Wang D, et al (2016) Automatic fish recognition and counting in video footage of fishery operations. In: Proceedings of 2015 international conference on computational intelligence and communication networks, CICN 2015, pp 296–299. https://doi.org/10.1109/CICN.2015.66

  81. Iscimen B, Kutlu Y, Uyan A, Turan CBT-SP and CAC (2015) Classification of fish species with two dorsal fins using centroid-contour distance. In: 23nd signal processing and communications applications conference (SIU), pp 1981–1984

  82. Kutlu Y, İşçimen B, Turan C (2017) Multi-stage fish classification system using morphometry. Fresenius Environ Bull 26:1910–1916

    Google Scholar 

  83. Deep BV, Dash R (2019) Underwater fish species recognition using deep learning techniques. In: 2019 6th International conference on signal processing and integrated networks, SPIN 2019, pp 665–669. https://doi.org/10.1109/SPIN.2019.8711657

  84. Zhang D, Lee DJ, Zhang M et al (2016) Object recognition algorithm for the automatic identification and removal of invasive fish. Biosyst Eng 145:65–75. https://doi.org/10.1016/j.biosystemseng.2016.02.013

    Article  Google Scholar 

  85. International I, On W, Learning M, Signal FOR (2012) Hidden Markov models for detecting anomalous fish trajectories in underwater footage. In: Spampinato C, Palazzo S (eds) Department of Electrical, Electronics and Computer Engineering University of Catania, Viale Andrea Doria, 6, 95127, Catania, Italy

  86. Hsiao YH, Chen CC, Lin SI, Lin FP (2014) Real-world underwater fish recognition and identification, using sparse representation. Ecol Inform 23:13–21. https://doi.org/10.1016/j.ecoinf.2013.10.002

    Article  Google Scholar 

  87. Chuang M, Hwang J, Williams K (2016) A feature learning and object recognition framework for underwater fish images. IEEE Trans Image Process 25:1862–1872. https://doi.org/10.1109/TIP.2016.2535342

    Article  MathSciNet  MATH  Google Scholar 

  88. Salimi N, Loh KH, Kaur Dhillon S, Chong VC (2016) Fully-automated identification of fish species based on otolith contour: using short-time Fourier transform and discriminant analysis (STFT-DA). PeerJ 4:e1664. https://doi.org/10.7717/peerj.1664

    Article  Google Scholar 

  89. Hasija S, Buragohain MJ, Indu S (2017) Fish species classification using graph embedding discriminant analysis. Int Conf Mach Vis Inf Technol 2017:81–86. https://doi.org/10.1109/CMVIT.2017.23

    Article  Google Scholar 

  90. Lantsova E, Voitiuk T, Zudilova TV, Kaarna A (2016) Using low-quality video sequences for fish detection and tracking. In: Proceedings of SAI computing conference 2016, pp 426–433. https://doi.org/10.1109/SAI.2016.7556017

  91. Shiau YH, Chen CC, Lin SI (2013) Using bounding-surrounding boxes method for fish tracking in real world underwater observation. Int J Adv Robot Syst. https://doi.org/10.5772/56631

    Article  Google Scholar 

  92. Kavasidis I, Palazzo S, Di Salvo R et al (2012) A semi-automatic tool for detection and tracking ground truth generation in videos. In: Proceedings of the 1st international workshop on visual interfaces for ground truth collection in computer vision applications, ACM, p 6

  93. Hsiao YH, Chen CC (2016) A sparse sample collection and representation method using re-weighting and dynamically updating OMP for fish tracking. In: 2016 IEEE international conference on image processing (ICIP) IEEE, pp 3494–3497

  94. Salman A, Maqbool S, Khan AH et al (2019) Real-time fish detection in complex backgrounds using probabilistic background modelling. Ecol Inform 51:44–51. https://doi.org/10.1016/j.ecoinf.2019.02.011

    Article  Google Scholar 

  95. Ye ZY, Zhao J, Han ZY et al (2016) Behavioral characteristics and statistics-based imaging techniques in the assessment and optimization of tilapia feeding in a recirculating aquaculture system. Trans ASABE 59:345–355. https://doi.org/10.13031/trans.59.11406

    Article  Google Scholar 

  96. Zhao J, Gu Z, Shi M et al (2016) Spatial behavioral characteristics and statistics-based kinetic energy modeling in special behaviors detection of a shoal of fish in a recirculating aquaculture system. Comput Electron Agric 127:271–280. https://doi.org/10.1016/j.compag.2016.06.025

    Article  Google Scholar 

  97. Chen G, Sun P, Shang Y (2017) Automatic fish classification system using deep learning. In: 2017 IEEE 29th International conference on tools with artificial intelligence, pp 24–29. https://doi.org/10.1109/ICTAI.2017.00016

  98. Qin H, Li X, Liang J et al (2016) DeepFish: accurate underwater live fish recognition with a deep architecture. Neurocomputing 187:49–58

    Article  Google Scholar 

  99. Rathi D, Jain S, Indu S (2017) Underwater fish species classification using convolutional neural network and deep learning. In: 2017 Ninth international conference on advances in pattern recognition (ICAPR), pp 344–349

  100. Villon S, Mouillot D, Chaumont M et al (2018) A deep learning method for accurate and fast identification of coral reef fishes in underwater images. Ecol Inform 48:238–244. https://doi.org/10.1016/j.ecoinf.2018.09.007

    Article  Google Scholar 

  101. Yamamoto S (2018) Enterprise requirements management knowledge towards digital transformation

  102. Li X, Shang M, Qin H, Chen L (2015) Fast accurate fish detection and recognition of underwater images with fast R-CNN. In: Ocean 2015—MTS/IEEE Washington, pp 1–5. https://doi.org/10.23919/OCEANS.2015.7404464

  103. Xu L, Wei Y, Wang X et al (2019) Binocular vision of fish swarm detection in real-time based on deep learning. In: Ocean 2018 MTS/IEEE Charleston, pp 1–6. https://doi.org/10.1109/OCEANS.2018.8604865

  104. Zhang D, Kopanas G, Desai C et al (2016) Unsupervised underwater fish detection fusing flow and objectiveness. In: 2016 IEEE winter conference on application of computer vision, WACVW 2016. https://doi.org/10.1109/WACVW.2016.7470121

  105. Li X, Shang M, Hao J, Yang Z (2016) Accelerating fish detection and recognition by sharing CNNs with objectness learning. In: Ocean 2016, Shanghai. https://doi.org/10.1109/OCEANSAP.2016.7485476

  106. Jäger J, Rodner E, Denzler J, et al (2016) SeaCLEF 2016: object proposal classification for fish detection in underwater videos. In: CLEF (working notes), pp 481–489

  107. Li X, Tang Y, Gao T (2017) Deep but lightweight neural networks for fish detection. In: Ocean 2017, Aberdeen 2017, pp 1–5. https://doi.org/10.1109/OCEANSE.2017.8084961

  108. Sung M, Yu S (2017) Vision based real-time fish detection using convolutional neural network. In: Oceans Aberdeen conference

  109. Mandal R, Connolly RM, Schlacher TA, Stantic B (2018) Assessing fish abundance from underwater video using deep neural networks. Int Jt Conf Neural Netw 2018:1–6. https://doi.org/10.1109/IJCNN.2018.8489482

    Article  Google Scholar 

  110. Xu W, Matzner S (2018) Underwater fish detection using deep learning for water power applications

  111. Levy D, Belfer Y, Osherov E, et al (2018) Automated analysis of marine video with limited data. In: 2018 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), IEEE, pp 1466–14668

  112. Zhao J, Li YH, Zhang FD et al (2018) Semi-supervised learning-based live fish identification in aquaculture using modified deep convolutional generative adversarial networks. Trans ASABE 61:699–710. https://doi.org/10.13031/trans.12684

    Article  Google Scholar 

  113. Jonas J, Wolff V, Fricke-neuderth K, Mothes O (2017) Visual fish tracking : combining a two-stage graph approach with CNN-features. In: Oceans Aberdeen conference, pp 1–6

  114. Huang R, Lai Y, Tsao C et al (2018) Applying convolutional networks to underwater tracking without training. IEEE Int Conf Appl Syst Invent 2018:342–345. https://doi.org/10.1109/ICASI.2018.8394604

    Article  Google Scholar 

  115. French G, Fisher M, Mackiewicz M, Needle C (2015) Convolutional neural networks for counting fish in fisheries surveillance video. Proc Mach Vis Anim Behav Work 7(1–7):10. https://doi.org/10.5244/C.29.MVAB.7

    Article  Google Scholar 

  116. Liu L, Lu H, Cao Z, Xiao Y (2018) Counting fish in sonar images. In: Liu L, Lu H, Cao Z, Xiao Y (eds) The 25th IEEE international conference on image processing, National Key Laboratory of Science and Technology on multi-spectral information processing school of automation, Huazhong University of Science and Technology, Wuhan , pp 3189–3193

  117. Beyan C, Katsageorgiou V, Fisher RB, Beyan C, Katsageorgiou V, Fisher RB (2018) Extracting statistically significant behaviour from fish tracking data with and without large dataset cleaning. IET Comput Vis 12:162–170. https://doi.org/10.1049/iet-cvi.2016.0462

    Article  Google Scholar 

  118. Zhao J, Bao W, Zhang F et al (2018) Modified motion influence map and recurrent neural network-based monitoring of the local unusual behaviors for fish school in intensive aquaculture. Aquaculture 493:165–175. https://doi.org/10.1016/j.aquaculture.2018.04.064

    Article  Google Scholar 

  119. Borland HP, Schlacher TA, Gilby BL et al (2017) Habitat type and beach exposure shape fish assemblages in the surf zones of ocean beaches. Mar Ecol Prog Ser 570:203–211. https://doi.org/10.3354/meps12115

    Article  Google Scholar 

  120. Gilby BL, Olds AD, Connolly RM et al (2017) Estuarine, Coastal and Shelf Science Umbrellas can work under water: using threatened species as indicator and management surrogates can improve coastal conservation. Estuar Coast Shelf Sci 199:132–140. https://doi.org/10.1016/j.ecss.2017.10.003

    Article  Google Scholar 

  121. Le J, Xu L (2017) An automated fish counting algorithm in aquaculture based on image processing. Adv Eng Res 113:358–366. https://doi.org/10.2991/ifmca-16.2017.56

    Article  Google Scholar 

  122. Spampinato C, Beauxis-Aussalet E, Palazzo S et al (2014) A rule-based event detection system for real-life underwater domain. Mach Vis Appl 25:99–117. https://doi.org/10.1007/s00138-013-0509-x

    Article  Google Scholar 

  123. Huang PX (2016) Hierarchical classification system with reject option for live fish recognition. Int J Adv Robot Syst 104:141–159. https://doi.org/10.1007/978-3-319-30208-9_11

    Article  Google Scholar 

  124. Lee JHLJH, Wu MYWMY, Guo ZCGZC (2010) A tank fish recognition and tracking system using computer vision techniques. In: 2010 3rd IEEE international conference on computer science and information technology, ICCSIT, vol 4, pp 528–532. https://doi.org/10.1109/ICCSIT.2010.5563625

  125. Shafait F, Mian A, Shortis M et al (2016) Fish identification from videos captured in uncontrolled underwater environments. ICES J Mar Sci J Cons 73:2737–2746. https://doi.org/10.1093/icesjms/fsw106

    Article  Google Scholar 

  126. Ishaq O, Sadanandan SK, Wählby C (2017) Deep fish: deep learning–based classification of zebrafish deformation for high-throughput screening. SLAS Discov 22:102–107. https://doi.org/10.1177/1087057116667894

    Article  Google Scholar 

  127. Wei G, Wei Z, Huang L et al (2018) Robust underwater fish classification based on data augmentation by adding noises in random local regions. In: Pacific Rim conference on multimedia, Springer, pp 509–518

  128. Liawatimena S, Heryadi Y, Lukas et al (2019) A fish classification on images using transfer learning and matlab. In: 1st 2018 Indonesian association for pattern recognition international conference (INAPR), Ina 2018, pp 108–112. https://doi.org/10.1109/INAPR.2018.8627007

  129. Fabic JN, Turla IE, Capacillo JA et al (2013) Fish population estimation and species classification from underwater video sequences using blob counting and shape analysis. In: 2013 IEEE international underwater technology symposium (UT), pp 1–6

  130. Han J, Honda N, Asada A, Shibata K (2009) Automated acoustic method for farmed fish counting and sizing during its transfer using DIDSON. Image (Rochester, NY). https://doi.org/10.1007/s12562-009-0162-5

    Article  Google Scholar 

  131. Garófano-Gómez V, Martínez-Capel F, Peredo-Parada M et al (2011) Assessing hydromorphological and floristic patterns along a regulated Mediterranean river: the Serpis River (Spain). Limnetica 30:307–328. https://doi.org/10.1002/rra

    Article  Google Scholar 

  132. Labuguen RT, Volante EJP, Causo A et al (2012) Automated fish fry counting and schooling behavior analysis using computer vision. In: Proceedings of 2012 IEEE 8th international colloquium on signal processing and its applications, CSPA 2012, pp 255–260. https://doi.org/10.1109/CSPA.2012.6194729

  133. Spampinato C, Chen-Burger Y-H, Nadarajan G, Fisher RB (2008) Detecting, tracking and counting fish in low quality unconstrained underwater videos. Image Process

  134. Zheng X, Zhang Y (2010) A fish population counting method using fuzzy artificial neural network. In: 2010 IEEE international conference on progress in informatics and computing (PIC), pp 225–228

  135. Kristiansen TS, Fernö A, Holm JC et al (2004) Swimming behaviour as an indicator of low growth rate and impaired welfare in Atlantic halibut (Hippoglossus hippoglossus L.) reared at three stocking densities. Aquaculture 230:137–151. https://doi.org/10.1016/S0044-8486(03)00436-8

    Article  Google Scholar 

  136. Xiao G, Zhang W, Zhang Y et al (2011) Online monitoring system of fish behavior. In: 2011 11th international conference on control, automation and systems (ICCAS), pp 1309–1312

  137. Sakakibara J, Nakagawa M, Yoshida M (2004) Stereo-PIV study of flow around a maneuvering fish. Exp Fluids 36:282–293. https://doi.org/10.1007/s00348-003-0720-z

    Article  Google Scholar 

  138. Wang L-S, Wang L, Wang L et al (2009) Effect of 1-butyl-3-methylimidazolium tetrafluoroborate on the wheat (Triticum aestivum L.) seedlings. Environ Toxicol 24:296–303. https://doi.org/10.1002/tox

    Article  Google Scholar 

  139. Papadakis VM, Papadakis IE, Lamprianidou F et al (2012) A computer-vision system and methodology for the analysis of fish behavior. Aquac Eng 46:53–59. https://doi.org/10.1016/j.aquaeng.2011.11.002

    Article  Google Scholar 

  140. Kane AS, Salierno JD, Gipson GT et al (2004) A video-based movement analysis system to quantify behavioral stress responses of fish. Water Res 38:3993–4001. https://doi.org/10.1016/j.watres.2004.06.028

    Article  Google Scholar 

  141. Xu J, Liu Y, Cui S, Miao X (2006) Behavioral responses of tilapia (Oreochromis niloticus) to acute fluctuations in dissolved oxygen levels as monitored by computer vision. Aquac Eng 35:207–217

    Article  Google Scholar 

  142. Xiao G, Feng M, Cheng Z et al (2015) Water quality monitoring using abnormal tail-beat frequency of crucian carp. Ecotoxicol Environ Saf 111:185–191. https://doi.org/10.1016/j.ecoenv.2014.09.028

    Article  Google Scholar 

  143. Papadakis VM, Glaropoulos A, Kentouri M (2014) Sub-second analysis of fish behavior using a novel computer-vision system. Aquac Eng 62:36–41. https://doi.org/10.1016/j.aquaeng.2014.06.003

    Article  Google Scholar 

  144. Beyan C, Fisher RB (2013) Detecting abnormal fish trajectories using clustered and labeled data. In: 2013 IEEE international conference on image processing, ICIP 2013, pp 1476–1480. https://doi.org/10.1109/ICIP.2013.6738303

  145. Beyan C, Fisher R (2013) Detection of abnormal fish trajectories using a clustering based hierarchical classifier. In: BMVC, Bristol, UK, pp 1–11. https://doi.org/10.5244/C.27.21

  146. Israeli D (1996) Monitoring the behavior of hypoxia-stressed Carassius auratus using computer vision. Aquac Eng 15:423–440. https://doi.org/10.1016/S0144-8609(96)01009-6

    Article  Google Scholar 

  147. Barry MJ (2012) Application of a novel open-source program for measuring the effects of toxicants on the swimming behavior of large groups of unmarked fish. Chemosphere 86:938–944. https://doi.org/10.1016/j.chemosphere.2011.11.011

    Article  Google Scholar 

  148. Xiao G, Fan W (2018) Advances in image and graphics technologies. In: 10th Chinese conference on image and graphics technologies and applications, vol 757, pp 18–27. https://doi.org/10.1007/978-981-10-7389-2

  149. Duarte S, Reig L, Oca J (2009) Measurement of sole activity by digital image analysis. Aquac Eng 41:22–27. https://doi.org/10.1016/j.aquaeng.2009.06.001

    Article  Google Scholar 

  150. Sadoul B, Evouna Mengues P, Friggens NC et al (2014) A new method for measuring group behaviours of fish shoals from recorded videos taken in near aquaculture conditions. Aquaculture 430:179–187. https://doi.org/10.1016/j.aquaculture.2014.04.008

    Article  Google Scholar 

  151. Liu Z, Li X, Fan L et al (2014) Measuring feeding activity of fish in RAS using computer vision. Aquac Eng 60:20–27. https://doi.org/10.1016/j.aquaeng.2014.03.005

    Article  Google Scholar 

  152. Chew BF, Eng H, Thida M (2009) Vision-based real-time monitoring on the behavior of fish school, pp 90–93

  153. Xiao G, Fan W, Mao J et al (2015) Advances in image and graphics technologies, pp 18–27. https://doi.org/10.1007/978-3-662-47791-5

  154. Zhou C, Lin K, Xu D et al (2018) Near infrared computer vision and neuro-fuzzy model-based feeding decision system for fish in aquaculture. Comput Electron Agric 146:114–124. https://doi.org/10.1016/j.compag.2018.02.006

    Article  Google Scholar 

  155. Zhou C, Xu D, Chen L et al (2019) Evaluation of fish feeding intensity in aquaculture using a convolutional neural network and machine vision. Aquaculture 507:457–465. https://doi.org/10.1016/j.aquaculture.2019.04.056

    Article  Google Scholar 

  156. Jager J, Wolff V, Fricke-Neuderth K et al (2017) Visual fish tracking: combining a two-stage graph approach with CNN-features. In: OCEANS 2017-Aberdeen, pp 1–6. https://doi.org/10.1109/OCEANSE.2017.8084691

  157. Miyazono T, Saitoh T (2018) Fish species recognition based on CNN using annotated image. In: iCatse international conference on IT convergence and security (ICITCS), pp 156–163

  158. Allken V, Handegard NO, Rosen S et al (2019) Fish species identification using a convolutional neural network trained on synthetic data. ICES J Mar Sci 76:342–349. https://doi.org/10.1093/icesjms/fsy147

    Article  Google Scholar 

  159. Zhao J, Bao WJ, Zhang FD et al (2017) Assessing appetite of the swimming fish based on spontaneous collective behaviors in a recirculating aquaculture system. Aquac Eng 78:196–204. https://doi.org/10.1016/j.aquaeng.2017.07.008

    Article  Google Scholar 

  160. Dhawal RS, Chen L (2018) A biometric-based model for fish species classification. Fish Res 204:324–336. https://doi.org/10.1016/j.fishres.2018.03.008

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China “Next generation precision aquaculture: R&D on intelligent measurement, control and equipment technologies” (No. 2017YFE0122100), and the Key R&D Program “Research and development of intelligent model and precise monitoring of shrimp processing” (No. 2018YFD0700904-2).

Author information

Authors and Affiliations

  1. College of Information and Electrical Engineering, China Agricultural University, 17 Tsinghua East Road, Beijing, 100083, People’s Republic of China

    Ling Yang, Yeqi Liu, Xiaomin Fang, Lihua Song, Daoliang Li & Yingyi Chen

  2. National Innovation Center for Digital Fishery, China Agricultural University, 17 Tsinghua East Road, Beijing, 100083, People’s Republic of China

    Ling Yang, Yeqi Liu, Xiaomin Fang, Lihua Song, Daoliang Li & Yingyi Chen

  3. Beijing Engineering and Technology Research Centre for the Internet of Things in Agriculture, China Agricultural University, 17 Tsinghua East Road, Beijing, 100083, People’s Republic of China

    Ling Yang, Yeqi Liu, Lihua Song & Yingyi Chen

  4. Precision Agricultural Technology Integration Research Base (Fishery), Ministry of Agriculture and Rural Affairs, China Agricultural University, 17 Tsinghua East Road, Beijing, 100083, People’s Republic of China

    Xiaomin Fang & Daoliang Li

  5. School of Information Science and Technology, Beijing Forestry University, Beijing, 100083, People’s Republic of China

    Huihui Yu

Authors
  1. Ling Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yeqi Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Huihui Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xiaomin Fang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Lihua Song
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Daoliang Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Yingyi Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yingyi Chen.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Liu, Y., Yu, H. et al. Computer Vision Models in Intelligent Aquaculture with Emphasis on Fish Detection and Behavior Analysis: A Review. Arch Computat Methods Eng 28, 2785–2816 (2021). https://doi.org/10.1007/s11831-020-09486-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11831-020-09486-2