1. Introduction
Occupying a large part of the earth’s area, the ocean is an important source of oil, natural gas, and mineral resources, attracting extensive attention from some adventurers and researchers, which inevitably leads to an increasing number of ocean exploration activities [
1]. As an important technology for marine exploration activities, underwater target detection is widely used in archaeology, marine environment monitoring, underwater navigation, fish farming, and other fields, and has received continuous attention [
2,
3,
4,
5,
6,
7]. However, the harsh marine environment makes underwater target detection still face various challenges.
The main challenges we are currently facing in underwater target detection are as follows: first, we have found from existing underwater datasets and images from real applications for which wavelength-dependent absorption and scattering [
8] degrade the quality of underwater images, causing visibility, weak contrast, and color variations, which make the underwater targets blurry. Then, because some underwater organisms have swarming habits and prefer to attach to objects such as mud, sand, and coral reefs, this causes the underwater target and the background area to obscure each other, making the detection accuracy suffer. Finally, because some embedded devices have limited computational power, which leads to the fact that some network models with large parameter counts cannot be applied to underwater embedded devices, so people are also actively pursuing underwater detection devices that can be applied to large parameters [
9].
In recent years, underwater target detection methods based on deep learning have been widely studied and gradually applied in the field of underwater biological detection. Deep learning-based underwater target detection is based on learning features iteratively, taking the output of each layer as the input of the next layer, and transforming the detailed features of the lower layer into the detailed features of the higher layer through nonlinear mapping between the lower layer and the higher layer. However, due to the relatively small dataset for underwater object detection, the available learning features are limited, and most underwater organisms occur in groups and are relatively small and fuzzy. Current deep learning-based detection algorithms are unable to effectively detect small objects, which poses a great challenge to the target detection task. To cope with this challenge, the YOLO series of algorithms now exist, including YOLOv1 [
10], YOLO9000 [
11], YOLOv3 [
12], YOLOv4 [
13], YOLOv5 [
14], YOLOv6 [
15], YOLOv7 [
16], YOLOv8 [
17], etc. The YOLO (You Only Look Once) series comprises typical single-stage target detection algorithms, which have faster detection speed and wider application scope than two-stage target detection algorithms such as RCNN [
18], Fast-RCNN [
19], Faster RCNN [
20], Mask-RCNN [
21], Cascade-RCNN [
22], and SSD [
23]. In order to adapt to different application scenarios and improve the detection accuracy for different detection objects, many researchers have successively proposed a variety of improved detection methods based on the YOLO series of algorithms. The literature [
24] has proposed an underwater target detection algorithm, YOLOv5s-CA, based on an improved YOLOv5, to improve the detection accuracy by embedding a Coordinate Attention (CA) module and a Squeeze-and-Excitation (SE) module. The literature [
25] also proposes a lightweight algorithm based on an improved YOLOv4, aiming to make the model have a smaller number of parameters and smaller size by combining MobileNetv2 and depth-separable convolution.
However, the target size of the underwater target detection studied in this paper is relatively small, and the previous YOLOv3, YOLOv4, and YOLOv5 algorithms are less effective at detecting small targets, and only the YOLOv7 algorithm has a better detection performance in small target detection at present. Therefore, for the different characteristics of underwater organisms, an underwater target detection algorithm based on the improved YOLOv7 is proposed, with the following four main improvements:
- (1)
The K-Means algorithm is used to re-cluster the underwater target dataset, and the anchor frames produced by the clustering are more in line with the smaller size of the underwater targets studied in this paper, which accelerates the convergence speed of the model and improves the detection accuracy of small-sized targets.
- (2)
Pconv (Partial Convolution) is introduced to optimize the multi-scale feature fusion module, which reduces the parameter count and computational workload while improving the detection speed.
- (3)
The ShapeIou_NWD loss function is used instead of the CIou loss function, which effectively solves the problem of slow convergence and reduced prediction accuracy during the training process by ignoring the shape and scale of the bounding box itself, as well as the possibility of detecting targets that are too small.
- (4)
Introducing the SimAM attention mechanism after the multi-scale feature fusion module enhances the detection accuracy of the network model without parameter increase.
Finally, several comparative experiments and ablation experiments are conducted to prove the effectiveness of the improved algorithm in this paper.
2. Materials and Methods
2.1. YOLOv7 Model
Compared with its variants YOLOv7-D6, YOLOv7-E6, YOLOv7-E6E, YOLOv7-W6, and YOLOv7-X, YOLOv7 has the smallest number of parameters, the fastest detection speed, and is most suitable to be applied to underwater target detection. Therefore, YOLOv7 is selected as the base algorithm to be improved in this paper. The network structure of YOLOv7 is shown in
Figure 1.
As can be seen from
Figure 1, the network structure of YOLOv7 consists of three parts: Input, Backbone, and Head. Input scales the image to be detected to a fixed size after inputting the image to fulfill the input demands of the Backbone, and then carries out the feature extraction through the Backbone, which contains three structures: CBS (Conv + BN + SiLU), ELAN_1, and MP. The CBS mainly uses convolutional layers for feature extraction, and ELAN_1, as a kind of efficient layer aggregation network, greatly improves the learning ability of the detected objects and enriches the diverse feature learning. The MP structure adds a MaxPool layer on top of the CBS, and the upper and lower layers are contacted. The MP structure adds the MaxPool layer on top of the CBS, which greatly improves the feature extraction ability by contacting the upper and lower layers for feature fusion of the features extracted from the branches. In the Head part, the SPPCSPC pyramid structure is employed to broaden the receptive field and make the Head network suitable for multi-size inputs, and then the bottom information is passed upwards from the bottom to the top through the pyramid network structure, so as to realize the fusion of feature information of different scales, and after fusing semantic information and spatial information at multiple scales, three target feature layers of different scales are outputted, which are respectively reused through the RepConv structure. The three obtained feature layers are reparametrized by the RepConv structure to adjust the number of channels, then the preliminary prediction results are obtained by the YOLO Head, and finally the preliminary prediction results are subjected to some post-processing operations, such as confidence filtering, NMS (Non-Maximum Suppression), and so on, in order to obtain the final results.
2.2. Kmeans
The a priori frame sizes used in YOLOv7 were obtained by clustering on the MS COCO dataset using the K-Means [
26] algorithm. The MS COCO dataset contains about 41% small targets (area < 32 × 32), and the rest are medium vs. large targets. However, in the marine life dataset, marine organisms are predominantly small- to medium-sized targets, which is significantly different from the size of the objects in the COCO dataset. Therefore, the original anchor frame data size is not suitable for the dataset in this paper. In order to improve the matching probability between the underwater targets and anchor frames, the anchor frame size was redesigned using the K-means clustering algorithm. The basic process is described as follows:
Randomly select k samples from the dataset as initial clustering centers .
For each sample in the dataset, calculate its distance from the K cluster centers and assign it to the class corresponding to the cluster center with the smallest distance.
For each category , recalculate its clustering center (i.e., the center of mass of all samples belonging to the class).
Repeat steps 2 and 3 until the position of the clustering center no longer changes.
Compared with the K-Means++ [
27] algorithm, the K-Means algorithm is simple in principle and easy to implement, only one hyperparameter k needs to be adjusted, and the K-Means++ algorithm relies on the already selected centroids each time the next centroids are selected.
Therefore, in this paper, the K-Means algorithm is selected to recluster the anchor frames. The anchor frames after reclustering using K-Means are presented in
Table 1 below:
2.3. ELAN_PC
ELAN [
28], mainly composed of VoVNet [
29] and CSPNet [
30], is an efficient layer aggregation network. It facilitates the network to acquire more features by regulating the shortest and longest gradient pathways. ELAN is mainly composed of two branches, the first branch mainly doing the change of channel number through the convolution of 1 × 1, and the second branch firstly doing the change of channel number through the convolution of 1 × 1, then doing the feature extraction through the four convolutional modules of 3 × 3, and finally superimposing the results of the two branches together. The ELAN can then effectively alleviate the problem of gradient disappearance when the model reaches a certain depth. However, the optimization of the ELAN network in terms of the number of parameters and the amount of computation is not ideal, so in this paper, while ensuring the structural integrity of the ELAN network, PConv is introduced to construct the ELAN_PC_1 and ELAN_PC_2 network modules, which use PConv to replace the convolution kernel of 3 × 3 convolutional layers in the ELAN network. The ELAN_PC_1 and ELAN_PC_2 network structures are shown in
Figure 2.
PConv [
31] is a new lightweight convolution module that reduces computational redundancy while reducing memory accesses. In this module, we employ a convolutional operation on a subset of the input channels to avoid excessive redundancy in the feature map. This approach, illustrated in
Figure 3, ensures that the convolutional operation is applied to a select set of channels while the remaining channels remain unchanged.
For continuous or regular memory accesses, the input or output channels are computed as representative of the entire feature map. The PConv is computed as , while the regular convolution is computed as , the memory accesses for the PConv are , and the memory accesses for the regular convolution are . If , then the PConv is computed as 1/16 of the regular convolution, and the memory accesses are 1/4.
It can be seen that the introduction of the PConv convolution module into the ELAN module can significantly reduce the amount of computation and memory access, thus making the model lightweight and speeding up the inference.
2.4. ShapeIoU_NWD
The YOLOv7 model employs the CIOU [
32] loss function, which is designed to account for the overlap area, centroid distance, and aspect ratio of the bounding box regression. However, this loss function does not consider the influence of the inherent properties of the bounding box itself, such as shape and scale. Consequently, the CIOU loss function may prevent the model from optimizing the similarity when the aspect ratio is the same [
33]. To solve this problem, this paper adopts ShapeIoU [
34], a bounding box regression method that focuses on the shape and scale of the bounding box itself. This method can calculate the loss by focusing on the shape and scale of the bounding box itself, which makes the bounding box regression more accurate. The parameter schematic of ShapeIoU is presented in
Figure 4.
where
represents the prediction frame,
represents the real frame,
and
represent the width and height of the real frame respectively, scale is the scale factor, which is related to the scale of the target in the dataset,
and
are the weight coefficients in the horizontal and vertical directions, respectively, and their values are related to the shape of the real frame. Its corresponding bounding box regression loss is as follows:
In order to further improve the accuracy of underwater small target detection, we have introduced the Normalized Wasserstein Distance [
35]. In this approach, the Gaussian distributions
and
, modeled for the two bounding boxes
and
, can be expressed as follows:
However, since
is a distance measure and not a similarity measure, it cannot be directly used as such. Therefore, in order to address this limitation, it is necessary to normalize the distance by transforming
into a value between 0 and 1. This process results in the normalized Gaussian Wasserstein distance (NWD), which enables more effective and meaningful comparisons and evaluations in the context of detecting small underwater objects. The formula is as follows:
Here, is a constant related to the dataset.
Therefore, the loss of our
can be obtained by taking the form of a weighted summation based on the introduced
loss function and
metric with the following formula:
where r is a scaling factor.
In this paper, we use the ShapeIoU_NWD loss function instead of the CIOU loss function, which takes into account both the effects of changes in the shape and scale of the bounding box itself, as well as the sensitivity of small target displacements, and provides a significant improvement in the detection accuracy of small- and medium-sized targets.
2.5. SimAM
The size of underwater targets is relatively small, and because of the presence of some mud, sand, and some other interfering background information, it occupies less information in the whole image. In order to improve the accuracy of small target detection and reduce the interference of background information, this paper adopts the attention mechanism to adaptively focus on the detail information related to the small target, and reduces the attention to other interference information. In contrast to the SE [
36] (Squeeze-and-Excitation) channel attention mechanism, which solely considers the internal channel information, and the CBAM [
37] (Convolutional Block Attention Module), which focuses on the local spatial location range, the SimAM [
38] attention mechanism does not introduce additional network parameters for feature maps, thereby deriving 3D attention weights. Furthermore, it is a plug-and-play feature that can be integrated into any position within the model. Consequently, in this paper, three SimAM attention mechanisms are incorporated after the multi-scale feature fusion module with the objective of enhancing the detection accuracy of the model. The structure of the SimAM attention mechanisms is depicted in
Figure 5.
The core idea of SimAM is based on the local self-similarity of images. In an image, neighboring pixels usually have strong similarity to each other, while the similarity between distant pixels is weak. SimAM takes advantage of this property to generate attention weights by calculating the similarity between each pixel in the feature map and its neighboring pixels. The formula is shown below:
where X is the input feature,
is the energy function on each channel. In order to prevent the possibility of too large a value for
, a sigmoid function is used to limit it; t is the value of the input feature,
is a constant value, which is 1 × 10
−4,
and
denote the mean and variance on each channel in X, respectively, and M = H × W denotes the number of values on each channel.
2.6. Proposed Improved Algorithm
In comparison to other one-stage target detection algorithms, such as YOLOv3 and YOLOv4, YOLOv7 exhibits superior performance in detecting small targets. However, the detection effect is less satisfactory when there are problems such as occlusion and low contrast of the target. Therefore, the following improvements are made to YOLOv7 to address these problems.
Figure 6 illustrates the network structure of the improved YOLOv7 algorithm.
3. Experiments
3.1. Experimental Dataset
The images for underwater target detection studied in this paper are from the 2020 National Underwater Robotics Professional Competition (URPC) dataset, which consists of 5543 images, with an image size of 640 × 640. Based on the definition of small, medium, and large targets in the COCO dataset, each type of target was divided into small, medium, and large targets, where the number of holothurian tags was 5537, in which the number of small targets was 61, the number of medium targets was 3487, and the number of large targets was 1989; the number of echinus tags was 22,343, in which the number of small targets was 2788, the number of medium targets was 13,696, and the number of large targets was 5859; the number of scallop tags was 6720, with the number of small targets 603, the number of medium targets was 1359, and the number of large targets 4758; and the number of starfish tags 6841, with the number of small targets 588, the number of medium targets 4059, and the number of large targets 2194. The results are shown in
Figure 7.
The sample images, labeled with location and category information, were subsequently stored in PASCAL VOC format. The dataset was randomly divided into a training set of 4434 images and a validation set of 1109 images according to the corresponding marine organism category in a ratio of 8:2.
3.2. Experimental Environment and Parameterization
This experiment is implemented in the ubuntu20.04 operating system based on the PyTorch deep learning framework, the GPU selection for the size of the memory 12 G NVIDIA GeForce RTX 3080Ti, the CPU configuration is a 12-core AMD Ryzen 9 5900X, PyTorch version 2.0.1, CUDA version 11.7, and the python language environment is 3.8.17.
The hyperparameters for this experiment are configured as follows: in the model training phase, the parameters are tuned using the SGD optimizer, with the initial learning rate set to 0.01 and momentum to 0.937, and the learning rate decayed using warm up, with the weight decay coefficient set to 0.0005. In addition, the batch size is set to 8, and a total of 150 rounds are trained.
3.3. Experimental Design
3.3.1. Evaluation Metrics
This experiment focuses on underwater target detection algorithms, which should satisfy both real-time and good detection performance, so the values of FPS, FLOPS, Params,
, precision and recall are used as evaluation indexes to compare the algorithms. FPS represents the frame rate, indicating the number of images processed per second. FLOPS represents the number of floating-point operations executed, serving as a measure of the model’s computational complexity, Params is the sum of the model parameters, which are used to evaluate the size of the model, and
is the average value of the precision of each type of underwater target detection. The formula for
is as follows:
where
is the number of underwater target classes and
(Average Precision) is the precision value of each class.
Additionally, the precision and recall values are derived through the following calculations:
where
is the number of positive samples predicted to be positive (true positive);
is the number of negative samples predicted to be positive (false positive);
is the number of positive samples predicted to be negative (false negative).
3.3.2. Experimental Results
Three images are extracted from the dataset for comparison. The three original images extracted are shown in
Figure 8a. Using YOLOv7 and this paper’s algorithm to detect these three pictures, the results will be detected for comparison, and the comparison results are presented in
Figure 8b,c.
As illustrated in
Figure 8, the YOLOv7 algorithm is more prone to miss detection and partially occluded cases are not recognized or recognized with lower confidence values compared to this paper’s algorithm, whereas this paper’s algorithm is able to detect as accurately as possible all types of underwater targets on the map.
Figure 9 illustrates the precision-recall (P-R) curves of the original YOLOv7 algorithm and the modified YOLOv7 algorithm after improving each module, which were trained on the identical underwater target dataset. Specifically,
Figure 9a illustrates the P-R curves of the YOLOv7 algorithm,
Figure 9b represents the P-R curves of the YOLOv7_kmeans,
Figure 9c represents the P-R curves of the YOLOv7_kmeans with the addition of the ELAN_PC module, and
Figure 9d represents the P-R curves of the YOLOv7_kmeans with the addition of the ELAN_PC module and the P-R curve of the ShapeIou NWD loss function.
Figure 9e represents the P-R curve for the final addition of the SimAM attention mechanism. As can be seen from
Figure 9a,e, the AP of holothurian increased by 1.1%, the AP of echinus increased by 0.8%, the AP of scallop increased by 0.4%, and the AP of starfish increased by 1.6%.
5. Conclusions
In this paper, an underwater target detection algorithm based on an improved YOLOv7 is proposed for underwater targets with some smaller sizes, complex backgrounds, and low contrast. By introducing the SimAM attention mechanism, the improved ELAN_PC structure and the ShapeIou_NWD loss function are used to improve the ability to extract features for smaller underwater targets and to reduce the loss of feature information. Meanwhile, in order to enhance the robustness of the algorithm in this paper, the K-Means anchor frame clustering algorithm is used to enhance the performance of the algorithm. Several comparison experiments and ablation experiments show that the improved YOLOv7 target detection algorithm achieves an average accuracy value of 85.7%, which is 1% higher than the original YOLOv7, and the speed reaches 122.9 frames/s, which is a good balance between detection speed and accuracy. Afterwards, the main feature extraction part of the model will be improved by using a lighter weight module to allow the algorithm to increase the inspection accuracy of underwater targets while being lighter weight.