Weak-Texture Seafloor and Land Image Matching Using Homography-Based Motion Statistics with Epipolar Geometry

Abstract

The matching of remote sensing images is a critical and necessary procedure that directly impacts the correctness and accuracy of underwater topography, change detection, digital elevation model (DEM) generation, and object detection. The texture of images becomes weaker with increasing water depth, and this results in matching-extraction failure. To address this issue, a novel method, homography-based motion statistics with an epipolar constraint (HMSEC), is proposed to improve the number, reliability, and robustness of matching points for weak-textured seafloor images. In the matching process of HMSEC, a large number of reliable matching points can be identified from the preliminary matching points based on the motion smoothness assumption and motion statistics. Homography and epipolar geometry are also used to estimate the scale and rotation influences of each matching point in image pairs. The results show that the matching-point numbers for the seafloor and land regions can be significantly improved. In this study, we evaluated this method for the areas of Zhaoshu Island, Ganquan Island, and Lingyang Reef and compared the results to those of the grid-based motion statistics (GMS) method. The increment of matching points reached 2672, 2767, and 1346, respectively. In addition, the seafloor matching points had a wider distribution and reached greater water depths of −11.66, −14.06, and −9.61 m. These results indicate that the proposed method could significantly improve the number and reliability of matching points for seafloor images.

1. Introduction

The surveying and mapping of underwater topography for coastal zones, islands, and reefs is one of the most crucial research fields in oceanography and provides key geographic information data for nearshore navigation [1,2,3], ocean geomorphology [4,5], coral reef studies [6,7], and hydrography [8,9]. Currently, underwater topography in coastal zones and island areas is primarily dependent on airborne light detection and ranging (LiDAR) and spaceborne photogrammetry, and the efficiency of shipborne acoustic systems in these areas is extremely low [10,11]. However, the detection area of this method is usually limited by the airborne platform [12]. Another mainstream method for underwater topography is spaceborne photogrammetry based on high-resolution multispectral stereo images, which can efficiently obtain expansive underwater topography of coastal, island, and reef areas [13]. In the process of spaceborne photogrammetry, remote sensing image matching is a critical and necessary procedure that directly impacts the correctness and accuracy of underwater topography [14]. Furthermore, image matching is identified as a crucial step, with the quality of the match exerting a direct influence on the performance of applications, such as change detection, object detection and tracking, and image stitching. Compared with conventional image matching for land areas [15,16,17], the remote sensing image matching of coastal zones and islands includes some areas covered by the water column with different depths, which decreases the image texture information and significantly increases the matching difficulty [18,19].

The process of remote sensing image matching can be divided into two steps: (1) extraction and matching of the point pairs from the stereo image pairs, and (2) optimization of the matching points [20]. Methods for the extraction and matching of the point pairs can be generally classified into two categories: area-based methods and feature-based methods [21]. Area-based methods, also referred to as correlation-like or template matching, establish correspondence between reference and sensed pixels using a similarity measure. Feature-based methods aim to identify distinctive features in images and then match these features to establish correspondence between reference and sensed images. However, area-based matching methods are only suitable for homologous images with linear radiometric differences and face challenges when used to extract common features between image pairs. Conversely, feature-based matching is more suitable for situations involving significant scales of offset, geometric distortion or scale differences between two images [22]. Typical solutions rely on feature detectors, descriptors, and matchers to generate putative correspondences. At present, feature-matching methods, such as invariant feature transform (SIFT) [23], speeded-up robust features (SURF) [24], accelerated KAZE (AKAZE) [25], and oriented brief (ORB) [26], are generally performed based on “feature points” detection and description by their local intensity or shape patterns.

After the process of extraction and matching of the point pairs, extensive matching points are extracted, including true and false matches. Therefore, the initial result of the matching points should be optimized to remove false matches. A large portion of true matches are eliminated to limit false matches [27]. Several techniques have been proposed to solve this problem, for example, the ratio test (RT) and GMS [9,28]. The RT identifies high-quality matches by comparing the distances of the two closest candidate points. However, due to the lack of more stringent constraints, many incorrect matches remain difficult to eliminate in complex scenes. GMS are related to optical flow, point-based coherence techniques, and patch-match-based matchers, directly using smoothness to help match estimations [29,30,31]. However, during the separation of true and false matches with GMS, the scale and rotation of matching points are only detected and estimated by five and eight discrete grid kernels [32,33,34]. In addition, a series of putative matches are included in one grid, in which the different scales and rotations of matches are averaged. Therefore, a large portion of true matching points could be considered false matches and removed.

For remote-sensing images of coastlines, islands, and reefs that contain seafloor image pixels, the local self-similarity of the seafloor terrain leads to weak texture features and a lack of salient point features [35]. In addition, the quality of imaging in water areas is affected by waves, solar flares, and water properties [36,37]. As a result, traditional matching methods have low matching accuracy. Feature-matching points that are automatically extracted in areas with poor texture still require a significant amount of manual editing before practical application [38]. To overcome the existing issues with the weak texture areas of the seafloor for remote sensing images, the homography-based motion statistics with an epipolar constraint (HMSEC) method is proposed. This method can effectively improve the number, reliability, and robustness of matching points for weak-texture seafloor images by translating a large number of putative matches into high-match quality and using the homograph and epipolar geometry to estimate the scale and orientation influence of matching points. In combination with ORB, the matching number and accuracy are ensured and enhanced, and the distribution of matching points in the seafloor area is improved with increasing depth. In addition, different experiments were performed using the island and reef images to validate the correctness and accuracy of the proposed method. Furthermore, various matching metrics were used to compare and estimate matching results obtained by the various methods. Finally, the matching points obtained by different methods were projected into the water-depth image to analyze the distribution of matching results and estimate the detection capability of HMSEC at different water depths.

2. Methodology

The HMSEC method with ORB was adopted to achieve robust feature matching and obtain highly reliable matching points based on the neighborhood information of matching points. The ORB method was used to generate a sufficient number of putative matching points. For each initial putative matching point generated by ORB, the neighborhood information used in HMSEC was used to determine correctness and remove false matching points. In addition, epipolar geometry was used as a constraint with homography-based motion statistics to ensure the reliability of the extracted true matching points. To realize this aim, the proposed method, HMSEC with ORB, has four parts: (1) putative matching, (2) filtering with motion statistics, (3) homography-based scale and orientation adaption, and (4) matching-metrics evaluation. Figure 1 shows the flow chart of HMSEC with ORB.

2.1. Filtering with Motion Statistics

HMSEC rely on the motion smoothness constraint, the assumption that neighboring pixels in one image move together as they often land in one object or structure. Although the assumption is not generally correct, e.g., it is violated in image boundaries, it suits most regular pixels. This is sufficient for our purpose as we are not targeting a final correspondence solution but a set of high-quality correspondences for RANSAC (Random Sample Consensus)-like approaches. The assumption means that neighboring true correspondences in one image are also close in other images, while false correspondences are not. This allows us to classify a correspondence as true or false by simply counting the number of its similar neighbors and the correspondences that are close to the reference correspondence in both images.

True matches are influenced by the smoothness constraints, while false matches are not. Let similar neighbors refer to those matches that are close to the reference match in both images. True matches often have more similar neighbors than false matches, as shown in Figure 2. To identify true matches, HMSEC use the number of similar neighbors.

Let M be all matches across images $I_1$ and $I_2$, and let $m_i$ be one match correspondence that connects the points $a_i$ and $b_i$ between two images. We define $m_i’s$ neighbors as $R_i$ = {$m_j$ | $m_j\in$ M, $m_j$ $\ne$ $m_i$, d$\left(a_i,a_j\right)$ < $\delta$}, and its similar neighbors as $C_i$ = {$m_j$ | $m_j\in$ $R_i$, d$\left(b_i,b_j\right)$ < $\delta$}, where d$\left(·,·\right)$ refers to the Euclidean distance of two points. We term $\left|C_i\right|$, the number of elements in $C_i$, motion support for $m_i$.

The motion support can be used as a discriminative feature to distinguish true and false matches. Modeling the distribution of $\left|C_i\right|$ for true and false matches, we obtain Equation (1), in which $B\left(·,·\right)$ refers to the binomial distribution, and ${|R}_i|$ refers to the number of neighbors for $m_i$. The symbol $t$ represents the probability of one true match supporting its neighbor pixels, which is close to the correct rate of correspondences. The false-match probability is denoted by $f$ and is usually small, as false correspondences approximately satisfy the random distribution in regular pixel regions.

$$\left|C_i\right|=\left\{\begin{array}{c}B\left({|R}_i|,t\right), if m_i is true\\ B\left({|R}_i|,f\right), if m_i is false\end{array}\right.$$

(1)

$$E_{\left|C_i\right|}=\left\{\begin{array}{c}{E}_t=\left|R_i\right| ·t, if m_i is true\\ E_f=\left|R_i\right| ·f, if m_i is false\end{array}\right.$$

(2)

$$V_{\left|C_i\right|}=\left\{\begin{array}{c}{V}_t=\left|R_i\right|·t·(1-t), if m_i is true\\ V_f=\left|R_i\right|·f·(1-f), if m_i is false\end{array}\right.$$

(3)

Furthermore, the expectation of $\left|C_i\right|$ can be derived and formulated as Equation (2), and its variance is expressed by Equation (3).

$$s={\displaystyle \frac{|E_t-E_f|}{\sqrt{V_t}+\sqrt{V_f}}}={\displaystyle \frac{\left|R_i\right|·(t-f)}{\sqrt{\left|R_i\right|·t·(1-t)}+\left|R_i\right|·f·(1-f)}}$$

(4)

Then, the separability between true and false matches can be defined as S and described by Equation (4), where S $\propto$ $\sqrt{\left|R_i\right|}$, and if $\left|R_i\right|\to \mathrm{\infty }$, $S\to \mathrm{\infty }$. This indicates that the value of S is greater, andthe separability becomes increasingly reliable when the number of feature points is sufficiently large. This occurs when $t$ is just slightly larger than $f$, and as a result, it is possible to obtain reliable matches from difficult scenes by increasing the number of detected feature points.

$$m_i\in \left\{\begin{array}{c} T, if \left|C_i\right|>{\alpha }_i\\ F , if \left|C_i\right|\le {\alpha }_i\end{array}\right.$$

(5)

$${\alpha }_i=\beta \sqrt{\left|R_i\right|}$$

(6)

In addition, it shows that improving feature quality ($t$) can also boost separability. The distinctive attributes allow us to decide if $m_i$ is true or false using the threshold $\left|C_i\right|$ given in Equations (5) and (6), where $T$ and $F$ denote true and false match sets, respectively. ${\alpha }_i$ is the threshold of $\left|C_i\right|$. The symbol $\beta$ is a hyperparameter, which is empirically set to range from 4 to 6.

2.2. Homography-Based Scale and Orientation Adaptation

In the matching process for two images, significant image changes in scale and rotation exist in each image [39]. To address this issue, multi-scale and multi-rotation solutions are adopted for image homography, which describes a geometric projective transformation from one image space to the other. I₁ and I₂ are shown in Figure 3.

$$\left[\begin{array}{c}{x}^{\prime }\\ y^{\prime }\\ 1\end{array}\right]={\displaystyle \frac{1}{s}}\cdot A\cdot R\cdot T\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\lambda \cdot H\cdot \left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\lambda \cdot \left[\begin{array}{ccc}{h}_{00}& h_{01}& h_{02}\\ h_{10}& h_{11}& h_{12}\\ h_{20}& h_{21}& h_{22}\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]$$

(7)

The homography between the two images is described by the H matrix, and the homographic transformation is represented by Equation (7), in which the coordinates of I₁ and I₂ are represented by $(x,y)$ and $(x^{\prime },y^{\prime })$; $(h_{00},h_{01},h_{02},h_{10},h_{11},h_{12},h_{20},h_{21},h_{22})$ represent the nine elements of the matrix H, while $h_{22}$ equals 1; the symbol $s$ is the scale factor between two images, and $\lambda$ is the reciprocal of $s$; A means the matrix of the camera’s internal parameters; and R and T represent the rotation and translation vectors of two images.

$$\left\{\begin{array}{c}s{x}^{\prime }=h_{01}x+h_{02}y+h_{03}\\ s{y}^{\prime }=h_{10}x+h_{11}y+h_{12}\\ s=h_{20}x+h_{21}y+1\end{array}\right.$$

(8)

$$\left\{\begin{array}{c}{x}^{\prime }={\displaystyle \frac{h_{01}x+h_{02}y+h_{03}}{h_{20}x+h_{21}y+1}}\\ y^{\prime }={\displaystyle \frac{h_{10}x+h_{11}y+h_{12}}{h_{20}x+h_{21}y+1}}\end{array}\right.$$

(9)

Through Equations (8) and (9), the scale factor of $s$ is removed, and the coordinate in image I₂, corresponding to the coordinate of the matching point in image I₁, can be determined. In this process, the rotation of two images is taken into account and eliminated by the rotation matrix R that is included in the matrix H.

In the process of the homography-based scale and orientation estimation, there are deviations in the coordinates and angles for the matching points. The coordinate deviation between the matching point after motion statistics and the point obtained by the homography matrix is statistically counted for image I₂, using standard deviation. Two-times standard deviation (${2\sigma }_d$) is used as a radius to construct a circle for each red point obtained by the homography matrix, represented in Figure 3. When the corresponding matching point after motion statistics is outside the circle, such as the green point P₁, this means its deviation is greater than the two-times standard of coordinate deviation, and it is an unreliable matching point. On the contrary, the matching points P₂ and P₃ are reliable matching points to be reserved. To improve the matching accuracy of the entire image, the angle deviation between the lines connects each matching point in I₁ with its corresponding matching point in I₂ and the calculation point. Then, the two-times standard deviation (${2A}_s$) of the angle deviation shown by the orange dashed line can be calculated and is used to evaluate the matching point correctness and reliability. Finally, for each matching point, when its coordinate and angle deviations in total are greater than the corresponding values of ${2\sigma }_d$ and $2A_s$, the point is considered unreliable and removed.

2.3. Matching-Metrics Evaluation

To evaluate the reliability and accuracy of HMSEC for nearshore remote sensing images, five matching metrics were applied based on the results of various algorithms and their comparisons. Therefore, this comprehensive set of metrics allowed us to embed our analysis into the existing body of work. The standard metric is a metric that is proposed for binary descriptor evaluation and used to assess the accuracy and reliability of feature-descriptor matching [17]. In addition, the putative match ratio (PMR) and matching score (MS) are also used for the raw matching performance of each image pair. Furthermore, a set of different metrics, the seafloor match ratio (SMR), seafloor matching number (SMN), and total matching number (TMN), were adopted to evaluate the image-matching results for the nearshore area via comparison with other methods and analysis of the HMSEC characteristics.

The following notations are used to illustrate the metrics in this paper: $M_p$ denotes all putative matches across two images; $F$ denotes all the features; $M_{in}$ denotes all inlier matches across two images; $M_e$ denotes all matches across two images with an epipolar constraint; $M_{eo}$ denotes all seafloor matches across two images with an epipolar constraint. The PMR was adopted and used to quantify the selectivity of the descriptor in terms of the fraction of the detected features initially identified as a pair of key point matches, which is represented by the formulation of ${|M}_p|$/$\left|F\right|$. The ratio of these two values was generally utilized to estimate the raw matching performance for each image. The MS is formulated as ${|M}_{in}|$/$\left|F\right|$ to represent the inlier ratio among all detected features, which is used to describe the robustness and transformation efficiency of feature points. The metric of the SMR is represented by ${|M}_{eo}|$/${|M}_e|$ and means the percentage of the seafloor matching points in the whole set of matching points. A higher percentage value expresses a better seafloor matching result.

In image-matching research, the parameters described above are generally used for the overall evaluation of matching results. In addition, recognition and determination by the human eye are also generally used to compare location matching points with true ground points or the relative location of matching points in left and right images. However, for islands and reefs located far away from land, it is difficult for humans to access them, and consequently, matching points in images are not often compared in situ with the ground point at the seafloor.

3. Study Area and Dataset

3.1. Study Area

This study focused on Zhaoshu Island, Ganquan Island, and Lingyang Reef in the South China Sea. Zhaoshu Island is a part of the Qilianyu Islands, which are situated at a 16.956 north latitude and 112.318 east longitude in China’s Xisha Islands. Ganquan Island and Lingyang Reef are in the Yongle Islands at a longitude of 111.592 and a latitude of 16.523. Zhaoshu Island, Ganquan Island, and Lingyang Reef are shown in Figure 4, and their locations are marked with red, blue, and orange stars, respectively. Zhaoshu Island and Ganquan Island include both land and seafloor areas. The WorldView-2 image shows a large reef flat containing various sediments in the shallow water of Zhaoshu Island. However, the sediment situation on Ganquan Island is relatively simple, and its water level is significantly lower than that on Zhaoshu Island. Lingyang Reef is a large atoll with a lagoon in its middle and is completely submerged. The radiation quality in Zhaoshu Island is slightly better than in the other two areas. Each of these three areas was selected to test the performance of the proposed method because they represent a significant area in the South China Sea.

3.2. Data and Experiment

The WorldView-2 satellite provides high-resolution data at a resolution of 0.52 m for panchromatic images and 1.8 m for multispectral images at an understellar point. WorldView-2 multispectral stereo image pairs of Zhaoshu Island, Ganquan Island, and Lingyang Reef with less than 10% cloud cover were used. The WorldView-2 multispectral image has 8 bands with a spectral range from 450 to 1040 nm [29]. The calibration accuracy of WorldView-2 image can reach 0.5%, providing high-quality geographic data. The wide coverage range and high sensitivity of these datasets make them powerful tools for global ocean measurements and monitoring. The images used in this study are detailed in

Table 1. Zhaoshu Island, Ganquan Island, and Lingyang Reef image data.

Study Area	Acquisition Time	Image Name	Resolution (m)
Zhaoshu Island	11 March 2017	17MAR11030327-M2AS-056861369020_01_P001	1.8
		17MAR11030446-M2AS-056861369020_01_P001
Ganquan Island	2 April 2014	14APR02033318-M2AS-054801903010_01_P001
		14APR02033200-M2AS-054801903010_01_P001
Lingyang Reef	2 April 2014	14APR02033159-M2AS-056861369050_01_P002
		14APR02033318-M2AS-056861369050_01_P002

. The images were preprocessed with atmospheric corrections using a fast line-of-sight atmospheric analysis of spectral hypercubes (FLAASH) model based on the environment for visualizing images (ENVI) 5.3 [40,41]. A solar glint effect exists in the image of Ganquan Island, which affects feature detection and matching-point extraction. Thus, the solar glint effect was eliminated using a linear regression method with the near-infrared band [42].

The newly proposed HMSEC with ORB method was used to perform matching for various remote sensing images around the islands and the reef. The matching results were compared with current frequently used methods to validate the matching capability and accuracy of HMSEC. The experiments were conducted as follows: (1) the HMSEC method was configured with the feature detections (ORB-HMSEC) to achieve matching extraction for three study areas; (2) The matching results of ORB-HMSEC were compared with those from other configured methods based on SIFT, SURF, AKAZE, and ORB with the RT, GMS, and HMSEC; and (3) Assessment metrics were used to assess the matching capabilities and accuracies of the various methods and ensure the correction and accuracy of HMSEC for land and seafloor areas.

4. Results

4.1. Matching-Point Results

To validate and analyze the robustness and accuracy of the proposed matching method, remote sensing images of Zhaoshu Island, Ganquan Island, and Lingyang Reef were used for correspondence acquisitions using eight different methods (

Table 2. Configuration methods for the extraction of matching points.

Matching Method	False-Match Filter	Method Configuration
SIFT	RT	SIFT-RT
SURF	RT	SURF-RT
AKAZE	RT	AKAZE-RT
	GMS	AKAZE-GMS
	HMSEC	AKAZE-HMSEC
ORB	RT	ORB-RT
	GMS	ORB-GMS
	HMSEC	ORB-HMSEC

). The methods were constructed by combining different matching methods, such as SIFT, SURF, ORB, and AKAZE, and the false match removal methods, the RT, GMS, and HMSEC. The SIFT and SURF algorithms use the differences in Gaussian key point detectors and Hessian matrix detectors, respectively, both of which rely on image gradient changes. In addition, to ensure the stability of key points and the accuracy of the descriptors, SIFT and SURF algorithms typically set high thresholds, which filter out many potential key points in weak-texture regions. Consequently, the number of matching points detected in weak-texture regions is generally low [43,44]; thus, GMS and HMSEC are not suitable for these tasks. ORB and AKAZE algorithms emphasize matching efficiency by using fast and simple key-point-detection algorithms. This generally results in the retention of more matching points; however, the accuracy is typically lower, and thus, they are suitable for the method proposed in this study. Generally, the number of feature points obtained by SIFT and SURF is reduced, especially for areas where the seafloor has a weak texture; therefore, GMS and HMSEC are not suitable for the two methods. In this study, GMS and HMSEC were combined with ORB and AKAZE. Based on the comparisons between the various methods, the matching reliability and accuracy of HMSEC were evaluated statistically by analyzing the characteristics of land and seafloor areas.

Point matching was achieved for Zhaoshu Island, Ganquan Island, and Lingyang Reef using the methods presented in Table 2. The matching results are shown in Figure 5a–c. The highest number of matching points was obtained using the ORB-HMSEC method (Figure 5a). The results obtained using the SIFT, SURF, ORB-GMS, and AKAZE-HMSEC methods are similar and higher than those obtained using the AKAZE-RT, ORB-RT, and AKAZE-GMS methods. For Ganquan Island, the matching result obtained using the ORB-HMSEC method was better than that obtained using other methods (Figure 5b). The ORB-RT and ORB-GMS methods yielded similar results that are higher than those obtained using the other tested methods. Lingyang Reef is completely submerged. The number of matching points extracted using the ORB-RT, ORB-GMS, and ORB-HMSEC methods are similar and significantly higher than those extracted using other methods (Figure 5c).

4.2. Comparison and Evaluation of the Matching Results

To evaluate the matching correctness and reliability of the ORB-HMSEC method, five matching metrics were used and compared with other methods. For Zhaoshu Island, Ganquan Island, and Lingyang Reef, the matching metric values obtained using the various methods are listed in Table 2,

Table 3. Comparison of the matching metrics obtained using different matching methods for Zhaoshu Island.

Zhaoshu Island	PMR (%)	MS (%)	SMR (%)	SMN	TMN
SIFT	40.15	36.91	57.58	399	693
SURF	49.85	40.68	57.02	256	449
AKAZE-RT	54.08	43.43	10	2	20
ORB-RT	48.11	34.31	4.942	17	344
AKAZE-GMS	54.09	42.44	0	0	25
AKAZE-HMSEC	53.57	43.79	43.44	182	419
ORB-GMS	48.11	40.54	26.92	91	338
ORB-HMSEC	50.44	53.65	60.20	1812	3010

and

Table 4. Comparison of the matching metrics obtained using different matching methods for Ganquan Island.

Ganquan Island	PMR (%)	MS (%)	SMR (%)	SMN	TMN
SIFT	39.17	32.72	0	0	124
SURF	37.66	21.75	0	0	144
AKAZE-RT	56.35	31.22	2.35	2	85
ORB-RT	30.57	20.79	39.10	646	1652
AKAZE-GMS	57.01	33.18	3.33	3	90
AKAZE-HMSEC	56.54	32.36	1.37	3	219
ORB-GMS	31.89	33.22	44.32	1144	2581
ORB-HMSEC	31.27	34.31	24.13	1291	5348

, respectively. The metrics obtained using the ORB-HMSEC method were highest at 53.65, 60.20, 1812, and 3010 for the MS, SMR, SMN, and TMN, respectively, but not for the PMR at 50.44 (Table 3). The corresponding minimum values were obtained using the SIFT, ORB-RT, AKAZE-GMS, AKAZE-GMS, and AKAZE-RT methods. When using the AKAZE-GMS method, the seafloor matching point could not be detected; thus, the values of the SMR and SMN are zero.

For Ganquan Island, the metric values of the MS, SMN, and TMN determined using the ORB-HMSEC method were the highest, reaching 34.31, 1291, and 5348 (Table 4). The PMR and SMR values were 31.27 and 24.13, which is in the middle of all results. The seafloor matching points could not be extracted by the SIFT and SURF methods, and the SMR and SMN values are zero. Using the AKAZE-RT, AKAZE-GMS, and AKAZE-HMSEC methods, the SMN metric reached two or three. The TMN value obtained using the AKAZE-GMS method was the lowest, reaching 90. The PMR and MS values obtained using the ORB-RT method were 30.57 and 20.79, respectively, which are lower than the other methods.

For Lingyang Reef, five metric values were obtained using various methods (

Table 5. Comparison of the matching metrics obtained using different matching methods for Lingyang Reef.

Lingyang Reef	PMR (%)	MS (%)	SMR (%)	SMN	TMN
SIFT	10.29	27.70	100	210	210
SURF	28.70	54.90	100	227	227
AKAZE-RT	20.89	38.61	100	60	60
ORB-RT	64.36	57.76	100	4821	4821
AKAZE-GMS	20.89	37.34	100	58	58
AKAZE-HMSEC	20.91	34.81	100	50	50
ORB-GMS	64.36	59.29	100	5023	5023
ORB-HMSEC	65.95	69.61	100	6369	6369

). The SMR value for each method was 100, and the SMN and TMN values were equal because the reef region covered by the image was submerged. The maximum PMR, MS, SMR, SMN, and TMN values obtained using the ORB-HMSEC method were 65.95, 69.61, 100, 6369, and 6369, respectively. The minimum PMR and MS values obtained using the SIFT method were 10.29 and 27.70, respectively. The matching point number was 50, and it was obtained using the AKAZE-HMSEC method.

4.3. Distributions of Seafloor and Land Matching Points

Underwater image texture becomes weaker with increasing water depth, which results in different distributions and matching results [45,46]. The ORB-GMS method achieved the second-best performance (Table 5). Consequently, we focused on comparing the matching capabilities of ORB-GMS and the proposed algorithm across various water depth intervals. The distributions of the seafloor and land matching points obtained using the ORB-HMSEC method at Zhaoshu Island, Ganquan Island, and Lingyang Reef were compared with the corresponding results of ORB-GMS. To illustrate the weakening of underwater textures and the reduction in the number of matching points with increasing water depth, the matching results of the two methods were overlaid with a bathymetric map to visually demonstrate the differences in the number of matching points at various depths. Distributions and comparisons of the matching points obtained using the ORB-GMS and ORB-HMSEC methods are shown in Figure 6. The white region represents the land on the island above the water surface, and the underwater region with different water depths is represented by different colors. The results for Zhaoshu Island and Ganquan Island reveal that the number of seafloor matching points was higher with the ORB-HMSEC method than the ORB-GMS method. The number of land matching points obtained from the ORB-HMSEC method was also improved. Furthermore, the seafloor point distribution with the ORB-HMSEC method was wider and deeper than that with the ORB-GMS method. Lingyang Reef is completely submerged, and consequently, its matching points are all on the seafloor. The distribution and number of matching points obtained using the ORB-HMSEC method were also higher than those obtained using the ORB-GMS method.

The capability of the ORB-HMSEC method was assessed at various water depths. The statistical distributions of matching-point numbers and different water depths at 1 m intervals are compared in Figure 7. The matching points obtained using the ORB-HMSEC method were improved at almost every water depth interval, except for the depth interval from 2 to 4 m at Ganquan Island. In this interval, hundreds of matching-point numbers were obtained using the two methods, and the matching number for GMS was slightly higher than that for HMSEC. For Zhaoshu Island, Ganquan Island, and Lingyang Reef, the maximum water depth of the matching points obtained by GMS is −5.88, −11.44, and −7.12 m. The corresponding values of HMSEC are −11.66, −14.06, and −9.61 m, and the deep increment reaches −5.78, −2.66, and −2.49 m, respectively.

5. Discussion

The results of various matching metrics (Table 2, Table 3 and Table 4) indicate that the PMR obtained using the ORB-HMSEC method is at the middle level because the feature number detected by ORB is notably larger than that detected by the other methods. The MS value obtained using the ORB-HMSEC method was the highest for the three images, indicating that the number of putative matching points and the conversion rate from features to putative matches were the highest. The TMN values for the three study areas were determined using various methods. The results show that the ORB-HMSEC method was significantly better than the other tested methods, reaching 3010, 5348, and 6369 for Zhaoshu Island, Ganquan Island, and Lingyang Reef, respectively; relative to the second-best values, the matching numbers improved by 2317, 2767, and 1346, respectively. Thus, the results demonstrate that numerous putative matching points can be transformed efficiently into reliable matching points using motion smoothness assumptions and motion statistics.

Compared with the ORB-GMS method, the ORB-HMSEC method achieved higher SMN and TMN values. In the GMS method, only five relative scales and eight relative rotations between image pairs were defined and considered for the matching estimation of scale and rotation enhancement (Figure 8). Furthermore, the grid-based framework divided two images into non-overlapping cells to reduce the complexity of the calculation, and only cell pairs, not corresponding match points, were classified (Figure 8a,b). The grid kernel was fixed on one image, and eight different grid kernels were used to simulate possible relative rotations [32]. Five different grid kernels were used to simulate the potential relative scales in other images. Therefore, during the matching process, the scale and rotation of the image pairs should be discrete to achieve rough simulations and estimations, but this also results in errors and the elimination of matching points [47,48]. To overcome this problem, a homography-based adaptation method was proposed and used in the HMSEC method. To accurately estimate the scale and rotation of each matching point, coordinate and angle deviations were used (Figure 3). These results demonstrate that ORB-HMSEC has higher filtering and estimating accuracy for matching points, and the number of matching points can be confirmed and improved (

Table 6. Increments of matching points between HMSEC and GMS.

Study Area	Increments of the Matching Points
	Land Points	Underwater Points	Total Points
Zhaoshu Island	951	1721	2672
Ganquan Island	2620	147	2767
Lingyang Reef	-	1346	1346

).

In underwater areas, image texture becomes weaker as the water depth increases. The distribution becomes wider, and the number of seafloor matching points acquired increases when using the ORB-HMSEC method (Figure 6 and Figure 7). The maximum water depth of the matching points also improved, reaching −11.66, −14.06, and −9.61 m at Zhaoshu Island, Ganquan Island, and Lingyang Reef, respectively. The number of seafloor matching points increased with increasing water depth (Figure 7), and between −3 and −4 m, the matching number peaks. In shallower areas ranging from 0 to −2 m, the wavelength and quantity of white and breaker waves increased, which may have reduced the number of matching points. Subsequently, increasing the depth reduced the image texture and the matching number. With the ORB-HMSEC method, the number of matching points improved with increasing water depth. Thus, the depth and distribution of underwater points could be increased.

Image homography was used to eliminate the matching error caused by multi-scale and multi-rotation changes in each image. In this procedure, the two-times standard of coordinate and angle deviations was adopted to further reduce the matching error and ensure the reliability and stability of the matching results in most situations. As shown in Figure 9, different values of the coordinate and angle thresholds were used to obtain matching points for analyzing the changing trend of the matching-point number. For the study areas of Zhaoshu Island, Ganquan Island, and Lingyang Reef, the value of 2${\mathsf{\sigma }}_{\mathrm{d}}$ is 0.96, 1.12, and 0.75 pixels. The corresponding value of 2${\mathrm{A}}_{\mathrm{s}}$ is 0.36, 1.44, and 1.80 degrees. Through the results shown in Figure 9, it is clear that the number of matching points changes relatively smoothly when values of the coordinate and angle threshold range from one-time to two-times standard deviations. These results demonstrate that the accuracy and reliability of matching points can be further evaluated and improved, using the adaptation constraint with the homography-based scale and orientation. Seafloor texture varies between island and reef locations. In relatively turbid and deep-water environments, the radiation information in the image and image texture are reduced. For this situation, two-time standard deviations of ${\mathsf{\sigma }}_{\mathrm{d}}$ and angle deviations of ${\mathrm{A}}_{\mathrm{s}}$ could be further improved, ranging from two times to one time to ensure matching accuracy and reliability in the ORB-HMSEC method.

6. Conclusions

This study proposes a novel matching method, ORB-HMSEC, that uses homography-based motion statistics. The proposed method efficiently transforms numerous putative matching points into reliable matching points using motion smoothness assumptions and motion statistics. In this process, the scale and rotation of each matching point in the image pairs can be estimated accurately to improve the accuracy and reliability of the matching points. According to the experimental results and comparisons with other methods, the ORB-HMSEC method efficiently improves the reliability and number of matching points both on the seafloor and in land regions. The increment of the matching points reached 2672, 2767, and 1346 for the WorldView-2 multispectral images of Zhaoshu Island, Ganquan Island, and Lingyang Reef, respectively. Furthermore, the seafloor matching points of ORB-HMSEC had wider distributions and deeper water depths of −11.66, −14.06, and −9.61 m, respectively. Thus, the ORB-HMSEC method is highly reliable, with a large number of matching points in the study area. In the future, we plan to conduct a marching experiment using various remote sensing images to further explore the potential of this proposed method in different environments.