Siam-EMNet: A Siamese EfficientNet–MANet Network for Building Change Detection in Very High Resolution Images

Abstract

As well as very high resolution (VHR) remote sensing technology and deep learning, methods for detecting changes in buildings have made great progress. Despite this, there are still some problems with the incomplete detection of change regions and rough edges. To this end, a change detection network for building VHR remote sensing images based on Siamese EfficientNet B4-MANet (Siam-EMNet) is proposed. First, a bi-branches pretrained EfficientNet B4 encoder structure is constructed to enhance the performance of feature extraction and the rich shallow and deep information is obtained; then, the semantic information of the building is input into the MANet decoder integrated by the dual attention mechanism through the skip connection. The position-wise attention block (PAB) and multi-scale fusion attention block (MFAB) capture spatial relationships between pixels in the global view and channel relationships between layers. The integration of dual attention mechanisms ensures that the building contour is fully detected. The proposed method was evaluated on the LEVIR-CD dataset, and its precision, recall, accuracy, and F1-score were 92.00%, 88.51%, 95.71%, and 90.21%, respectively, which represented the best overall performance compared to the BIT, CDNet, DSIFN, L-Unet, P2V-CD, and SNUNet methods. Verification of the efficacy of the suggested approach was then conducted.

1. Introduction

Remote sensing as an effective means to obtain spatial and temporal information of ground objects with full coverage and long time series provides an effective guarantee for land cover monitoring [1,2], agricultural surveys [3], disaster assessment [4,5], military reconnaissance [6], etc. A VHR remote sensing image consists of rich spatial information and detailed features that can be used to detect changes to specific ground objects, such as buildings [7]. Buildings are important carriers of urban development. Real-time and accurate monitoring of building change information can provide a scientific basis and reference for ecological environmental protection, natural resource management, and urban expansion analysis [8,9]. Therefore, how to detect building changes automatically and efficiently using multi-temporal VHR remote sensing images has become a research hotspot that has received a significant amount of attention [10].

Building change information can be detected by the direct comparison of images [11,12] or by post-classification comparisons [13,14]. The former method uses differences between images and obtains change results by clustering or thresholding [15]. Yuan et al. [16] used multichannel Gabor filters to process images at different scales and orientations and extract building texture features, which solved false changes caused by projection differences. In addition to the difference calculation using the radiometric luminance values of each band of image pixels, some feature indices can be used. Examples include the commonly used morphological building index (MBI) [17] or the normalized difference built-up index (NDBI) [18]. The post-classification comparison method often uses various machine learning models to classify multi-temporal remote sensing images and then perform change detection. Among them, random forest (RF) [19], decision tree [20], support vector machine (SVM) [21], Markov random field (MRF) [22], and conditional random field (CRF) [23] have been applied. Tan et al. [24] constructed an SVM-based hyperspectral remote sensing image classification model, and the results showed that the radial basis kernel function had the highest classification accuracy when SVM performed hyperspectral classification. Zhang et al. [25] used the RF and utilized it to obtain image-level change detection results to reduce environmental effects such as illumination and observation angle, and then fused pixel-level change results and image objects to achieve image-level and target-level building change detection. Although classical change detection methods are well researched, the detection results are evidently affected by atmospheric conditions, and often perform poorly in complex scenes, which cannot meet the demand of high-precision change detection.

After several years of development, the satellites have evolved from the kilometer level to the meter and submeter level, and the increased spatial resolution allows us to perform finer monitoring of ground changes. At the same time, deep learning methods have injected new vigor into building change detection research. Zhu et al. [26] combined an improved SegNet network with image morphology to identify new buildings, but it is not sensitive to changes with large structural differences. A Unet++-based network with multiple side outputs was proposed by Peng et al. [27]. Since the depth features of a single image are not extracted, it limits the accuracy of change detection. Daudt et al. [28] proposed three end-to-end structures, in which the siamese structure with bi-branches can extract feature information separately and effectively enhance the accuracy of change detection. Fang et al. [29] combined the siamese network and NestedUNet, and proposed the SNUNet change detection network, which obtained better results on the CDD dataset through dense skip connections between the encoder–decoder. Chen et al. [30] divided the image into multiscale subregions, extracted features using a bi-branches pyramid model, and produced a remotely sensed building change detection dataset (LEVIR-CD). Although these siamese building change detection methods enhance the efficiency of change detection and achieve better accuracy, most models have inadequate feature extraction of the images and are prone to missed detection.

The attention mechanism can focus attention on the region of interest, improving the building change detection accuracy to some extent [31,32,33]. DASNet [34] introduces an extended attention mechanism consisting of two components, namely channel attention mechanism and spatial attention mechanism, which enhances the performance of the model in detecting building changes. Lei et al. [35] proposed an SNLRUX++ network for building change detection, which improves the prediction of feature maps at different scales by cascading multiscale feature fusion methods on dense building detection performance. In a study by Wang et al. [36], Unet++ with a multilevel difference module is combined to highlight change regions while reducing the influence of “pseudo-change”. The above deep learning method extracts the image features at a deeper level compared to traditional methods. However, continuous down-samplings causes insufficient spatial position information, which will lead to an incomplete detection of change regions and rough building edges.

Based on this, a siamese EfficientNet B4-MANet network (Siam-EMNet) is proposed for building change information extraction of VHR images. This article’s primary contributions are as follows:

(1) A bi-branched EfficientNet B4 encoder structure is designed. This encoder structure is compounded and expanded according to the width, depth, and resolution of the network, which helps to better predict building change regions and acquire higher quality prediction results. Meanwhile, pretrained weights are used to make the experimental results converge more stably.

(2) A Siam-EMNet change detection network is constructed. The decoder integrates PAB and MFAB from the MANet to up-sample the feature mapping of the encoder. The details are recovered and improved in the up-sampling process, the edge information of the changing regions can be detected more accurately, and the missed detection of small regions can be effectively avoided.

(3) The Siam-EMNet model is optimized using a hybrid loss function with a weighted combination of dice loss and cross-entropy loss to reduce the detection error caused by the imbalance of change and unchanged samples.

The rest of the paper is arranged as follows: Section 2 explains the proposed Siam-EMNet method; Section 3 performs the analysis of experimental results; Section 4 discusses the method of this paper; and Section 5 concludes the work of this paper.

2. Building Change Detection Framework

To detect building changes, a Siam-EMNet network model is proposed, which uses bi-temporal images as inputs and outputs binary detection maps. Figure 1 shows the network structure, where the encoder consists of siamese bi-branches EfficientNet B4, and the bi-branches share weights. To improve the accuracy of the model, the structure uses the compound scaling method on the network’s width, depth, and resolution. The PAB module in the decoder is used to acquire the spatial dependencies between pixels, and the MFAB module acquires the channel dependencies between arbitrary feature maps by fusing high-level and low-level semantic features, which combined with the dual attention mechanism can effectively use the multiscale feature information in the image to improve the accuracy.

The encoder–decoder6based network has high feature extraction accuracy [37]. The Siam-EMNet network can comprehensively and efficiently extract bi-temporal feature information from VHR remote sensing images and achieve the efficient fusion of multilevel information. EfficientNet B4 [38] pretrained on ImageNet is used as the backbone in the encoder to extract deep features, which improves the overall accuracy. Moreover, by using a skip connection, the encoder’s features are transferred to the decoder, which makes deep and shallow features more efficient to fuse.

2.1. Bi-Branches EfficientNet B4 Encoder

Generally, the accuracy of the model is optimized by deepening the network depth, expanding the network width, and improving the network resolution. AlexNet [39] combined dropout, ReLU, LRN, and other technologies with CNN for the first time, extending the width and depth of the CNN. VGGNet [40], ResNet [41], and InceptionNet [42] optimize the model by deepening the network depth and expanding the network width, respectively. Huang et al. [43] improved model performance by increasing image resolution. In most of the past research, only either width, depth, or resolution has been adjusted. EfficientNet is a series of CNN families proposed by Tan et al. [38] based on existing CNNs. The study extended different combinations of network width, depth, and image resolution to explore the effect of different combinations on the experimental accuracy, with a total of eight versions proposed, from EfficientNet B0 to EfficientNet B7. As shown in

Table 1. EfficientNet B0 network architecture.

Block $\mathit{i}$	Operator ${\widehat{\mathit{F}}}_{\mathit{i}}$	Layers ${\widehat{\mathit{L}}}_{\mathit{i}}$	Resolution ${\widehat{\mathit{H}}}_{\mathit{i}}\times {\widehat{\mathit{W}}}_{\mathit{i}}$	Channels ${\widehat{\mathit{C}}}_{\mathit{i}}$
1	Conv 3 × 3	1	2242	32
2	MBConv, k3 × 3	1	1122	16
3	MBConv, k3 × 3	2	1122	24
4	MBConv, k5 × 5	2	562	40
5	MBConv, k3 × 3	3	282	80
6	MBConv, k5 × 5	3	142	112
7	MBConv, k5 × 5	4	142	192
8	MBConv, k3 × 3	1	72	320
9	Conv1 × 1&Pooling&FC	1	72	1280

with B0 as an example, each network is divided into a total of nine blocks.

In this study, a bi-branch EfficientNet B4 was used to extract feature information, with the two branches sharing weights and making efficient use of the information. Figure 2 shows the structure of the single-branch EfficientNet B4. The first block is a convolutional layer with a convolutional kernel size of 3 × 3 steps of 2. Block 2 to Block 8 repeat the stacked MBConv structure, while Block 9 is a normal 1 × 1 convolutional layer.

EfficientNet optimizes the network from three dimensions of width, depth, and resolution; its composite parameters are obtained using neural architecture search (NAS) technology, which is described as follows:

$$\underset{d,w,r}{\max } Accuracy \left(Nas\left(w,d,r\right)\right)$$

(1)

$$s.t. Nas \left(w,d,r\right)=\underset{i=1,\dots s}{\odot }{\widehat{F}}_i^{d\cdot {\widehat{L}}_i}\left({\mathrm{X}}_{\langle \mathrm{w}\cdot {\widehat{\mathrm{C}}}_i,\mathrm{r}\cdot {\widehat{\mathrm{H}}}_i,\mathrm{r}\cdot {\widehat{\mathrm{W}}}_i\rangle }\right)$$

(2)

$$\mathrm{Memory} \left(Nas\right) \le target\_memory$$

(3)

$$\mathrm{FLOPS} \left(Nas\right) \le target\_flops$$

(4)

where ${\widehat{F}}_i, {\widehat{L}}_i, {\widehat{H}}_i, {\widehat{W}}_i, \mathrm{and} {\widehat{C}}_i$ are the predefined parameters of the basic network (Table 1), w, d, and r are the coefficients used to scale the width, depth, and resolution of the network, respectively; in Equation (2), $s.t.$ denotes the constraints, $\underset{i=1,\dots s}{\odot }$ denotes the concatenated multiplication operation, ${\widehat{F}}_i^{d\cdot {\widehat{L}}_i}$ denotes that$F_i$ is repeated ${\widehat{L}}_i$times in the block and d is used to scale the depth, ${\mathrm{X}}_{\langle \mathrm{w}\cdot {\widehat{\mathrm{C}}}_i,\mathrm{r}\cdot {\widehat{\mathrm{H}}}_i,\mathrm{r}\cdot {\widehat{\mathrm{W}}}_i\rangle }$ denotes the feature matrix of the input block; target_memory and target_flops in Equations (3) and (4) are restricted to Memory (Nas) and FLOPS (Nas), respectively, and the Memory (Nas) and FLOPS (Nas) of each model are optimal values less than or equal to the restricted values.

2.2. Dual Attention Mechanism Decoder

The local feature information obtained using traditional CNNs may lead to target misdetection. The decoder of the Siam-EMNet network model learns from the decoding structure of MANet, which has been maturely used for semantic segmentation of medical images [44]. To establish a rich context connection model on local features, dual attention modules (PAB and MFAB) are designed to capture spatial and channel information in the decoder part. Inspired by the literature [45], PAB [33] and MFAB [46] are introduced. The PAB module makes use of rich spatial context information to model, thus enhancing its representation ability. The MFAB captures feature–channel relationships by combining high-level and low-level feature mappings. According to the importance of the building segmentation task, the feature map is enhanced and suppressed. The structures of the two modules are shown in Figure 3, with PAB in (a) and MFAB in (b).

2.2.1. Position-Wise Attention Block

In Figure 3a, first, A$\in {ℛ}^{C\times H\times W }$with local features is inputted to the convolution layer to generate two new feature maps B and D, respectively, where $\left\{\mathrm{B},\mathrm{D}\right\}\in {ℛ}^{C\times H\times W }$; B and D are resampled as ${ℛ}^{C\times N}$ (N is the total number of pixels N = H × W), then transposed and multiplied, and the spatial attention map $Q\in {ℛ}^{N\times N}$ is computed using the softmax layer:

$$Q_{ji}=\frac{\exp \left(B_i\cdot D_j\right)}{{{\displaystyle \sum }}_{i=1}^N\exp \left(B_i\cdot D_j\right)}$$

(5)

where Q_ji denotes the effect of position i on position j. Meanwhile, another feature mapping $E \in {ℛ}^{C\times H\times W }$. generated by A is resampled to ${ℛ}^{C\times N}$.

Then, the transpose of matrices E and Q is multiplied and the result is resampled to ${ℛ}^{C\times H\times W }$; finally, a scale parameter α is applied, and a pixel-by-pixel summation operation is performed on feature A to obtain the final output P$\in {ℛ}^{C\times H\times W }$.

$$P_j=\sigma {\displaystyle \sum }_{i=1}^N{Q}_{ji}{E}_i+A_j$$

(6)

where $\sigma$ is initialized to 0 and more new weight values are obtained by training. P selectively aggregates contexts based on the spatial attention graph, improving intra-class compactness and semantic consistency.

2.2.2. Multiscale Fusion Attention Block

In Figure 3b, the structure of the MFAB module is shown, which operates in the following steps.

(1) The high-level features $D{H}_{in}^{*}$are fed into 1 × 1 and 3 × 3 convolution layers to obtain $D{H}_{in}^{ } \in {ℛ}^{C\times H\times W }$, $D{H}_{in}^{ }$and $D{L}_{in}^{ }$ have the same number of channels. The output feature mapping u = [$u_1, u_2, \dots ,u_k$] can be computed as follows:

$$u_k=v_k\ast D_{in}={\displaystyle \sum }_{i=1}^k\left(v_k^i\right)\ast d^i$$

(7)

where $D_{in}$ = [$d^1$, $d^2$,…, $d^k$], $D_{in}\in \left(D{L}_{in} \mathrm{or} D{H}_{in}\right)$, $v_k^{ }$ = [$v_k^1$, $v_k^2$,…, $v_k^i$] denotes the convolution kernel of the channel corresponding to $D_{in}$, and * denotes the convolution.

(2) Compressing the features U and generating channel statistics are achieved using global average pooling, and the statistics are denoted as S₁ and S₂, respectively. Its kth pixel can be expressed as

$$S_{k1}=F_L\left(D{L}_{in}\right)=\frac{1}{H\times W}{\displaystyle \sum }_{i=1}^H{\displaystyle \sum }_{j=1}^W{u}_k\left(i,j\right)$$

(8)

$$S_{k2}=F_L\left(D{H}_{in}\right)=\frac{1}{H\times W}{\displaystyle \sum }_{i=1}^H{\displaystyle \sum }_{j=1}^W{u}_k\left(i,j\right)$$

(9)

where W and H denote the width and height, respectively, and $u_k$denotes the feature map of each channel.

(3) A bottleneck layer is used to limit the complexity of the model and acquire channel relationship information $x_1$ and $x_2$.

$$x_1=F_{LS}\left(S_1,R\right)={\theta }_1\left(R_1{\theta }_2\left(R_2{\theta }_1\right)\right)$$

(10)

$$x_2=F_{HS}\left(S_2,R\right)={\theta }_1\left(R_1{\theta }_2\left(R_2{\theta }_2\right)\right)$$

(11)

where $R_1$ and $R_2$denote fully connected layers, and ${\theta }_1$and ${\theta }_2$denote sigmoid and ReLU, respectively. The channel-by-channel outputs of the low-level features $x_1$ and the high-level features $x_2$ are then combined using the F_add function.

$$x=F_{add}(\cdot )=x_1+x_2$$

(12)

(4) DH_out is obtained by activating the channel relation information x and rescaling the feature U.

$$\tilde{D}{H}_{out-k}=F_{base}\left(T_k,V_k\right)=V_k{T}_k$$

(13)

where T_k$\in {ℛ}^{H\times W }$is the set of information on the scaling of the feature U, V_k is the set of channel relationship information, $\tilde{D}{H}_{out-k}$ = [$\tilde{D}{H}_{out-1}, \tilde{D}{H}_{out-2}, \dots ,\widetilde{ D}{H}_{out-k}$], and $F_{base}\left(T_k, V_k\right)$ is obtained from V_k. The feature mapping T_k$\in {ℛ}^{H\times W }$is obtained by multiplying them channel by channel. In addition, the final output $D{H}_{out}^{*}$ is obtained by connecting with $D{H}_{out}$ and $D{L}_{in}$ in this paper.

2.3. Loss Function

The dice loss function is a good choice for scenes with an imbalance between changed and unchanged samples. The function focuses more on mining the information of change regions during training, but its training is unstable for small target buildings, and in addition, it can produce gradient overfitting in extreme cases. The cross-entropy loss function measures the distance between the real and predicted outputs. The lower the entropy value is, the closer the two probability distributions are, and this value also determines the experimental training gradient. The smaller the value is, the slower the gradient updates, and the larger the gradient is, the faster the parameter updates. The update iteration of the parameter can compensate for the unstable results using a single dice loss [47] function. Therefore, a hybrid loss function with a weighted combination of dice loss and cross-entropy loss was used to optimize Siam-EMNet. It is defined as

$$\begin{array}{cc}{L}_{loss}& =L_{Dice}+L_{CE}\hfill \\ & =-\frac{1}{Z}{\displaystyle \sum }_{\mathrm{i}=1}^Z\beta p_i\log y_i+\alpha {\frac{p_i{y}_i}{p_i+y_i}}_{ }\hfill \end{array}$$

(14)

where p_i and y_i denote the predicted feature maps and ground truth, respectively, and Z denotes the batch size. Furthermore, two hyperparameters (0 < β < 5 and 0 < α < 5) are used to control the effect of the loss function. The best performance was achieved when β = 0.5 and α = 2 by adjusting the parameters during training.

3. Experimental Results and Analysis

3.1. Dataset

The LEVIR-CD [30] of the building change detection dataset published in 2020 was used. This dataset consists of 637 pairs of 1024 pixels × 1024 pixels VHR Google Earth images, with bi-temporal images from more than 20 areas in Texas, US, with a spatial resolution of 0.5 m (Figure 4). A total of 31,333 buildings of varying styles are included in the dataset. The method in this paper takes 445 pairs of images as the training set, 128 pairs for the test set, and 64 pairs for the validation set.

3.2. Experimental Design

To verify the superiority of the Siam-EMNet network in building change detection, two groups of comparative experiments were designed. Group 1 is the effect of different encoders on the model. In the Siam-EMNet network model, the pretrained EfficientNet B4 encoder structure is used (see Section 2.1). To verify its validity and reasonableness, it is compared with the encoder structures of MobileNet V2 [48], XCEP [49], ResNet34 [41], and VGG13 [40]; Group 2 is the comparison between different methods. The proposed Siam-EMNet network model is compared with BIT [10], CDNet [50], DSIFN [51], L-Unet [52], P2V-CD [53], and SNUNet [29].

To speed up training and increase training samples, the original images were cropped to 256 pixels × 256 pixels. The method was implemented on the PyTorch platform with all model environments using an NVIDIA GeForce RTX 3090, and 64 GB of RAM. We used the Adam optimizer, with an initial learning rate of 0.0001, an adjustment multiplier of 0.1, 80 iterations, and eight batches.

3.3. Evaluation Indicators

To effectively evaluate the detection performance of the Siam-EMNet network model for building change detection, the recall, precision, accuracy, and F1-score are selected as evaluation indices. The equations are shown as follows:

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(15)

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(16)

$$\mathrm{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$$

(17)

$$\mathrm{F}1\text{-}\mathrm{score}=\frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recal}}$$

(18)

TN, TP, FN, and FP in (15), (16), and (17) are the numbers of true negatives, true positives, false negatives, and false positives, respectively.

3.4. Analyzing the Experimental Results

3.4.1. Experimental Results Obtained from Different Encoders

In this paper, EfficientNet B4 is compared with some currently popular encoders, and the quantitative results in

Table 2. Quantitative assessment results of EfficientNet B4 and other encoders.

Encoder	Loss	Precision	Recall	Accuracy	F1-Score
MobileNet V2	0.04159	81.28	77.90	83.63	76.58
XCEP	0.03957	76.39	81.58	83.32	74.45
ResNet 34	0.03489	85.69	79.13	85.45	79.41
VGG 13	0.032	85.28	79.24	84.67	80.22
EfficientNet B4	0.02652	92.00	88.51	95.71	90.21

Optimal results are shown in bold.

show that EfficientNet B4 has a good performance with 92.00% precision, 88.51% recall, 95.71% accuracy, and 90.21% F1-score on the LEVIR-CD dataset. Its F1-score was improved by 13.63%, 15.76%, 10.80%, and 9.99% compared to that of MobileNet V2, XCEP, ResNet34, and VGG13, respectively. The significant effect of the EfficientNet B4 encoder on detecting building changes in the dataset is demonstrated.

3.4.2. Comparison between Different Methods

To verify the effectiveness of Siam-EMNet, some SOTA (state-of-the-art) methods were compared, including BIT, CDNet, DSIFN, L-Unet, P2V-CD, and SNUNet. A similar preprocessing method was used in all experiments, and the parameter values were adjusted to make the experimental results comparable. A comparison in

Table 3. Quantitative assessment results of different methods.

Methods	Precision	Recall	Accuracy	F1-Score
BIT	89.21	88.05	94.66	88.63
CDNet	89.93	86.85	95.02	88.36
DSIFN	94.56	80.82	94.93	87.15
L-Unet	90.60	88.46	95.44	89.52
P2V-CD	91.89	87.38	95.57	89.58
SNUNet	88.18	88.12	94.89	88.15
Siam-EMNet (ours)	92.00	88.51	95.71	90.21

Optimal results are shown in bold.

shows that the proposed method achieved the highest recall, accuracy, and F1-score of the four evaluation indicators. Precision was suboptimal, 2.56% lower than DSIFN, but the recall, accuracy, and F1-score of the proposed method were 7.69%, 0.78%, and 3.06% higher than DSIFN. Experimental results show that Siam-EMNet performs well in detecting building changes.

To further illustrate the excellent performance of the method in this paper, three scenarios were employed for elaboration in the experiments, as shown in Figure 5, Figure 6 and Figure 7 (256 pixels × 256 pixels).

According to the quantitative assessment results in

Table 4. Quantitative change detection results of large target buildings.

Methods	Precision	Recall	Accuracy	F1-Score
BIT	98.94	95.43	98.17	97.15
CDNet	98.85	97.13	98.88	98.26
DSIFN	98.97	82.09	93.98	89.73
L-Unet	99.09	97.52	98.93	98.30
P2V-CD	98.24	98.38	98.92	98.31
SNUNet	98.60	96.76	98.64	97.67
Siam-EMNet (ours)	99.62	97.76	98.99	98.68

Optimal results are shown in bold.

, the Siam-EMNet model did not achieve optimal values for the recall metrics but did achieve optimal values for precision, accuracy, and F1-score. Figure 5 shows the change detection results for the large target buildings. It can be seen from the change detection results that, for the large target building scene, the BIT, CDNet, and DSIFN models have missed detection, especially DSIFN, which has a large area of missed detection, and the building edge integrity is poor; the P2V-CD model basically identifies changing building areas and obtains the best recall in Table 4, but there are misdetections in the red box; the L-Unet and SNUNet models have more complete building detection, but poor detection in the blue box at the edge of the image, with misdetection problems; the Siam-EMNet method is highly fused with deep and shallow information, can efficiently use the edge information of buildings in VHR remote sensing images, and the detection of the building edge is more consistent with the actual edge.

Figure 6 shows the change detection results for small target buildings. From the comparison results, the BIT, DSIFN, and P2V-CD models significantly missed detection. In contrast, the performance of L-Unet, P2V-CD, and SNUNet has been improved, especially the L-Unet method, which has high precision, although there are still some missed detections. SNUNet, although it obtains the best recall, has a 3.9% lower precision than that of the proposed method. The Siam-EMNet method maintains good detection performance in some small target buildings, which improves the problems of missed detection and edge blur in the detection of small target buildings. The results of the quantitative analysis in

Table 5. Quantitative change detection results of small target buildings.

Methods	Precision	Recall	Accuracy	F1-Score
BIT	88.60	86.65	94.19	87.61
CDNet	90.72	85.44	94.67	88.00
DSIFN	91.42	83.54	94.20	87.30
L-Unet	92.51	85.79	95.02	89.02
P2V-CD	92.41	87.14	95.28	89.69
SNUNet	89.13	90.06	94.93	89.59
Siam-EMNet (ours)	93.03	89.38	95.54	90.04

Optimal results are shown in bold.

show that the proposed Siam-EMNet model has the best precision, accuracy, and F1-score, although its recall is suboptimal. As a result of combining qualitative and quantitative results, it can be concluded that Siam-EMNet provides the best overall performance.

Figure 7 shows the change detection results for dense building clusters. In Figure 7, the buildings are intricate and are susceptible to interference by shadows and adjacent ground objects in the background. From Figure 7d–j, it can be seen that all seven models can determine the location of the changing building, but there is a large gap in the building edge detection effect. The red dashed box shows that the CDNet, L-Unet, P2V-CD, and SNUNet models are highly affected by the background information and are less effective in recognizing the edges of the buildings, with evident misdetection problems. The BIT model is better at this location but has detection errors in areas where the buildings are very densely populated, such as at the blue dashed box. DSIFN shows strong performance in the detection of building changes, but the fit of building edge is low and missed detection occurs. The Siam-EMNet method effectively avoids the influence of roads and shadows in the background, detects the complete edge information of buildings, has a strong anti-interference property, and has better edge detection results.

Table 6. Quantitative change detection results of dense building clusters.

Methods	Precision	Recall	Accuracy	F1-Score
BIT	89.77	92.84	93.98	91.28
CDNet	89.22	93.89	94.47	91.49
DSIFN	97.46	87.11	95.13	91.99
L-Unet	93.68	92.30	95.55	92.98
P2V-CD	93.83	91.87	95.47	92.84
SNUNet	89.02	93.14	93.63	91.03
Siam-EMNet (ours)	93.63	94.68	96.24	94.15

Optimal results are shown in bold.

shows the quantitative assessment results of the seven models. In Table 6, although the DSIFN method precision is better than the proposed method, the proposed method’s recall, accuracy, and F1-score are optimal, with better overall performance than the other six methods.

4. Discussion

(1)

Validation of model generalization

A single dataset cannot measure the generalizability of the model. In addition to the LEVIR-CD dataset, experiments were also performed on the WHU-CD [54] change detection dataset, which is provided by Shunping Ji’s team at Wuhan University, with a spatial resolution of 0.075 m and a spatial size of 32,507 pixels × 15,354 pixels, and the original image is cropped to 256 pixels × 256 pixels by preprocessing. The method in this paper uses 3107 pairs of images as the training set, in addition to 923 pairs for the test set and 433 pairs for the validation set. The quantitative assessment results of the WHU-CD dataset are shown in

Table 7. Quantitative assessment results of WHU-CD dataset.

Methods	Precision	Recall	Accuracy	F1-Score
BIT	85.39	91.18	94.32	88.19
CDNet	82.72	88.42	93.78	85.47
DSIFN	95.85	81.80	94.69	88.27
L-Unet	66.31	84.99	89.12	74.50
P2V-CD	93.22	88.01	95.38	90.54
SNUNet	90.85	89.87	95.03	90.36
Siam-EMNet (ours)	92.22	93.46	95.92	92.77

Optimal results are shown in bold.

, with 92.22%, 93.46%, 95.92%, and 92.77% for the Siam-EMNet method’s precision, recall, accuracy, and F1-score, respectively. Compared to SOTA methods, Siam-EMNet has an optimal recall, accuracy, and F1-score, and precision is only lower than DSIFN; however, the recall, accuracy, and F1-score of the proposed method were 11.66%, 1.23%, and 4.5% higher than DSIFN. The higher the recall, the lower the missed rate of the model, and consequently the more complete the detection of the change regions. Using two datasets, the proposed network model is validated for better generalization.

(2)

Effect of different versions of EfficientNet on change detection results

EfficientNet has eight versions of EfficientNet B0-B7. Different versions of EfficientNet have different abilities to extend depth, width, and resolution, and have certain requirements on image resolution and feature type as well as computer capability. To explore the performance difference of different versions of EfficientNet in building change detection, this paper uses the EfficientNet B0–B6 structure as the encoder. Due to the fact that EfficientNet B7 has relatively high computing power requirements, it is not used as a comparative experiment. As seen from the quantitative assessment results in

Table 8. Quantitative assessment results of EfficientNet B0–B6.

Encoder	Loss	Precision	Recall	Accuracy	F1-Score
EfficientNet B0	0.03189	84.96	80.41	84.07	80.17
EfficientNet B1	0.0289	85.57	82.39	85.33	82.40
EfficientNet B2	0.0291	85.49	82.08	84.72	82.27
EfficientNet B3	0.02938	87.42	80.53	85.81	82.17
EfficientNet B4	0.02652	92.00	88.51	95.71	90.21
EfficientNet B5	0.02735	84.88	82.12	88.34	82.28
EfficientNet B6	0.02708	86.19	83.71	89.11	83.71

Optimal results are shown in bold.

, the EfficientNet B4 architecture selected in this paper has the best performance in detecting changes in buildings.

(3)

Effect of different loss functions the change detection results

In this paper, the effect of hybrid loss functions on model performance is explored through ablation experiments. As seen from the quantitative assessment results in

Table 9. Quantitative assessment results of the loss function.

Loss	Precision	Recall	Accuracy	F1-Score
LCE	93.09	84.00	93.68	88.08
LDice	88.80	86.96	92.11	86.77
LDice + LCE	92.00	88.51	95.71	90.21

Optimal results are shown in bold.

, when the cross-entropy loss function is used alone, precision is slightly higher than the hybrid loss function used by the proposed method, but the recall, accuracy, and F1-score values are lower than the hybrid loss function used. Therefore, the combination of the dice loss function and the cross-entropy loss function optimizes the proposed model to the greatest extent and reduces the detection error caused by the imbalance between the changed and unchanged samples.

5. Conclusions

A VHR remote sensing image building change detection network based on Siamese EfficientNet B4-MANet (Siam-EMNet) was proposed to solve the problems of the incomplete detection of change areas and rough edges in the field of building change detection. The encoder structure compound expands the width, depth, and resolution of the network and enhances the ability to extract building feature information. The decoder structure integrates PAB and MFAB, enhancing the detection of building edge details and effectively avoiding the missed detection of small regions. The encoder can effectively extract the bi-temporal feature information of VHR remote sensing images by skipping connections between the encoder and decoder to achieve efficient multilevel information fusion. The results obtained using the LEVIR-CD dataset show that Siam-EMNet is effective for building change detection with a precision, recall, accuracy, and F1-score of 92.00%, 88.51%, 95.71%, and 90.21%, respectively. Compared to BIT, CDNet, DSIFN, L-Unet, P2V-CD, and SNUNet, Siam-EMNet achieves the best comprehensive performance. Additionally, the WHU-CD dataset was used to verify the generalizability of the network model, and the results show that the proposed network model has a good generalization. Despite the outstanding advantages of EfficientNet B4 in terms of accuracy, the model itself has a large number of parameters. Therefore, in future research, the number of parameters of the model can be reduced by pruning to improve the adaptability of the model.