Search Results (111)

The objective of pan-sharpening is to effectively fuse high-resolution panchromatic (PAN) images with limited spectral information and low-resolution multispectral (LR-MS) images, thereby generating a fused image with a high spatial resolution and rich spectral information. However, current fusion techniques face significant challenges, including insufficient edge detail, spectral distortion, increased noise, and limited robustness. To address these challenges, we propose a multi-frequency spectral–spatial interaction enhancement network (MFSINet) that comprises the spectral–spatial interactive fusion (SSIF) and multi-frequency feature enhancement (MFFE) subnetworks. The SSIF enhances both spatial and spectral fusion features by optimizing the characteristics of each spectral band through band-aware processing. The MFFE employs a variant of wavelet transform to perform multiresolution analyses on remote sensing scenes, enhancing the spatial resolution, spectral fidelity, and the texture and structural features of the fused images by optimizing directional and spatial properties. Moreover, qualitative analysis and quantitative comparative experiments using the IKONOS and WorldView-2 datasets indicate that this method significantly improves the fidelity and accuracy of the fused images. Full article

Image preprocessing and fusion are commonly used for enhancing remote-sensing images, but the resulting images often lack useful spatial features. As the majority of research on image fusion has concentrated on the satellite domain, the image-fusion task for Unmanned Aerial Vehicle (UAV) images has received minimal attention. This study investigated an image-improvement strategy by integrating image preprocessing and fusion tasks for UAV images. The goal is to improve spatial details and avoid color distortion in fused images. Techniques such as image denoising, sharpening, and Contrast Limited Adaptive Histogram Equalization (CLAHE) were used in the preprocessing step. The unsharp mask algorithm was used for image sharpening. Wiener and total variation denoising methods were used for image denoising. The image-fusion process was conducted in two steps: (1) fusing the spectral bands into one multispectral image and (2) pansharpening the panchromatic and multispectral images using the PanColorGAN model. The effectiveness of the proposed approach was evaluated using quantitative and qualitative assessment techniques, including no-reference image quality assessment (NR-IQA) metrics. In this experiment, the unsharp mask algorithm noticeably improved the spatial details of the pansharpened images. No preprocessing algorithm dramatically improved the color quality of the enhanced images. The proposed fusion approach improved the images without importing unnecessary blurring and color distortion issues. Full article

Beijing Satellite 3 is a high-performance optical remote sensing satellite with a spatial resolution of 0.3–0.5 m. It can provide timely and independent ultra-high-resolution spatial big data and comprehensive spatial information application services. At present, there is no relevant research on the fusion method of BJ-3A satellite images. In many applications, high-resolution panchromatic images alone are insufficient. Therefore, it is necessary to fuse them with multispectral images that contain spectral color information. Currently, there is a lack of research on the fusion method of BJ-3A satellite images. This article explores six traditional pixel-level fusion methods (HPF, HCS, wavelet, modified-IHS, PC, and Brovey) for fusing the panchromatic image and multispectral image of the BJ-3A satellite. The fusion results were analyzed qualitatively from two aspects: spatial detail enhancement capability and spectral fidelity. Five indicators, namely mean, standard deviation, entropy, correlation coefficient, and average gradient, were used for quantitative analysis. Finally, the fusion results were comprehensively evaluated from three aspects: spectral curves of ground objects, absolute error figure, and object-oriented classification effects. The findings of the research suggest that the fusion method known as HPF is the optimum and appropriate technique for fusing panchromatic and multispectral images obtained from BJ-3A. These results can be utilized as a guide for the implementation of BJ-3A panchromatic and multispectral data fusion in real-world scenarios. Full article

Pan-sharpening is a fusion process that combines a low-spatial resolution, multi-spectral image that has rich spectral characteristics with a high-spatial resolution panchromatic (PAN) image that lacks spectral characteristics. Most previous learning-based approaches rely on the scale-shift assumption, which may not be applicable in the full-resolution domain. To solve this issue, we regard pan-sharpening as a multi-task problem and propose a Siamese network with Gradient-based Spatial Attention (GSA-SiamNet). GSA-SiamNet consists of four modules: a two-stream feature extraction module, a feature fusion module, a gradient-based spatial attention (GSA) module, and a progressive up-sampling module. In the GSA module, we use Laplacian and Sobel operators to extract gradient information from PAN images. Spatial attention factors, learned from the gradient prior, are multiplied during the feature fusion, up-sampling, and reconstruction stages. These factors help to keep high-frequency information on the feature map as well as suppress redundant information. We also design a multi-resolution loss function that guides the training process under the constraints of both reduced- and full-resolution domains. The experimental results on WorldView-3 satellite images obtained in Moscow and San Juan demonstrate that our proposed GSA-SiamNet is superior to traditional and other deep learning-based methods. Full article

Unmanned aerial vehicles (UAVs) are extensively used to gather remote sensing data, offering high image resolution and swift data acquisition despite being labor-intensive. In contrast, satellite-based remote sensing, providing sub-meter spatial resolution and frequent revisit times, could serve as an alternative data source for phenotyping. In this study, we separately evaluated pan-sharpened Pléiades satellite imagery (50 cm) and UAV imagery (2.5 cm) to phenotype durum wheat in small-plot (12 m × 1.10 m) breeding trials. The Gaussian process regression (GPR) algorithm, which provides predictions with uncertainty estimates, was trained with spectral bands and а selected set of vegetation indexes (VIs) as independent variables. Grain protein content (GPC) was better predicted with Pléiades data at the growth stage of 20% of inflorescence emerged but with only moderate accuracy (validation R²: 0.58). The grain yield (GY) and protein yield (PY) were better predicted using UAV data at the late milk and watery ripe growth stages, respectively (validation: R² 0.67 and 0.62, respectively). The cumulative VIs (the sum of VIs over the available images within the growing season) did not increase the accuracy of the models for either sensor. When mapping the estimated parameters, the spatial resolution of Pléiades revealed certain limitations. Nevertheless, our findings regarding GPC suggested that the usefulness of pan-sharpened Pléiades images for phenotyping should not be dismissed and warrants further exploration, particularly for breeding experiments with larger plot sizes. Full article

Achieving a balance between spectral resolution and spatial resolution in multi-spectral remote sensing images is challenging due to physical constraints. Consequently, pan-sharpening technology was developed to address this challenge. While significant progress was recently achieved in deep-learning-based pan-sharpening techniques, most existing deep learning approaches face two primary limitations: (1) convolutional neural networks (CNNs) struggle with long-range dependency issues, and (2) significant detail loss during deep network training. Moreover, despite these methods’ pan-sharpening capabilities, their generalization to full-sized raw images remains problematic due to scaling disparities, rendering them less practical. To tackle these issues, we introduce in this study a multi-spectral remote sensing image fusion network, termed TAMINet, which leverages a two-stream coordinate attention mechanism and multi-detail injection. Initially, a two-stream feature extractor augmented with the coordinate attention (CA) block is employed to derive modal-specific features from low-resolution multi-spectral (LRMS) images and panchromatic (PAN) images. This is followed by feature-domain fusion and pan-sharpening image reconstruction. Crucially, a multi-detail injection approach is incorporated during fusion and reconstruction, ensuring the reintroduction of details lost earlier in the process, which minimizes high-frequency detail loss. Finally, a novel hybrid loss function is proposed that incorporates spatial loss, spectral loss, and an additional loss component to enhance performance. The proposed methodology’s effectiveness was validated through experiments on WorldView-2 satellite images, IKONOS, and QuickBird, benchmarked against current state-of-the-art techniques. Experimental findings reveal that TAMINet significantly elevates the pan-sharpening performance for large-scale images, underscoring its potential to enhance multi-spectral remote sensing image quality. Full article

Pan-sharpening is an important means to improve the spatial resolution of multispectral (MS) images. Although a large number of pan-sharpening methods have been developed, improving the spatial resolution of MS while effectively maintaining its spectral information has not been well solved so far, and it has also been taken as a criterion to measure whether the sharpened product can meet the practical needs. The back-projection (BP) method iteratively injects spectral information backwards into the sharpened results in a post-processing manner, which can effectively improve the generally unsatisfied spectral consistency problem in pan-sharpening methods. Although BP has received some attention in recent years in pan-sharpening research, the existing related work is basically limited to the direct utilization of the BP process and lacks a more in-depth intrinsic integration with pan-sharpening. In this paper, we analyze the current problems of improving the spectral consistency based on BP in pan-sharpening, and the main innovative works carried out on this basis include the following: (1) We introduce the spatial consistency condition and propose the spatial–spectral BP (SSBP) method, which takes into account both spatial and spectral consistency conditions, to improve the spectral quality while effectively solving the problem of spatial distortion in the results. (2) The proposed SSBP method is analyzed theoretically, and the convergence condition of SSBP and a more relaxed convergence condition for a specific BP type, degradation transpose BP, are given and proved theoretically. (3) Fast computation of BP and SSBP is investigated, and non-iterative fast BP (FBP) and fast SSBP algorithms (FSSBP) methods are given in a closed-form solution with significant improvement in computational efficiency. Experimental comparisons with combinations formed by seven different BP-related post-processing methods and up to 18 typical base methods show that the proposed methods are generally applicable to the optimization of the spatial–spectral quality of various sharpening methods. The fast method improves the computational speed by at least 27.5 times compared to the iterative version while maintaining the evaluation metrics well. Full article

Salt marshes are highly important wetlands; however, external pressures are causing their widespread deterioration and loss. Continuous monitoring of their extent is paramount for the preservation and recovery of deteriorated and threatened salt marshes. In general, moderate-resolution satellite remote sensing data allow for the accurate detection of salt marsh shorelines; however, their detection in narrow and fringing salt marshes remains challenging. This study aims to evaluate the ability of Landsat-5 (TM), Landsat-7 (ETM+), and Sentinel-2 (MSI) data to be used to accurately determine the shoreline of narrow and fringing salt marshes, focusing on three regions of the Aveiro lagoon in Mira, Ílhavo and S. Jacinto channels. Shorelines were determined considering the Normalized Difference Vegetation Index (NDVI), and the accuracy of this methodology was evaluated against reference shorelines by computing the Root Mean Square Error (RMSE). Once validated, the method was used to determine historical salt marsh shorelines, and rates of change between 1984 and 2022 were quantified and analyzed in the three locations. Results evidence that the 30 m resolution Landsat data accurately describe the salt marsh shoreline (RMSE~15 m) and that the accuracy is maintained when increasing the spatial resolution through pan-sharpening or when using 10 m resolution Sentinel-2 (MSI) data. These also show that the salt marshes of the Ílhavo and S. Jacinto channels evolved similarly, with salt marsh shoreline stability before 2000 followed by retreats after this year. At the end of the four decades of study, an average retreat of 66.23 ± 1.03 m and 46.62 ± 0.83 m was found, respectively. In contrast to these salt marshes and to the expected evolution, the salt marsh of the Mira Channel showed retreats before 2000, followed by similar progressions after this year, resulting in an average 2.33 ± 1.18 m advance until 2022. Full article

Pan-sharpening aims to create high-resolution spectrum images by fusing low-resolution hyperspectral (HS) images with high-resolution panchromatic (PAN) images. Inspired by the Swin transformer used in image classification tasks, this research constructs a three-stream pan-sharpening network based on the Swin transformer and a multi-scale feature extraction module. Unlike the traditional convolutional neural network (CNN) pan-sharpening model, we use the Swin transformer to establish global connections with the image and combine it with a multi-scale feature extraction module to extract local features of different sizes. The model combines the advantages of the Swin transformer and CNN, enabling fused images to maintain good local detail and global linkage by mitigating distortion in hyperspectral images. In order to verify the effectiveness of the method, this paper evaluates fused images with subjective visual and quantitative indicators. Experimental results show that the method proposed in this paper can better preserve the spatial and spectral information of images compared to the classical and latest models. Full article

Rapid and accurate estimation of forest aboveground biomass (AGB) with fine details is crucial for effective forest monitoring and management, where forest height plays a key role in AGB quantification. In this study, we propose a random forest (RF)-based down-scaling method to map forest height and biomass at a 15-m resolution by integrating Landsat 8 OLI and Ice, Cloud and Land Elevation Satellite-2 (ICESat-2) LiDAR data. ICESat-2 photon data are used to derive canopy parameters along 15-m segments, which are considered sample plots for the extrapolation of discrete forest height. Fourteen variables associated with spectral features, textual features and vegetation index are extracted from pan-sharpened Landsat 8 images. A regression function is established between these variables and ICESat-2-derived forest height to produce a 15-m continuous forest height distribution data based on the 30-m forest height product using the RF algorithm. Finally, a wall-to-wall forest AGB at 15-m spatial resolution is achieved by using an allometric model specific to the forest type and height. The Jilin Province in northeast China is taken as the study area, and the forest AGB estimation results reveal a density of 61.15 Mg/ha with a standard deviation of 89.46 Mg/ha. The $R^2$ between our predicted forest heights and the ICESat-2-derived heights reaches 0.93. Validation results at the county scale demonstrate reasonable correspondence between the estimated AGB and reference data, with consistently high $R^2$ value exceeding 0.65. This downscaling method provides a promising scheme to estimate spatial forest AGB with fine details and to enhance the accuracy of AGB estimation, which may facilitate carbon stock measurement and carbon cycle studies. Full article

Based on deep learning, various pan-sharpening models have achieved excellent results. However, most of them adopt simple addition or concatenation operations to merge the information of low spatial resolution multi-spectral (LRMS) images and panchromatic (PAN) images, which may cause a loss of detailed information. To tackle this issue, inspired by capsule networks, we propose a plug-and-play layer named modified dynamic routing layer (MDRL), which modifies the information transmission mode of capsules to effectively fuse LRMS images and PAN images. Concretely, the lower-level capsules are generated by applying transform operation to the features of LRMS images and PAN images, which preserve the spatial location information. Then, the dynamic routing algorithm is modified to adaptively select the lower-level capsules to generate the higher-level capsule features to represent the fusion of LRMS images and PAN images, which can effectively avoid the loss of detailed information. In addition, the previous addition and concatenation operations are illustrated as special cases of our MDRL. Based on MIPSM with addition operations and DRPNN with concatenation operations, two modified dynamic routing models named MDR–MIPSM and MDR–DRPNN are further proposed for pan-sharpening. Extensive experimental results demonstrate that the proposed method can achieve remarkable spectral and spatial quality. Full article

In recent years, the demand for very high geometric resolution satellite images has increased significantly. The pan-sharpening techniques, which are part of the data fusion techniques, enable the increase in the geometric resolution of multispectral images using panchromatic imagery of the same scene. However, it is not trivial to choose a suitable pan-sharpening algorithm: there are several, but none of these is universally recognized as the best for any type of sensor, in addition to the fact that they can provide different results with regard to the investigated scene. This article focuses on the latter aspect: analyzing pan-sharpening algorithms in relation to different land covers. A dataset of GeoEye-1 images is selected from which four study areas (frames) are extracted: one natural, one rural, one urban and one semi-urban. The type of study area is determined considering the quantity of vegetation included in it based on the normalized difference vegetation index (NDVI). Nine pan-sharpening methods are applied to each frame and the resulting pan-sharpened images are compared by means of spectral and spatial quality indicators. Multicriteria analysis permits to define the best performing method related to each specific area as well as the most suitable one, considering the co-presence of different land covers in the analyzed scene. Brovey transformation fast supplies the best results among the methods analyzed in this study. Full article

Improving the accuracy of DEMs is a critical goal in digital terrain analysis. The combination of multi-source data can be used to increase DEM accuracy. Five typical geomorphic study areas in the Loess Plateau in Shaanxi were selected for a case study and a 5 m DEM unit was used as the basic data input. Data from three open-source databases of DEM images, the ALOS, SRTM and ASTER, were obtained and processed uniformly through a previously geographical registration process. Three methods, Gram–Schmidt pan sharpening (GS), weighted fusion and feature-point-embedding fusion, were used for mutual enhancement of the three kinds of data. We combined the effect of these three fusion methods in the five sample areas and compared the eigenvalues taken before and after the fusion. The main conclusions are as follows: (1) The GS fusion method is convenient and simple, and the three combined fusion methods can be improved. Generally speaking, the fusion of ALOS and SRTM data led to the best performance, but was greatly affected by the original data. (2) By embedding feature points into three publicly available types of DEM data, the errors and extreme error value of the data obtained through fusion were significantly improved. Overall, ALOS fusion resulted in the best performance because it had the best raw data quality. The original eigenvalues of the ASTER were all inferior and the improvement in the error and the error extreme value after fusion was evident. (3) By dividing the sample area into different areas and fusing them separately according to the weights of each area, the accuracy of the data obtained was significantly improved. In comparing the improvement in accuracy in each region, it was observed that the fusion of ALOS and SRTM data relies on a gentle area. A high accuracy of these two data will lead to a better fusion. Merging ALOS and ASTER data led to the greatest increase in accuracy, especially in the areas with a steep slope. Additionally, when SRTM and ASTER data were merged, the observed improvement was relatively stable with little difference. Full article

LNC (leaf nitrogen content) in crops is significant for diagnosing the crop growth status and guiding fertilization decisions. Currently, UAV (unmanned aerial vehicles) remote sensing has played an important role in estimating the nitrogen nutrition of crops at the field scale. However, many existing methods of evaluating crop nitrogen based on UAV imaging techniques usually have used a single type of imagery such as RGB or multispectral images, seldom considering the usage of information fusion from different types of UAV imagery for assessing the crop nitrogen status. In this study, GS (Gram–Schmidt Pan Sharpening) was utilized to fuse images from two sensors of digital RGB and multispectral cameras mounted on UAV, and the specific bands of the multispectral cameras are blue, green, red, rededge and NIR. The color space transformation method, HSV (Hue-Saturation-Value), was used to separate soil background noise from crops due to the high spatial resolution of UAV images. Two methods of optimizing feature variables, the Successive Projection Algorithm (SPA) and the Competitive Adaptive Reweighted Sampling method (CARS), combined with two regularization regression algorithms, LASSO and RIDGE, were adopted to estimate the LNC, compared to the commonly used Random Forest algorithm. The results showed that: (1) the accuracy of LNC estimation using the fusion image is improved distinctly by a comparison to the original multispectral image; (2) the denoised images performed better than the original multispectral images in evaluating LNC in rice; (3) the RIDGE-SPA combined method, using SPA to select the MCARI, SAVI and OSAVI, had the best performance for LNC in rice, with an R² of 0.76 and an RMSE of 10.33%. It can be demonstrated that the information fusion of multiple-sensor imagery from UAV coupling with the methods of optimizing feature variables can estimate the rice LNC more effectively, which can also provide a reference for guiding the decision making of fertilization in rice fields. Full article

This study aims to investigate potential archaeological proxies at a large Bronze Age fortification in Hungary, namely the Csanádpalota–Juhász T. tanya site, using open-access satellite data. Available Sentinel-2 images acquired between April 2017 and September 2022 were used. More than 700 images (727) were initially processed and filtered, accounting at the end of more than 400 (412) available calibrated Level 2A Sentinel images over the case study area. Sentinel-2 images were processed through image analysis. Based on pan-sharpened data, the visibility of crop marks was improved and enhanced by implementing orthogonal equations. Several crop marks, some still unknown, were revealed in this study. In addition, multi-temporal phenological observations were recorded on three archaeological proxies (crop marks) within the case study area, while an additional area was selected for calibration purposes (agricultural field). Phenological observations were performed for at least four complete phenological cycles throughout the study period. Statistical comparisons between the selected archaeological proxies were applied using a range of vegetation indices. The overall results indicated that phenological observations could be used as archaeological proxies for detecting the formation of crop marks. Full article