World Applied Sciences Journal 4 (3): 363-370, 2008

ISSN 1818-4952
© IDOSI Publications, 2008

Enhancement of Cloud-Associated Shadow Areas in

Satellite Images Using Wavelet Image Fusion
1
A. Abd-Elrahman, 2I.F. Shaker, 3A.K. Abdel-Gawad and 3A. Abdel-Wahab

1
School of Forest Resources and Conservation, Geomatics, University of Florida,
2
Public Works Department, Ain Shams University, Cairo, Egypt
3
Department of Civil Eng., Engineering Research Division, National Research Center, Cairo, Egypt

Abstract: Shadows created by small patches of clouds obscure ground features from satellite imagery. In this
research, a new technique was developed to enhance cloud-associated shadow areas in satellite images while
preserving details underneath these areas. The developed algorithm utilizes wavelet analysis to decompose
cloudy images into several frequency level components. Image details underneath shadow areas are preserved
in the output image by fusing the cloudy image with another cloud-free image. The developed technique is
implemented on a cloudy Landsat7 Panchromatic subscene. The results showed that the developed technique
was successful in enhancing the cloudy image through preserving the obscured details underneath the cloud-
associated shadow. The ability to maintain such details under shadow areas and the amount of introduced
artifacts increased with the increase of the wavelet decomposition level. Generally, two or three wavelet
decomposition levels were found to be sufficient for the analysis used in this study. The results obtained when
using two images with the same spatial resolution were found to be better than those obtained using images
with different spatial resolution.

Key words: Wavelet % Fusion % Shadow % Frequency decomposition % Landsat

INTRODUCTION resolution imagery and suffer costly computational

overhead in addition to shadow edge processing
Space satellite imagery is very cost effective in problems. Other methods utilized overlapping imagery
mapping extended regions. This imagery is historically to detect occlusion and remove shadows [4]. Fusion
utilized in mapping roads, hydrological features, techniques were also used to account for cloud and
vegetation, etc. The existence of high quality images shadow defects on certain image using different
enabled mapping changes in land development in the cloud/shadow free images [5, 6]. Although these methods
last two decades. Nevertheless, one of the problems that are widely used, they tend to ignore the spatial content
hindered the use of such imagery is the existence of underneath shadow areas. Hence, this research is devoted
clouds and their associated shadows. Although cloud to developing a technique to reduce the effect of cloud-
areas can easily be detected and replaced from other associated shadows using image wavelet decomposition
imagery due to their high reflectance, the shadows caused techniques. Although this research deals with a specific
by these clouds represent areas with low illumination type of shadow, the developed technique could be
conditions that double the image defected areas. This extended to handle other types of shadows resulting from,
effect of clouds and cloud-associated shadows are widely for example, high buildings and trees.
spread with approximately 66% of the earth surface is A milestone in developing such a technique is to
covered by clouds throughout the year [1]. be able to automatically detect clouds and shadows in
Several methods were used to restore the image the images. Features utilized for cloud and shadow
content in shadow areas. Some of these methods detection can be classified into three domains [7]:
identifies and removes the image illumination variations spectral, spatial and temporal. Spectral features, such as
using surface reflectance and variations constraints [2, 3]. image gray levels or color information, are commonly
Such methods were implemented mostly on high spatial used. Some approaches improve results by using spatial

Corresponding Author: Dr. A. Abd-Elrahman, School of Forest Resources and Conservation, Geomatics, University of Florida,
363
 World Appl. Sci. J., 4 (3): 363-370, 2008

information working on a regional level, instead of pixel ψ (2 j −1 x) = ∑ g j +1 (k )φ (2 j x − k ) (2)

k
level. Some other methods utilize temporal redundancy to
integrate and improve results. Where h(k) and g(k) are low and high pass filters
Many algorithms are available to detect clouds and respectively. Any function f(x) can be represented by
their associated shadows. Concepts such as simple given scaling function and its wavelet derivatives as:
thresholding, neural networks, fuzzy logic, etc were
utilized in these algorithms [8]. Scene properties such f ( x) = ∑ c j −10 ( k )φ (2 j −1 x − k ) + ∑ d j −1 (k )ψ j , k (2 j −1 x − k )
k k
as scene date and time, sensor and sun locations, etc, (3)
although not used in this implementation, could also
be utilized in the cloud and shadow detection process. For orthonormal scaling and wavelet functions the
The methodology applied in this research utilizes scaling (approximation) and wavelet (detail) coefficients
cloud/shadow geometric constraints in addition to their can be represented in terms of their values in a previous
spectral characteristics [9]. scale as follows:
One of the objectives of the methodology
developed in this research is to enhance shadow areas by c j −1 (k ) = ∑ h(m − 2k )c j ( m) (4)
m
preserving details underneath these shadows. A wavelet
decomposition technique is suggested to achieve this d j −1 ( k ) = ∑ g (m − 2k )d j ( m) (5)
m
objective by fusing this image with another cloud-free
image. The results of this technique depend on the Recalling that the scaling filter is a low pass filter and
accuracy of the cloud/shadow detection process and on the wavelet filter is a high pass filter, it can be concluded
the fusion algorithm. that obtaining the approximation and detail coefficient
It should be mentioned here that this research is constitutes a single step in a an iterative filter bank that
only concerned with shadow areas. Unlike shadow areas, results in multiple level decomposition of the signal. This
cloud regions are generally opaque and do not have iterative filter bank forms the basis of the discrete wavelet
image details underneath. Accordingly, the techniques transform. A reverse operation can also be used to
developed in this study, which utilizes wavelet image completely reconstruct the signal. In image analysis, a
fusion to capture details information, was applied only in generalized form of the one-dimensional discrete wavelet
shadow areas. On the other hand, to enhance the overall transform can be used. This means that four components
output image, clouds were substituted for directly using are obtained at each wavelet decomposition level. These
traditional pixel to pixel replacement from another cloud- components are calculated through filtering the image
free image. using high and low pass filters.
For an image f(x, y) of size M*N, at each
BACKGROUND decomposition level, the results are one smooth
image resulting from the filtering of the input image
In this section background information related to using low pass filters in the horizontal and vertical
wavelet image decomposition and image fusion is directions. The resulting component is denoted fLL(x, y)
reviewed. and called approximation subimage. Three other details
subimages fLH(x, y) fHL(x, y) and fHH(x, y) result from
Discrete Wavelet Transform: Wavelet decomposition filtering the original images by a combination of low
and reconstruction was historically used in image and high pass filters in the horizontal and vertical
fusion and image compression applications. Discrete directions. These three subimages represent
wavelet transform will be utilized in this research to decomposition details in the horizontal, vertical and
enhance cloud-related shadow areas by adjusting their diagonal directions, respectively. It should be mentioned
frequency content. For discrete wavelet transform, let here that the size of the obtained subimages at each
φ jk = φ (2 j x − k ) and ψ jk = ψ (2 j x − k ) be sets of dilated decomposition level is half the size of the original image.
and scaled mother and wavelet functions, respectively. The approximation subimage could, in turn, be composed
Both functions can be constructed from the higher level to produce the next image decomposition level.
scaling functions [10]:
Image Fusion: Generally, image fusion can be defined as
φ (2 j −1
x) = ∑ h j +1 (k )φ (2 x − k )
j
(1) the combination of two or more images to form a new
k

364
 World Appl. Sci. J., 4 (3): 363-370, 2008

image using certain algorithms [11]. Fusing two images in

order to get rid of defective parts in one of the images is
a typical image fusion application. Image fusion is
performed at three different processing levels according
to the stage at which the fusion takes place [12]. These
three processing levels are pixel, feature and decision
levels. Image fusion at pixel level means fusion at the
lowest processing level, referring to the merging of
measured physical parameters. Most research utilizing
image fusion techniques to correct for image defects
tackle pixel-based image fusion, which presumes
accurately registered imagery [13].
Fusion at the feature level requires the extraction of
objects recognized in the various data sources, e.g.,
using segmentation procedures. Features correspond to
characteristics extracted from the initial images, which
depend on the feature’s environment such as extent,
shape and neighborhood [14]. Decision level fusion is
a method that uses interpreted data where the input Fig. 1: Features partially obscured by cloud shadows
images are processed individually for information
Defected Image Cloud-free Image
extraction. The obtained information is then combined
by applying decision rules to reinforce common
HPF LPF HPF LPF
interpretation, resolve differences and furnish a better
understanding of the observed objects [15]. - - Wavelet
Decomposition
- -

Details Approx Details Approx

METHODOLOGY

Many of the techniques developed to enhance HPF’

W avelet LPF’

cloud-associated shadows ignore the information in the Reconstruction

+ +
shadow regions. This is very clear when visually
inspecting shadow areas, where energy indirectly
Enhanced
reaching these areas is reflected to the sensor causing Image
some details to appear in the image. Figure 1 presents
parts of a runway and road intersection that are partially Fig. 2: Schematic representation of developed
shown in shadow areas. methodology
In our developed algorithm, the defected cloudy
image is fused with another cloud-free image to form image. A technique developed by [9] that utilizes not only
a new enhanced image from the high frequency the spectral properties of the clouds and shadows, but
components of the cloudy image and the low frequency also incorporates the spatial relationships between
information of the cloud-free image. Details shown in them is suggested. In this technique, the results of a
shadow areas due to energy indirectly reaching these neural network classification process, which mainly
areas were conserved. Figures 2 presents schematic depends on performing automatic classification based on
diagrams of the methodology adopted in this image spectral properties, are improved by defining the
approach. It should be mentioned here that the spatial relationship between cloud patches and their
developed methodology of capturing and preserving corresponding shadow patches.
high frequency component was applied in shadow areas A mathematical model for the spatial relationship is
only. Areas identified as clouds were simply replaced by proposed and a least squares adjustment process is
corresponding areas in the cloud-free image. performed to estimate model parameters. Low spectral
threshold is used to define shadow pixels resulting on
Cloud/shadow Detection Algorithm: Cloud/shadow more pixels mistakenly identified as shadow. The model is
detection is the first step towards producing an enhanced then used to filter out non-shadow pixels.

365
 World Appl. Sci. J., 4 (3): 363-370, 2008

Image Fusion Using Discrete Wavelet Transform: The

developed methodology suggests that both defected and
cloud-free images are composed using discrete
wavelet transform to create different high and low
frequency components. The high frequency component
contains image details such as noise, edges and details.
On the other hand, the low frequency (approximation)
components contain basic image information. Considering
wavelet decomposition of the shadow areas of an image,
it could be easily shown that the details components
contain image information located in these areas.
Information beneath shadows could be preserved if
the details information in these areas were used in the
image reconstruction process while neglecting the
approximation component. However, neglecting the
approximation component produces an image with only
high frequency information. To solve this problem, the Fig. 3: Study area-Nov 2001 Landsat 7 ETM
approximation component of another cloud-free image panchromatic sub-scene
was suggested to replace the approximation component
of the defective image dominated primarily by the cloud-
associated shadows. The developed wavelet-based image
fusion technique is represented schematically in figure 2.

EXPERIMENTS AND RESULTS

Test Data: The developed technique is implemented on

the panchromatic band of a November 2001 Landsat 7
ETM satellite subscenes, shown in Figure 3. Features in
these images are partially obscured by cloud-associated
shadows. For simplicity, boundary problems were solved
by allowing the application to process the original scene
outside the boundaries of the test subscenes when this is
needed to perform the analysis.
Two cloud-free images, shown in Figure 4,5, were
tested to enhance shadow areas in the defected cloudy
image using wavelet fusion technique. The first cloud-free Fig. 4: First cloud-free images-July 2000 Landsat7 ETM
image is another ETM Panchromatic image of the same panchromatic subscene
resolution taken in July 2000. The second test image is a
30 meters resolution Landsat ETM taken in May 2001. computed values for the shadow classification results
This image was pre-processed by conversion to gray level commission and omission errors were 11% and 9%,
image and resampling to 15 meter pixel size. respectively.

Shadow Detection: The cloud/shadow detection algorithm Shadow Enhancement Using Discrete Wavelet Analysis:
described in section 3.1 was implemented on the Two cloud-free images were used to implement the
cloudy image. This algorithm utilized image spectral shadow removal technique using image fusion approach.
characteristics and cloud/shadow spatial constraints for The first cloud-free image is another ETM Panchromatic
the detection. The results of the cloud and shadow image of the same resolution. The second image is a 30
identification step were two binary masks, one for clouds (resampled to 15) meter resolution Landsat ETM image
and the other for shadow regions. Figures 6 and 7 composed of the three visible bands and converted to
show the binary image of detected clouds and shadows gray level image using band averaging algorithm.
for the study areas. Using manual digitizing as reference, These two images were shown in Figures 4 and 5 above.

366
 World Appl. Sci. J., 4 (3): 363-370, 2008

Fig. 5: Second cloud-free image–May 2000 Landsat Fig. 7: Binary image for detected shadows in the
ETM visible bands subscene converted to gray study area
level image

Fig. 8: Third level ‘Bior 2,2’ wavelet decomposition of

defected image
Fig. 6: Binary image for detected clouds in the study area
frequency component of the wavelet decomposition of
The defected image was composed using Bior 2,2 the original image and the low frequency component of
wavelet transform filter using Matlab Software. Figure 8 the wavelet decomposition of the cloud-free image as
shows the decomposed images up to three decomposition presented in Figure 2. This approach could be looked at
levels. In this figure, we can notice that cloud areas do not as a typical image fusion application based on wavelet
show much high frequency content in the detail decomposition utilized to filter out certain components in
components. However, shadow areas contain high one of the images involved.
frequency components that represent underneath features Testing several decomposition levels showed that
(or noise). details from the original image in the shadow area
The defected and the cloud-free images were mostly exist in the detail components of the third wavelet
decomposed to the same level of wavelet decomposition. decomposition level. However, image reconstruction
The resulting image is then reconstructed from the high of high and low frequency components of two different

367
 World Appl. Sci. J., 4 (3): 363-370, 2008

Fig. 9: Image reconstructed from second level Fig. 11: Cloud-free image with artificially masked details
decomposition of the cloudy image and the
high-resolution cloud-free image

Fig. 12: Image reconstructed from second level

decomposition of the cloudy image and the
Fig. 10: Image reconstructed from third level artificially altered high-resolution cloud-free
decomposition of the cloudy image and the high- image
resolution cloud-free image
Testing Details Capturing in Shadow Areas: To prove
images results in many introduced artifacts in the the concept of capturing details information in the
produced image. These artifacts are due to differences in shadow areas in the reconstructed image, the cloud-free
frequency content at each decomposition level of the image was artificially altered, as shown in Figure 11, by
participating images in addition to uncertainties in the masking out details in areas where shadows exist in the
space-frequency localization. Figures 9 and 10 show the defected image. These details should be obtained after
resulting image after applying this approach using second applying the fusion process only from the shadow areas
and third wavelet decomposition levels. in the cloudy image.

368
 World Appl. Sci. J., 4 (3): 363-370, 2008

decomposition. This figure shows that more details were

picked from shadow areas in the cloudy image. However,
the third level decomposition caused patchy image
appearance and unwanted artifacts.

Fusion with Lower Resolution Imagery: The final test

was performed by applying the technique using lower
resolution cloud-free image. As described in section 4.1,
the image is a 30 meter resolution Landsat ETM image
resampled to 15 meters ground pixel size. A second level
wavelet transform was used for the image fusion The
shadow enhancement result from this experiment is
shown in Figure 14. Comparing the images in Figure 14
and 9, it could be noticed that more undesirable artifacts
are shown in figure 14, which could be attributed to the
differences in frequency content and distribution in the
fused images. Moreover, the lower quality appearance
of the image shown in figure 14 reflects the effect of using
Fig. 13: Image reconstructed from third level the approximation component of a lower resolution image
decomposition of the cloudy image and the in the fusion process.
artificially altered high-resolution cloud-free
image CONCLUSION

A new technique for cloud-associated shadows

elimination was developed. The technique was based on
wavelet image decomposition and fusion with
another cloud-free image. The developed technique was
successful in enhancing cloud-associated shadow
areas in the image while preserving details in the shadow
areas. The ability to maintain these details increased
with the increase of wavelet decomposition level.
However, the higher this level is, the more artifacts in
the output image. Generally, two or three wavelet
decomposition levels were found to be sufficient for the
analysis used in this study.
The results obtained using two images with the
same spatial resolution were found to be better than
those obtained when using images with different
spatial resolution. This could be attributed to the
differences in frequency content in the two fused image.
Fig. 14: Image reconstructed from second level More research is recommended in this area to determine
decomposition of the cloudy image and the the best decomposition components to participate in the
low-resolution (resampled) cloud-free image fusing process according to each image’s frequency
content.
Figure 12 shows the image resulting from applying
the fusion process using second level wavelet REFERENCES
decomposition. In this figure, the details underneath
shadows in the defected image were picked up in the 1. Belward, A.S. and C.R. Valenzuela, 1991. Remote
resulting image although these details were masked Sensing and Geographic Information Systems for
out in the cloud-free image. Figure 13 shows the results Resource Management in Developing Countries,
of applying the same experiment using third level wavelet Kluwer Academic Publishers, London.

369
 World Appl. Sci. J., 4 (3): 363-370, 2008

2. Finlayson, G.D., S.D. Hordley and M.S. Drew, 2002. 9. Shaker, I., A. Abd-Elrahman, A. Abdel-Gawad and
Removing Shadows from Images, In Proc. of M.A. Abdelwahab, 2005. Toward Removing
European Conf. on Computer Vision, 4: 823.836. Cloud and Shadow Effects in Satellite Images
3. Marini, D. and A. Rizzi,, 2000. A computational Using Image Fusion Techniques. The Scientific
approach to color adaptation effects, Image and Engineering Bulletin, Faculty of Engineering,
Vision Computing, 18(13): 1005-1014. Ain Shams University, Cairo, Egypt, 40(2):
4. Zhou, G., Z. Qin, S. Benjamin and W. Schickler, 2003. 112-117.
Technical Problems of Deploying National Urban 10. Burrus, C.S., R.A. Gopinath and H. Guo, 1998.
Large-scale True Orthoimage Generation. The 2nd Introduction to Wavelets and Wavelet Transforms,
Digital Government Conference, Boston, May 18-21, a primer, Upper Saddle River, Prentice Hall, NJ.
2003, pp: 383-3871. 11. Gens, R., V. Zoltan and C. Pohl, 1998. Image and data
5. Berbar, M.A. and S.F. Gaber, 2004. Clouds and fusion concept and implementation of a multimedia
shadows detection and removing from remote tutorial, In: Fusion of earth data, Sophia Antipolis,
sensing images, Electrical, Electronic and France, 28-30 January, 1998, pp: 217-222.
Computer Engineering, ICEEC '04’, 2004 International 12. Pohl, C., 1999. Tools and Method for Fusion of
Conference, 5-7 Sept. 2004, pp: 75-79. Different Spatial Resolution, International Archives
6. Wang, B. and A. Ono, 1999. Automated Detection of Photogrammetry and Remote Sensing, Valladolid,
and Removal of Clouds and Their Shadows from Spain, 3-4 June, 1999, 32, Part 7-4-3 W6.
Landsat TM Images. IECE Transactions Inf.& Syst., 13. Della, M.R.R., M. Fiani, A. Fortunato and P. Pistillo,
(E82-D)2:453-460. 2004. Using the Data Fusion Technique for
7. Prati, A., I. Mikic, R. Cucchiara and M.M. Trivedi, Producing Thematic Map, The International Archives
2001. Comparative Evaluation of Moving Shadow of the Photogrammetry, Remote Sensing and Spatial
Detection Algorithms, IEEE CVPR Workshop on Information Sciences, Vol. 34, Part XXX, pp: 721-726.
Empirical Evaluation Methods in Computer Vision, 14. Brisco, B. and B. Brown, 1995. Multidate SAR/TM
Kauai, Hawaii, 9-14 December, 2001. Synergism for Crop Classification in Western
8. Lisens, G., P. Kempencers, F. Fierens and J. Van Canada, Photogrammetric Engineering and Remote
Rensbergen, 2000. Development of cloud, snow and Sensing, 61(8): 1009-1014.
shadow masking algorithms for Vegetation imagery, 15. Burt, P.J. and R.J. Kolczynski, 1993. Enhanced
Proceedings of Geoscience and Remote Sensing image capture through fusion. In: Proc. Fourth
Symposium IGARSS 2000, Honolulu, Hawaii, International Conference on Computer Vision, Berlin,
24-28 July, 2000, 2: 834-836. Germany, pp: 173-182.

370