“Digital Image Processing”

Course Code- CSE 314 M

Image Restoration
Techniques

(Chapter 5 of textbook)

1
Recall…
Key Stages in DIP: Image Restoration
Ima ge
Re s tora tion S e gme nta tion

Ima ge Morphologica l
Enha nce me nt P roce s s ing

Ima ge Re pre s e nta tion

Acquis ition & De s cription

P roble m Doma in Obje ct

Re cognition
Colour Ima ge Ima ge
P roce s s ing Compre s s ion
2
Image Degradation and Restoration

h(x,y)

• Image Degradation Model: Spatial domain representation can be modeled by:

g(x,y) h(x,y) f (x,y) (x,y)

• Frequency domain representation can be modeled by:

G(u,v) H (u,v)F(u,v) N(u,v)

3
 Common Noise Models

• Most types of noise are modeled as known

probability density functions (PDF).
• Noise model is decided based on understanding
of the physics of the sources of noise.
– Gaussian: poor illumination
– Rayleigh: range image
– Gamma/Exponential: laser imaging
– Uniform is least used.
– Impulse: faulty switch during imaging,
– Salt and Pepper Noise

• Parameters can be estimated based on

histogram on small flat area of an image

4
Some important Noise PDFs
Consider the following image which contains 3 gray levels.

Histogram of the image

1

0.8

0.6

0.4

0.2

0
0 50 100 150 200 250

5
Some important Noise PDFs
Continued…

Noisy Image

Histogram of the
Noisy Image

6
Gaussian Rayleigh Gamma
Some important Noise PDFs
Continued…

Noisy Image

Histogram of the
Noisy Image

Exponential Uniform Salt & Pepper 7

 Restoration in the presence of Noise
Only: Spatial Filtering
If the only degradation in the image is additive noise then the image degraded by the additive
noise is given by Degradation function h(x,y) is ignored.

g(x,y) f (x,y) (x,y)

The following spatial filters are used to remove the presence of noise in
degraded image.

• Mean filters • Order statistics filters • Adaptive filters

– Arithmetic mean filter – Median filter – Adaptive local noise reduction
– Geometric mean filter – Max and min filters filter
– Harmonic mean filter – Mid-point filter – Adaptive median filter
– Contra-harmonic mean filter

8
 • Mean filters: There are four main types of noise reduction mean filters that can be
used for image restoration/enhancement.

1. Arithmetic Mean Filter: Let Sxy be the coordinates in a subimage window of size
m x n centered at point (x,y). The value of the restored image at any point (x,y) is
given by:

➢ This operation can be implemented by a convolution mask in which all its components have a value 1/mn.

➢ Local variations in the image is smoothed and noise is reduced as a result of blurring.

9
2. Geometric Mean Filter: Let Sxy be the coordinates in a subimage window of size m x n
centered at point (x,y). The value of the restored image at any point (x,y) is given by:

➢ In this filter, each pixel is given by the product of the pixels in the subimage windows, raised to the
power of 1/mn.
➢ Achieves more smoothing, but looses less image details.

3. Harmonic Mean Filter: Let Sxy be the coordinates in a subimage window of size m x n centered at
point (x,y). The value of the restored image at any point (x,y) is given by:

➢ Harmonic mean filter works well with the salt noise but fails for the pepper noise.
➢ It works well with other types of noise as well, such as Gaussian noise. 10
4. Contra-Harmonic Mean Filter: Let Sxy be the coordinates in a subimage window of
size m x n centered at point (x,y). The value of the restored image at any point (x,y) is
given by:

➢ Q is the order of the filter. This filter is well suited for reducing the effects of salt-and-pepper noise.

➢ For negative values of Q, it eliminates the salt noise and for positive values of Q, it eliminates the
pepper noise.

➢ The filter becomes the arithmetic mean filter for Q=0, and harmonic mean filter for Q= -1.

11
• Mean filters Original Additive
– Arithmetic mean filter Gaussian
– Geometric mean filter noise with
– Harmonic mean filter =0, 2=400.
– Contra-harmonic mean filter

3x3Artithmetic
Mean Filter 3x3 Geometric
Mean Filter
(Sharper details)
 Pepper Noise
• Mean filters with Pp=0.1
– Arithmetic mean filter
– Geometric mean filter
Salt Noise
– Harmonic mean filter
with Ps=0.1
– Contra-harmonic mean filter

3x3 3x3
Contraharmonic Contraharmonic
With Q=1.5 With Q=-1.5
(good for Pepper) (good for Salt)
• Order Statistic filters: There are four main types of noise reduction order statictic
filters that can be used for image restoration/enhancement.

1. Median Filter: Let Sxy be the coordinates in a subimage window of size m x n

centered at point (x,y). The value of the restored image at any point (x,y) is given by:

fˆ(x,y) median g(s,t)

(s,t) Sxy

➢ The value of a pixel in is replaced by the median of the gray level in the neighborhood
characterized by Sxy subimage.

➢ Provides less blurring than the other linear smoothing filters.

➢ Median filters are very effective in bipolar or unipolar impulse (Salt & Pepper) noise.

14
2. Max and Min Filters: Let Sxy be the coordinates in a subimage window of size m x n
centered at point (x,y). The value of the restored image at any point (x,y) is given by:

fˆ(x,y) max g(s,t)

(s,t) Sxy

➢ Useful for finding the brightest points in the image. Also, reduces the pepper noise.

fˆ(x,y) min g(s,t)

(s,t) Sxy

➢ Useful for finding the darkest points in the image. Also, reduces the salt noise.

3. Midpoint Filter: Let Sxy be the coordinates in a subimage window of size m x n

centered at point (x,y). The value of the restored image at any point (x,y) is given by:

1
fˆ(x,y) max g(s,t) min g(s,t)
2 ( s,t) S xy (s,t) S xy

➢ This filter combines the order statistics and averaging. Works best for Gaussian and uniform noise.
• Order statistics filters
– Median filter Some traces of
pepper noise
– Max and min filters
– Mid-point filter Salt & Pepper 3x3
Noise with Median Filter
Pa=Pb=0.1 (first pass)

Completely
removed

3x3
3x3 Median Filter
Median Filter (third pass)
(second pass)
 Some dark border pixels
• Order statistics filters are removed
– Median filter
– Max and min filters
– Mid-point filter

Some white dots are darker now.

Pepper Noise 3x3

with Pb=0.1 Max Filter
Good for pepper

3x3
Salt Noise with Min Filter
Pa=0.1 Good for Salt
Restoration in the presence of Noise
(1) Adaptive Local Noise Reduction Filter:
Mean and variance are the simplest statistical measures of a random noise.
• Mean gives the measure of the average gray level in a local region,
• Variance gives the measure of the average contrast in the region.

• Consider a filter operating in a local region, Sxy, where the response of the filter at any point (x,y)
depends on:
a) g(x,y), the value of the noisy image at (x,y)
b) , the additive noise variance,
c) mL , local mean of pixels in Sxy.
d) L , the local variance in Sxy.

18
(1) Adaptive Local Noise Reduction Filter: Continued…

• Consider an adaptive filter where, the following conditions are satisfied:

1. If = 0, the filter should return g(x,y), zero-noise case [ f(x,y)=g(x,y) ].

2. If L >> , the filter should return a value close to g(x,y). High local variance is associated
with edges and should be preserved.

3. If L = , return arithmetic mean of Sxy. This occurs when local noise has the same properties
of the entire image. Averaging simply reduces the noise.

• According to the preceding assumptions the filter response can be modeled as:

• The only unknown parameter in the above adaptive filter model is , The other
parameters can be calculated at the local neighborhood of Sxy.
19
(1) Adaptive Local Noise Reduction Filter: Continued…

Image corrupted
by Gaussian Noise
with 7x7
2
Arithmetic Mean
=1000 Filter
μ= 0

7x7
7x7 Adaptive Filter
Geometric Mean
Filter

20
 (2) Adaptive Median Filter:
Adaptive median filter can filter impulse noise with very high probabilities. Additionally
smoothes the nonimpulse noise which is not the feature of a traditional median filter.

• The filter uses the following parameters in the neighborhood of Sxy :

zmin = minimum gray level value in Sxy.
zmax = maximum gray level value in Sxy.
zmed = median of gray levels in Sxy.
zxy = gray level at coordinates (x,y).
Smax = maximum allowed size of Sxy.

• Note that unlike the other filters the size of Sxy increases during the filtering operation.

• Changing size of the filter mask does not change the fact that the output of the filter is still a
single value centering the mask.
21
(2) Adaptive Median Filter: Continued…

• The Adaptive median filtering Algorithm: Two levels exist (Levels A and B)

Level A: A1 = zmed – zmin

A2 = zmed – zmax
if A1>0 and A2<0, goto Level B else
increase the window size
if window size<Smax repeat LevelA
else output zxy.

Level B: B1 = zxy – zmin

B2 = zxy – zmax
if B1>0 and B2<0, output zxy.
else output zmed.

22
(2) Adaptive Median Filter: Continued…

Undesired
discontinuities

Image corrupted by salt 7x7 Median Filter Adaptive Median Filter

& pepper noise with with Smax=7
Pa=Pb=0.25

23
(3) Periodic Noise Removal by Frequency Domain Filtering:

• Bandreject, bandpass and notch filters can be used for periodic noise removal.

• Bandreject filters remove/attenuate a band of frequencies about the origin of the

Fourier transform.

24
(3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…
❑ Ideal Bandreject filter is given by:

– W is the width of the band

– D0 is the radial center
– D(u,v) distance from the origin.

❑ Butterworth Bandreject filter is given by:

– W is the width of the band

– D0 is the radial center
– D(u,v) distance from the origin.
– n is the order of filter 25
(3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…
❑ Gaussian Bandreject filter is given by:
– W is the width of the band
– D0 is the radial center
– D(u,v) distance from the origin.

Image corrupted by Spectrum of

sinusoidal noise corrupted image

Butterworth Bandreject
Filter (n=4) Filtered Image

26
(3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…
❑ Bandpass filters perform the opposite function of the bandreject filters
and the filter transfer function of a bandpass filter is given by:

H (u,v)bp 1 H (u,v)br

Spectrum of
Image corrupted by corrupted image
sinusoidal noise

Butterworth Bandpass Noise Image

filter

27
 (3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…

❑ Notch filters rejects/passes frequencies in a predefined neighborhoods about the center freq.
• Notch filters appear in symmetric pairs due to the symmetry of the Fourier transform.
• The transfer function of ideal notch filter of radius of D0 , with centers at (u0,v0) and by symmetry
at (-u0,-v0), is given by:

0 if D1(u,v) D0 or D2 (u,v) D0
H (u,v)
1 otherwise
1/ 2
2 2
D1 (u,v) (u M / 2 u 0 ) (v N / 2 v 0)
1/ 2
2 2
D 2(u,v) (u M / 2 u 0 ) (v N / 2 v0 )
28
(3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…

• The transfer function of Butterworth notch filter of order n and of radius of D0 , with centers at
(u0,v0) and by symmetry at (-u0,-v0), is given by:

• The transfer function of Gaussian notch filter of radius of D0 , with centers at (u0,v0) and by
symmetry at (-u0,-v0), is given by:

29
(3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…
Notch filters transfer function comparison:

Ideal Notch
Filter

Gaussian Notch
Butterworth Notch Filter
Filter (of order 2)

30
 (3) Periodic Noise Removal by Frequency Domain Filtering:
Continued…
Notch filters Example:

Original Noisy image with undesired

horizontal scanning lines

Spectrum of the image A simple ideal Notch filter

along the vertical axis

The filtered noise pattern filtered image free of

Corresponding the horizontal horizontal scanning lines.
artifacts
31
Estimating the Degradation Function
There are 3 principal methods of estimating the degradation function for Image Restoration:
(1) Observation (2) Experimentation (3) Mathematical Modelling

The degradation function H can be estimated by visually looking into a small section of the image
containing simple structures, with strong signal contents, like part an object and the background.
Given a small sub-image gs(x,y), we can manually (i.e. filtering) remove the degradation in that
region with an estimated subimage fˆ (x,y)and assuming that the additive noise is negligible in such an
area with a strong signals content.
Gs (u,v)
H s (u,v)
Fsˆ(u,v)

Having Hs(u,v) estimated for such a small subimage, the shape of this degradation function can be
used to get an estimation of H(u,v) for the entire image.
32
(2) Estimation by Image Experimentation:

• If we have the acquisition device producing degradation on images, we can use the
same device to obtain an accurate estimation of the degradation.
• This can be achieved by applying an impulse (bright dot) as an input image. The
Fourier transform of an impulse is constant, therefore.

G(u,v)
H (u,v)
A

Where, A is a constant describing the strength of the impulse. Note that the effect of noise on
an impulse is negligible.

33
 Estimation by Image Experimentation: Continued…

Impulse Image Degraded Impulse Image

consider as h(x,y)

Simply take the Fourier transform of the degraded image and after normalization by a
constant A, use it as the estimate of the degradation function H(u,v).
34
 (3) Estimation by Mathematical Modelling:

❑ Sometimes the environmental conditions that causes the degradation can be modeled
by mathematical formulation. For example the atmospheric turbulence can be modeled
by:
2
k (u v )2 5/6
k is a constant that depends on the nature of the Turbulence
H (u,v) e

• This equation is similar to Gaussian LPF and would produce blurring in the image according to the
values of k . For example if k =0.0025, the model represents severe turbulence, if k =0.001, the
model represents mild turbulence and if k =0.00025, the model represents low turbulence.

• Once a reliable mathematical model is formed the effect of the degradation can be obtained easily.

35
(3) Estimation by Mathematical Modelling: Continued…

Illustration of the atmospheric turbulence model

Severe turbulenc
Negligible turbulence k=0.0025

Low turbulence
Mild turbulence k=0.00025
k=0.001

36
 (3) Estimation by Mathematical Modelling: Continued…

❑ In some applications the mathematical model can be derived by treating that the image is
blurred by uniform linear motion between the image and the sensor during image
acquisition. The motion blur can be modeled as follows:
• Let f(x,y) be subject to motion in x- and y-direction by time varying motion
components x0(t) and y0(t).
• The total exposure is obtained by integrating the instantaneous exposure over the time
interval during the shutter of the imaging device is open.
• If T is the duration of the exposure, then

g(x,y) is the blurred image

37
(3) Estimation by Mathematical Modelling: Continued…

By assuming that the linear uniform motion is in x-direction only at a rate of x0(t)=at/T,
the image covers a distance , when t=T.
T T
j 2 ux 0(t) j2 uat/T
H (u,v) e dt e dt
0 0

T j ua
sin( ua)e
ua

If we allow the motion in y-direction, with y0(t)=bt/T, the model becomes,

T j (ua vb)
H (u,v) sin (ua vb) e
(ua vb)

38
(3) Estimation by Mathematical Modelling: Continued…

Illustration of the result of the modelled motion blur

Original Image Blurred image with a=b=0.1 and T=1

39