DIGITAL IMAGE

PROCESSING
Official Open

Image Acquisition
Official Open

Imaging
◼ Image: a visual representation in form of a function 𝑓(𝑥, 𝑦) where 𝑓 is related
to the brightness (or color) at spatial coordinates 𝑥, 𝑦 .
◼ Function 𝑓(𝑥, 𝑦) is characterized by two components:
◼ the amount of source illumination incident on the scene being viewed
(illumination)
◼ the amount of illumination reflected by the objects in the scene (reflectance)
◼ 𝑓 𝑥, 𝑦 = 𝑖 𝑥, 𝑦 𝑟(𝑥, 𝑦)
◼ 0 ≤ 𝑖 𝑥, 𝑦 < ∞
◼ 0 ≤ 𝑟 𝑥, 𝑦 ≤ 1
◼ 0: total absorption
◼ 1: total reflectance
Official Open

Imaging (cont’d)
◼ Sampling and quantization to convert continuous images to digital images.
Official Open

Imaging (cont’d)
◼ Digital image: discrete samples 𝑓[𝑥, 𝑦] representing continuous image
𝑓(𝑥, 𝑦). Each element of 2D-array 𝑓[𝑥, 𝑦] is called a pixel.
◼ In computer, think of a digital image as a matrix of 𝑀 rows and 𝑁 columns.
𝑓[0,0] ⋯ 𝑓[0, 𝑁 − 1]
𝑓[𝑥, 𝑦] = ⋮ ⋱ ⋮
𝑓[𝑀 − 1,0] ⋯ 𝑓[𝑀 − 1, 𝑁 − 1]
DIGITIZATION

𝑓(𝑥, 𝑦) Spatially 𝑓[𝑥, 𝑦]

Sample
Q

Mathematical Representation
( as a function )
Official Open

Coordinate Convention

Matlab:
𝑓[𝑥, 𝑦]: gray level or intensity
Official Open

Gray levels
◼ Number of gray levels depends on number of bits used for the value of each
pixel.
◼ 8 bits:
◼ 28 = 256 gray levels
◼ Values: 0 to 255
◼ 0: black
◼ 255: white
Official Open

Color Components
◼ Usually color images are
presented using RGB three
color channels.
◼ Think of color images as
having three (3) matrices,
each for one color.

reference link
Official Open

Spatial Domain Processing

◼ Spatial domain refers to the image plane itself.

◼ Spatial domain techniques operate directly on the pixels of an image.

◼ Two important categories:

◼ Intensity (gray-level) transformation

◼ Spatial filtering (other names: neighbourhood processing, spatial convolution)

◼ In general, obtaining a new image 𝑔[𝑥, 𝑦] from an image 𝑓[𝑥, 𝑦]:

𝑔 𝑥, 𝑦 = 𝑇 𝑓[𝑥, 𝑦]

◼ 𝑇 : an operator on 𝑓 defined over a neighbourhood of point (𝑥, 𝑦).

Official Open

Intensity Transformation

• Pixel at (𝑥0 , 𝑦0 )
• Its value is 𝑟 (= 𝑓[𝑥0 , 𝑦0 ])

• Intensity transformation: 𝑇() operates

on one pixel only.
𝑠 ≜ 𝑔 𝑥, 𝑦
𝑠 = 𝑇(𝑟)
Official Open

Image Negatives
◼ Intensity levels of 𝑓: [0, 𝐿 − 1]
◼ 𝑠 is then calculated by: 𝑠 = 𝐿 − 1 − 𝑟
◼ 𝑟 =0⟹𝑠 =𝐿−1
◼ 𝑟 =𝐿−1⟹𝑠 =0
Official Open

Log Transformations
◼ 𝑠 is then calculated by:
𝑠 = 𝑐 × log(1 + 𝑟)
◼ Maping a narrow range of low
intensity values in the input into a
wider range of output levels.
◼ 0, 𝐿Τ4 ⟹ [0, 3𝐿Τ4]
◼ Higher values of input levels are
mapped to a narrower range in the
output.
◼ We use a transformation of this type to
expand the values of dark pixels in an
image.
Official Open

Power-Law (Gamma) Transformations

◼ 𝑠 is then calculated by:
𝑠 = 𝑐 × 𝑟𝛾
◼ Power-law curves with fractional
values of 𝛾 map a narrow range of dark
input values into a wider range of
output values, with opposite being true
for higher values of input levels.
◼ Curves generated with values of 𝛾 > 1
have exactly the opposite effect as
those generated with values of 𝛾 < 1.
◼ Used in gamma correction.
Official Open

Contrast Stretching
◼ Contrast stretching expands the range of intensity levels in an image so that
it spans the ideal full intensity range.

◼ Two location points (𝑟1 , 𝑠1 ) and (𝑟2 , 𝑠2 )

control the shape of the transformation.

◼ 𝑟1 = 𝑠1 and 𝑟2 = 𝑠2 ?
◼ 𝑟1 = 𝑟2 , 𝑠1 = 0, and 𝑠2 = 𝐿 − 1?
Official Open

Contrast Stretching (cont’d)

◼ Let’s set (𝑟1 , 𝑠1 ) = (𝑟𝑚𝑖𝑛 , 0) and
(𝑟2 , 𝑠2 ) = (𝑟𝑚𝑎𝑥 , 𝐿 − 1)
◼ 𝑟𝑚𝑖𝑛 and 𝑟𝑚𝑎𝑥 denote the minimum
and maximum intensity levels in the
input image

𝑟𝑚𝑖𝑛 𝑟𝑚𝑎𝑥
Official Open

Histograms
◼ Let 𝑟𝑘 , 𝑘 = 0,1, ⋯ , 𝐿 − 1, denote the intensities of an 𝐿-level digital image 𝑓.
The image has 𝑀 rows and 𝑁 columns. Total number of pixels: 𝑀𝑁.
◼ Let 𝑛𝑘 be the number of pixels in 𝑓 with intensity 𝑟𝑘 .
◼ The unnormalized histogram of 𝑓:
ℎ 𝑟𝑘 = 𝑛𝑘 , for 𝑘 = 0,1, ⋯ , 𝐿 − 1
◼ The subdivisions of the intensity scale are called histogram bins.
◼ The normalized histogram of 𝑓:
ℎ 𝑟𝑘 𝑛𝑘
𝑝 𝑟𝑘 = = , for 𝑘 = 0,1, ⋯ , 𝐿 − 1
𝑀𝑁 𝑀𝑁
Official Open

Histograms (cont’d)

Image
20 pixels
unnormalized histogram normalized histogram
Official Open

Histograms (cont’d)
Official Open

Histograms (cont’d)

High contract: wide

Low contrast: narrow histogram that covers a
histogram located wide range of the intensity
typically toward the scale, and the distribution
middle of the intensity of pixels is not too far
scale from uniform
Official Open

Histograms Equalization

High contract: wide

Low contrast: narrow histogram that covers a
histogram located wide range of the intensity
typically toward the scale, and the distribution
middle of the intensity of pixels is not too far
scale from uniform.
Official Open

Histograms Equalization (cont’d)

◼ Producing an output image that has a near uniformly distributed histogram.
◼ Mapping:
𝑘

𝑠𝑘 = 𝑇 𝑟𝑘 = 𝐿 − 1 ෍ 𝑝𝑟 𝑟𝑗 , 𝑘 = 0,1, ⋯ , 𝐿 − 1
𝑗=0
◼ Example:
◼ 𝑠0 = 𝐿 − 1 𝑝𝑟 𝑟0
◼ 𝑠2 = (𝐿 − 1) 𝑝𝑟 𝑟0 + 𝑝𝑟 𝑟1 + 𝑝𝑟 (𝑟2 )
Official Open

Histograms Equalization (cont’d)

◼ 𝐿 = 8; 𝑀 = 𝑁 = 64

Intensity distribution and histogram values

Official Open

Histograms Equalization (cont’d)

◼ 𝑠0 = 𝐿 − 1 𝑝𝑟 𝑟0 = 7 × 0.19 = 1.33

◼ 𝑠1 = 𝐿 − 1 𝑝𝑟 𝑟0 + 𝑝𝑟 (𝑟1 )
= 7 × 0.19 + 0.25 = 3.08
◼ 𝑠2 = 4.55
◼ 𝑠3 = 5.67
◼ 𝑠4 = 6.23
◼ 𝑠5 = 6.55
◼ 𝑠6 = 6.86
◼ 𝑠7 = 7
Official Open

Histograms Equalization (cont’d)

Official Open

Spatial Correlation
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3
Image: 5 × 5

4 3 2 1 -3

1 0 3 3 5

-2 0 1 4 4

5 6 7 9 -1
0 -1 0 Kernel: 3 × 3
-1 5 -1 (Filter)
0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6
-2 0 1 4 4

5 6 7 9 -1

0 -1 0 −5 ∙ 0 + 3 ∙ −1 + 2 ∙ 0 +
4 ∙ −1 + 3 ∙ 5 + 2 ∙ −1 +
-1 5 -1
1 ∙ 0 + 0 ∙ −1 + 3 ∙ 0
0 -1 0
=6
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1
-2 0 1 4 4

5 6 7 9 -1

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4

5 6 7 9 -1

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7
5 6 7 9 -1

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7 9
5 6 7 9 -1

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7 9 2
5 6 7 9 -1

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7 9 2
5 6 7 9 -1
-5

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7 9 2
5 6 7 9 -1
-5 -9

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3

4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7 9 2
5 6 7 9 -1
-5 -9 3

0 -1 0

-1 5 -1

0 -1 0
Official Open

Spatial Correlation (cont’d)

-5 3 2 -5 3 Image: 5 × 5
4 3 2 1 -3
Output
1 0 3 3 5
6 1 8
-2 0 1 4 4
-7 9 2
5 6 7 9 -1
-5 -9 3

0 -1 0 Kernel: 3 × 3
-1 5 -1 (Filter)
0 -1 0
Official Open

2D Convolution
◼ Given a filter, before performing 2D spatial correlation, flip the kernel in both
horizontal and vertical direction (pre-rotated by 180°).

-1 -2 -3 1 2 1
rotate by 180°

3 8 4 4 8 3

1 2 1 -3 -2 -1
Official Open

2D Convolution (cont’d)
◼ Mathematical definition
𝑦 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 ∗ 𝜔 𝑥, 𝑦
∞ ∞

= ෍ ෍ 𝑓 𝑥, 𝑦 ⋅ 𝜔[𝑥 − 𝑠, 𝑦 − 𝑡]
𝑠=−∞ 𝑡=−∞

◼ Kernel size: 𝑚 × 𝑛; image size: 𝑀 × 𝑁

◼ Output size: 𝑆𝑣 × 𝑆ℎ
◼ 𝑆𝑣 = 𝑚 + 𝑀 − 1
◼ 𝑆ℎ = 𝑛 + 𝑁 − 1
Official Open

Smoothing Low-Pass Filters

◼ Smoothing (also called averaging) spatial filters are used to reduce sharp
transitions in intensity.
◼ Random noise typically consists of sharp transitions in intensity, an obvious
application of smoothing is noise reduction.
◼ Smoothing prior to image resampling to reduce aliasing.
◼ Two examples we will consider:
◼ Box filter kernels
◼ Low-pass Gaussian filter kernels
Official Open

LPF – Box Filter Kernels

Box kernel: 3 × 3
Official Open

LPF – Gaussian Filter Kernels

𝑠 2 +𝑡 2
−
𝐺 𝑠, 𝑡 = 𝐾𝑒 2𝜎2
Official Open

LPF – Median Filters

◼ This is a nonlinear filtering. An example of order-statistic filters.
◼ It is used to reduce noise, especially salt-and-pepper noise (impulse noise).
◼ What is median?
◼ Median is the middle number in a sorted list of numbers.
◼ Example: {3, 13, 2, 34, 11, 26, 47}
◼ Sorted list: {2, 3, 11, 13, 26, 34, 47}
◼ Median:13
 Official Open

LPF – Median Filters (cont’d)

Median filter: 3 × 3

Image: 5 × 5
20, 25, 30, 30, 30, 70, 80, 80, 255

0, 20, 30, 70, 80, 80, 100, 100, 255

 Official Open

LPF – Median Filters (cont’d)

Image: 5 × 5
0, 70, 80, 80, 100, 100, 110, 120, 130

Filtered Image
Official Open

Sharpening (High-pass) Spatial Filters

◼ Sharpening is used to highlight transitions in intensity.

◼ Image blurring could be accomplished in the spatial domain by pixel

averaging (smoothing).

◼ Averaging is analogous to integration → sharpening can be accomplished by

spatial differentiation.

◼ We are interested in the behavior of these derivatives in areas of constant

intensity, at the onset and end of discontinuities (step and ramp
discontinuities), and along intensity ramps.
Official Open

Derivatives
◼ Taylor’s series about 𝑥:

◼ If Δ𝑥 = 1: ◼ If Δ𝑥 = −1:
Official Open

First-Order Derivatives
◼ Use linear terms.

𝜕𝑓(𝑥) 𝜕𝑓(𝑥)
= 𝑓 𝑥 + 1 − 𝑓(𝑥) = 𝑓 𝑥 − 𝑓(𝑥 − 1)
𝜕𝑥 𝜕𝑥

forward difference backward difference

𝜕𝑓(𝑥) 𝑓 𝑥 + 1 − 𝑓 𝑥 − 1
=
𝜕𝑥 2
central difference
Official Open

Second-Order Derivatives

𝜕 2 𝑓(𝑥)
𝑓 𝑥 + 1 + 𝑓 𝑥 − 1 = 2𝑓 𝑥 +
𝜕𝑥 2

𝜕 2 𝑓(𝑥)
2
= 𝑓 𝑥 + 1 + 𝑓 𝑥 − 1 − 2𝑓 𝑥
𝜕𝑥
 Official Open

Derivatives: An Example

first derivative: resulting in thick edges

 Official Open

The Laplacian
◼ We have seen an example in 1D. Our images are 2D signals.
◼ The Laplacian
𝜕 2𝑓 𝜕 2𝑓
∇2 𝑓 ≜ 2 + 2
𝜕𝑥 𝜕𝑦

𝜕2𝑓 𝜕2𝑓
= 𝑓 𝑥 + 1, 𝑦 + 𝑓 𝑥 − 1, 𝑦 − 2𝑓 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 + 1 + 𝑓 𝑦 − 1, 𝑥 − 2𝑓 𝑥, 𝑦
𝜕𝑥 2 𝜕𝑦 2

∇2 𝑓 = 𝑓 𝑥 + 1, 𝑦 + 𝑓 𝑥 − 1, 𝑦 + 𝑓 𝑥, 𝑦 + 1 + 𝑓 𝑦 − 1, 𝑥 − 4𝑓 𝑥, 𝑦

𝜕 2 𝑓(𝑥)
= 𝑓 𝑥 + 1 + 𝑓 𝑥 − 1 − 2𝑓 𝑥
𝜕𝑥 2
Official Open

The Laplacian (cont’d)

◼ To implement ∇2 𝑓 = 𝑓 𝑥 + 1, 𝑦 + 𝑓 𝑥 − 1, 𝑦 + 𝑓 𝑥, 𝑦 + 1 + 𝑓 𝑦 − 1, 𝑥 − 4𝑓 𝑥, 𝑦
Official Open

The Laplacian (cont’d)

Official Open

The Laplacian (cont’d)

◼ Other Laplacian kernels:
Official Open

Sharpening Using First Gradient

◼ The gradient of an image 𝑓 at coordinates (𝑥, 𝑦) is defined as the two-
dimensional column vector.
𝜕𝑓
𝑔𝑥 𝜕𝑥
∇𝑓 ≡ grad 𝑓 = 𝑔 = 𝜕𝑓
𝑦
𝜕𝑦
◼ The magnitude (length) of vector ∇𝑓, denoted by 𝑀(𝑥, 𝑦), is
𝑀 𝑥, 𝑦 = mag ∇𝑓 = 𝑔𝑥2 + 𝑔𝑦2
◼ 𝑀(𝑥, 𝑦) is an image of the same size as the original. It is called gradient
image.
◼ In some implementations: 𝑀 𝑥, 𝑦 ≈ 𝑔𝑥 + 𝑔𝑦
Official Open

Sharpening Using First Gradient (cont’d)

𝑓(𝑥 − 1, 𝑦 − 1)
𝑓(𝑥 + 1, 𝑦)

Suppose that we are now at this pixel: 𝑓(𝑥, 𝑦)

Official Open

Sharpening Using First Gradient (cont’d)

𝑓(𝑥 − 1, 𝑦 − 1)
𝑓(𝑥 + 1, 𝑦)

Suppose that we are now at this pixel: 𝑓(𝑥, 𝑦)

How do we calculate:
𝜕𝑓 𝑀 𝑥, 𝑦 = mag ∇𝑓 = 𝑔𝑥2 + 𝑔𝑦2
𝑔𝑥 𝜕𝑥
∇𝑓 ≡ grad 𝑓 = 𝑔 = 𝜕𝑓
𝑦
𝜕𝑦 𝑀 𝑥, 𝑦 ≈ 𝑔𝑥 + 𝑔𝑦
Official Open

Sharpening Using First Gradient (cont’d)

𝑓(𝑥 − 1, 𝑦 − 1)
𝑓(𝑥 + 1, 𝑦)

Answer: Simplest approximation of 𝑔𝑥 and 𝑔𝑦 :

𝑔𝑥 = 𝑧8 − 𝑧5 𝑀 𝑥, 𝑦 = mag ∇𝑓 = 𝑔𝑥2 + 𝑔𝑦2

𝑔𝑦 = 𝑧6 − 𝑧5
𝑀 𝑥, 𝑦 ≈ 𝑔𝑥 + 𝑔𝑦
Official Open

Sharpening Using First Gradient (cont’d)

◼ Two other definitions, proposed by Roberts in 1965, use cross differences.

Answer: Simplest approximation of 𝑔𝑥 and 𝑔𝑦 :

𝑔𝑥 = 𝑧9 − 𝑧5 𝑀 𝑥, 𝑦 = mag ∇𝑓 = 𝑧9 − 𝑧5 2 + 𝑧8 − 𝑧6 2

𝑔𝑦 = 𝑧8 − 𝑧6
𝑀 𝑥, 𝑦 ≈ 𝑧9 − 𝑧5 + 𝑧8 − 𝑧6
Official Open

Sharpening Using First Gradient (cont’d)

◼ The difference terms can be implemented using the two kernels

𝑔𝑥 = 𝑧9 − 𝑧5
𝑔𝑦 = 𝑧8 − 𝑧6
-1 0

0 1

0 -1

Roberts cross-gradient operators 1 0

Official Open

Example