Fundamental of Computer Vision and

pattern recognition
2. Image Representation

CV
1
PR
IMAGES IN THE SPATIAL DOMAIN

 A (digital) image is defined by integrating

and sampling continuous (analog) data in
a spatial domain.
 It consists of a rectangular array of pixels
(x, y, u), each combining a location (x, y)
and a value u, the sample at location (x, y)
is normally the set of all integers.

CV
2
PR
LEFT-HAND COORDINATE SYSTEM
• The thumb defines the x-axis, and the pointer the y-axis while looking
into the palm of the hand.

CV
3
PR
PIXELS AND WINDOWS

 Following this given

representation, we may think
about a pixel as a tiny shaded
square.
 Image window is a segment of
the image

CV
4
PR
IMAGE FORMATION MODEL

CV
5
PR
REPRESENTING DIGITAL IMAGES

CV
6
PR
REPRESENTING DIGITAL IMAGES

CV
7
PR
REPRESENTING DIGITAL IMAGES

CV
8
PR
SPATIAL AND INTENSITY RESOLUTION
• Spatial resolution is a measure of the smallest discernible detail in an
image. Spatial resolution can be stated in several ways,
• with line pairs per unit distance
• dots (pixels) per unit distance.
• Dots per unit distance is a measure of image resolution used in the
printing and publishing industry.
• This measure usually is expressed as dots per inch (dpi).
• For example; newspapers are printed with a resolution of 75 dpi,
magazines at 133 dpi, glossy brochures at 175 dpi

CV
9
PR
SPATIAL AND INTENSITY RESOLUTION
• Measures of spatial resolution must be stated with respect to spatial
units.
• Image size by itself does not tell the complete story. For example, to
say that an image has a resolution of 1024 * 1024 pixels is not a
meaningful statement without stating the spatial dimensions
encompassed by the image.
• Size by itself is helpful only in making comparisons between imaging
capabilities.
• For instance, a digital camera with a 20-megapixel CCD imaging chip
can be expected to have a higher capability to resolve detail than an
8-megapixel camera,
CV
10
PR
SPATIAL AND INTENSITY RESOLUTION
• Intensity resolution similarly refers to the smallest discernible change in intensity
level.
• Based on hardware considerations, the number of intensity levels usually is an
integer power of two.
• The most common number is 8 bits, with 16 bits being used in some applications
in which enhancement of specific intensity ranges is necessary.
• Intensity quantization using 32 bits is rare. Sometimes one finds systems that can
digitize the intensity levels of an image using 10 or 12 bits, but these are not as
common.
• Unlike spatial resolution, which must be based on a per-unit-of-distance basis to
be meaningful, it is common practice to refer to the number of bits used to
quantize intensity as the “intensity resolution.”
• For example, it is common to say that an image whose intensity is quantized into
256 levels has 8 bits of intensity resolution.
CV
11
PR
SPATIAL AND INTENSITY RESOLUTION
• Effects of reducing spatial
resolution. The images
shown are at:
(a) 930 dpi,
(b) 300 dpi,
(c) 150 dpi, and
(d) 72 dpi.

CV
12
PR
SPATIAL AND INTENSITY RESOLUTION
• (a) 774 × 640, 256-level image.
(b)-(d) Image displayed in 128, 64,
and 32 intensity levels, while
keeping the spatial resolution
constant. (Original image courtesy of the Dr.
David R. Pickens, Department of Radiology
&Radiological Sciences, Vanderbilt University Medical
Center.)
• The 128- and 64-level images are
visually identical for all practical
purposes. However, the 32-level
image
• in Fig. 2(d) has a set of almost
imperceptible, very fine ridge-like
structures in areas of constant
intensity.
CV
13
PR
SPATIAL AND INTENSITY RESOLUTION
• (e)-(h) Image displayed in 16, 8, 4,
and 2 intensity levels.
• These structures are clearly visible in
the 16-level image in Fig. 2.24(e).
• This effect, caused by using an
insufficient number of intensity levels
in smooth areas of a digital image, is
called false contouring, so named
because the ridges resemble
topographic contours in a map.
• False contouring generally is quit
objectionable in images displayed
using 16 or fewer uniformly spaced
intensity levels, as the images inFigs.
2.24(e)-(h) show.

CV
14
PR
IMAGE INTERPOLATION
• Interpolation is used in tasks such as zooming, shrinking, rotating, and
geometrically correcting digital images.
• Interpolation is the process of using known data to estimate values at
unknown locations.
• For example, an image of size 500 * 500 pixels has to be enlarged 1.5
times to 750 * 750 pixels. A simple way to visualize zooming is to create
an imaginary 750 * 750 grid with the same pixel spacing as the original
image, then shrink it so that it exactly overlays the original image.
• The method just discussed is called because it nearest neighbor
interpolation assigns to each new location the intensity of its nearest
neighbor in the original image

CV
15
PR
 IMAGE INTERPOLATION

original

• (a) which was obtained by reducing the resolution of the 930 dpi to 72 dpi (the size shrank from 2136 *
2140 to 165 * 166 pixels) and then zooming the reduced image back to its original size.
• (b) and (c) are the results of repeating the same procedure but using, respectively, bilinear and bicubic
interpolation for both shrinking and zooming. The result obtained by using bilinear interpolation is a
significant improvement over nearest neighbor interpolation, but the resulting image is blurred slightly.
Much sharper results can be obtained using bicubic interpolation, as (c) shows.
CV
16
PR
 RELATIONSHIPS BETWEEN PIXELS
(NEIGHBORS OF A PIXEL)
• A pixel p at coordinates (x, y) has two horizontal and two
vertical neighbors with coordinates
• This set of pixels, called the 4-neighbors of p, is denoted N4(
p).
(x + 1, y), (x − 1, y), (x, y + 1), (x, y − 1)

• These neighbors, together with the 4-neighbors, are called the

8-neighbors of p, denoted by N 8(p).
x + 1, y + 1), (x + 1, y − 1), (x − 1, y + 1), (x − 1, y − 1)
• The set of image locations of the neighbors of a point p is
called the neighborhood of p.
• The neighborhood is said to be closed if it contains p.
Otherwise, the neighborhood is said to be open.
CV
17
PR
BASIC MATHEMATICAL TOOLS
• An elementwise operation involving one or more images is carried
out on a pixel-bypixel basis.

• The elementwise product (often denoted using the symbol } or z) of

these two images is

CV
18
PR
ARITHMETIC OPERATIONS
• Arithmetic operations between two images f (x, y) and g(x, y) are
denoted as

CV
19
PR
IMAGE SUBTRACTION

(a) Infrared image of (b) Image resulting from setting to zero the least
significant bit of every pixel in (a). (c) Difference of the two images,
scaled to the range [0, 255] for clarity. (Original image courtesy of NASA.)
CV
20
PR
IMAGE SUBTRACTION

(a) Difference between the 930 dpi and 72 dpi images (b) Difference
between the 930 dpi and 150 dpi images. (c) Difference between the
930 dpi and 300 dpi images. CV
21
PR
IMAGE SUBTRACTION
Digital subtraction angiography. (a)
Mask image. (b) A live image. (c)
Difference between (a) and(b). (d)
Enhanced difference image.
(Figures (a) and (b) courtesy of the Image Sciences
Institute, University Medical Center, Utrecht, The
Netherlands.)

CV
22
PR
IMAGE MULTIPLICATION

(a) Digital dental X-ray image. (b) ROI mask for isolating teeth with
fillings (white corresponds to 1 and black corresponds to 0). (c)
Product of (a) and (b).. CV
23
PR
LOGICAL
OPERATIONS
Illustration of logical operations
involving foreground (white) pixels.
Black represents binary 0’s and
white binary 1’s. The dashed lines
are shown for reference only. They
are not part of the result.

CV
24
PR
GEOMETRIC
TRANSFORMATIONS

CV
25
PR
THE BASICS OF INTENSITY TRANSFORMATIONS
AND SPATIAL FILTERING
• The spatial domain processes are based on the expression

• where f (x, y) is an input image, g(x, y) is the output image, and T is

an operator on f defined over a neighborhood of point (x, y). The
operator can be applied to the pixels of a single image (our
principal focus in this chapter) or to the pixels of a set of images,
such as performing the elementwise sum of a sequence of images
for noise reduction

CV
26
PR
A 3 × 3 neighborhood about a point
(x0 , y0 ) in an image.
The neighborhood is moved from
pixel to pixel in the image to
generate an output image. The
value of a pixel at location (x0 , y0 )
is f (x0 , y0 ), the value of the image
at that location.

CV
27
PR
IMAGE NEGATIVES
• The negative of an image with intensity levels in the range [0,L − 1] is
obtained by using the negative transformation function:

• Reversing the intensity levels of a digital image in this manner

produces the equivalent of a photographic negative. This type of
processing is used, for example, in enhancing white or gray detail
embedded in dark regions of an image, especially when the black
areas are dominant in size.

CV
28
PR
IMAGE NEGATIVES
(a) A digital
mammogram.
(b) Negative
image obtained
using Equation.
(Image (a) Courtesy of General
Electric Medical Systems.)

CV
29
PR
LOG TRANSFORMATIONS
The general form of the log transformation

(a) Fourier spectrum displayed as a grayscale image. (b) Result of applying the
log transformation in Eq. (3-4) with c = 1. Both images are scaled to the range
[0, 255]. CV
30
PR
POWER-LAW (GAMMA) TRANSFORMATIONS
• Power-law transformations have the form where c and g are positive constants

(a) Intensity ramp image. (b)

Image as viewed on a
simulated monitor with a
gamma of 2.5. (c) Gamma
corrected image.
(d) Corrected image as viewed
on the same monitor. Compare
(d) and (a).

CV
31
PR
CONTRAST
ENHANCEMENT USING
POWER-LAW INTENSITY
TRANSFORMATIONS
(a) Magnetic resonance image
(MRI) of a fractured human
spine (the region of the fracture is
enclosed by the circle).
(b)–(d) Results of applying the
transformation
in Eq. with c = 1 and g = 0 6 . ,
0.4, and 0.3, respectively.

(Original image courtesy of Dr.

David R. Pickens, Department of
Radiology and Radiological Sciences,
Vanderbilt University
Medical Center.)

CV
32
PR
CONTRAST
ENHANCEMENT USING
POWER-LAW INTENSITY
TRANSFORMATIONS
(a) Aerial image.
(b)–(d) Results of applying the
Transformation in Eq. with
g = 3 0 . , 4.0, and 5.0,
respectively.
(c = 1 in all cases.)

(Original image courtesy of NASA.)

CV
33
PR
CONTRAST STRETCHING
• Low-contrast images can result from
poor illumination, lack of dynamic range
in theimaging sensor, or even the wrong
setting of a lens aperture during image
acquisition.
• Contrast stretching expands the range of
intensity levels in an image so that it
spans the ideal full intensity range of the
recording medium or display device.

Contrast stretching.
(a) Piecewise linear transformation function.
(b) A low contrast electron microscope image
of pollen, magnified 700 times.
(c) Result of contrast stretching.
(d) Result of thresholding.
(Original image courtesy of Dr. Roger Heady,
Research School of Biological Sciences, Australian National
University, Canberra, Australia.) CV
34
PR
INTENSITY-LEVEL SLICING
• There are applications in which it is of interest to highlight a specific range
of intensities in an image.
• Some of these applications include enhancing features in satellite imagery,
such as masses of water, and enhancing flaws in X-ray images.
• The method, called intensity-level slicing, can be implemented in several
ways, but most are variations of two basic themes.
• One approach is to display in one value (say, white) all the values in the
range of interest and in another (say, black) all other intensities. This
transformation, produces a binary image.
• The second approach, based on the transformation, brightens (or darkens)
the desired range of intensities, but leaves all other intensity levels in the
image unchanged

CV
35
PR
 INTENSITY-LEVEL SLICING

a) Aortic angiogram. (b) Result of using a slicing transformation with the range of intensities of interest
selected in the upper end of the gray scale. (c) Result of using the transformation with the selected range
set near black, so that the grays in the area of the blood vessels and kidneys were preserved. (Original image CV
courtesy of Dr. Thomas R. Gest, University of Michigan Medical School.) 36
PR
HISTOGRAM PROCESSING
• Let rk , fork = 0,1, 2,…,L − 1, denote the intensities of an L-level digital image, f (x, y).
The unnormalized histogram of f is defined as;

• where nk is the number of pixels in f with intensity rk , and the subdivisions of the
intensity scale are called histogram bins. Similarly, the normalized histogram of f is
defined as

• where, as usual, M and N are the number of image rows and columns, respectively.
Mostly, we work with normalized histograms, which we refer to simply as
histogramsor image histograms. The sum of p rk ( ) for all values of k is always 1.
CV
37
PR
HISTOGRAM PROCESSING

Four image types and their corresponding histograms. (a) dark; (b) light; (c) low contrast; (d) high
contrast. The horizontal axis of the histograms are values of rk and the vertical axis are values of p (rk). CV
38
PR
HISTOGRAM EQUALIZATION

Histogram equalization.
(a) Original histogram. (b) Transformation function. (c) Equalized histogram.
CV
39
PR
HISTOGRAM EQUALIZATION

CV
40
PR
HISTOGRAM
SPECIFICATION.
(a) Histogram of a 3-bit image.
(b) Specified histogram.
(c) Transformation function
obtained from the specified
histogram.
(d) Result of histogram
specification. Compare the
histograms in (b) and (d)

CV
41
PR
 (a) An image, and (b) its histogram.

(a) Histogram
equalization
transformation
obtained using
the histogram
in Fig. 3.23(b).
(b) Histogram
equalized image.
(c) Histogram of
equalized image.
Histogram specification.
(a) Specified histogram. (b) Transformation G zq ( ), CV
labeled (1), and G sk −1( ), labeled (2). (c) Result of 42
histogram specification. (d) Histogram of image (c). PR
LOCAL HISTOGRAM EQUALIZATION

(a) Original image. (b) Result of global histogram equalization.

(c) Result of local histogram equalization.
CV
43
PR
LOCAL ENHANCEMENT USING HISTOGRAM STATISTICS.

(a) Original image. (b) Result of local enhancement based on local

Histogram statistics.
CV
44
PR
FUNDAMENTALS OF SPATIAL FILTERING
• Spatial filtering is used in a broad spectrum of image processing
applications, so a solid understanding of filtering principles is
important.
• Where “filtering” refers to passing, modifying, or rejecting specified
frequency components of an image.
• For example, a filter that passes low frequencies is called a lowpass
filter. The net effect produced by a lowpass filter is to smooth an
image by blurring it.
• We can accomplish similar smoothing directly on the image itself by
using spatial filters.
CV
45
PR
FUNDAMENTALS OF SPATIAL FILTERING
• Spatial filtering modifies an image by replacing the value of each
pixel by a function of the values of the pixel and its neighbors.
• If the operation performed on the image pixels is linear, then the
filter is called a linear spatial filter.
• Otherwise, the filter is a nonlinear spatial filter.

CV
46
PR
 SPATIAL CORRELATION AND CONVOLUTION

The mechanics of linear spatial filtering

using a 3 × 3kernel. The pixels are shown
as squares to simplify the graphics.Note
that the origin of the image is at the top
left, but the origin of the kernel is at
its center. Placing the origin at the
center of spatially symmetric kernels
simplifies writing expressions for
CV
linear filtering. 47
PR
KERNEL

Examples of smoothing kernels:

(a) is a box kernel; (b) is a Gaussian Kernel.
CV
48
PR
HOW SPATIAL FILTER KERNELS ARE CONSTRUCTED
• Consider three basic approaches for constructing spatial filters, one approach is based
on formulating filters based on mathematical properties. For example, a filter that
computes the average of pixels in a neighborhood blurs an image. Computing an
average is analogous to integration. Conversely, a filter that computes the local
derivative of an image sharpens the image.
• A second approach is based on sampling a 2-D spatial function whose shape has a
desired property. For example, we will show in the next section that samples from a
Gaussian function can be used to construct a weighted-average (lowpass) filter. These
2-D spatial functions sometimes are generated as the inverse Fourier transform of 2-D
filters specified in the frequency domain.
• A third approach is to design a spatial filter with a specified frequency response. This
approach is based on the concepts discussed in the previous section, and falls in the
area of digital filter design. A 1-D spatial filter with the desired response is obtained
(typically using filter design software). The 1-D filter values can be expressed as a
vector v, and a 2-D separable kernel can then be obtained using Eq. (3-42). Or the 1-D
filter can be rotated about its center to generate a 2-D kernel that approximates a
circularly symmetric function
CV
49
PR
SMOOTHING (LOWPASS) SPATIAL FILTERS
• Smoothing (also called averaging) spatial filters are used to reduce sharp
transitions in intensity. Because random noise typically consists of sharp
transitions in intensity, an obvious application of smoothing is noise reduction.
• Smoothing is used to reduce irrelevant detail in an image, where “irrelevant”
refers to pixel regions that are small with respect to the size of the filter kernel.
• Another application is for smoothing the false contours that result from using an
insufficient number of intensity levels in an image.
• Smoothing filters are used in combination with other techniques for image
enhancement, such as the histogram processing techniques.
• In addition to being useful in countless applications of image processing, lowpass
filters are fundamental, in the sense that other important filters, including
sharpening (highpass), bandpass, and bandreject filters, can be derived from
lowpass filters

CV
50
PR
LOWPASS FILTERING
WITH A BOX KERNEL.
(a) Test pattern of size
1024 × 1024 pixels.
(b)-(d) Results of
lowpass filtering with
box kernels
of sizes 3 × 3, 11 × 11,
and 21 × 21,
respectively.

CV
51
PR
LOWPASS GAUSSIAN FILTER KERNELS

(a) Sampling a Gaussian function to obtain a discrete Gaussian kernel.

The values shown are for K = 1 and s = 1. (b) Resulting 3 × 3 kernel

CV
52
PR
LOWPASS FILTERING WITH A GAUSSIAN KERNEL.

(a)A test pattern of size 1024 × 1024. (b) Result of lowpass filtering the pattern with a
Gaussian kernel of size 21 × 21, with standard deviations s = 3.5. (c) Result of using a kernel of
size 43 × 43, with s = 7.
CV
53
PR
SHARPENING (HIGHPASS) SPATIAL FILTERS
• Sharpening highlights transitions in intensity. Uses of image
sharpening range from electronic printing and medical imaging to
industrial inspection and autonomous guidance in military systems.
• We saw that image blurring could be accomplished in the spatial
domain by pixel averaging (smoothing) in a neighborhood. Because
averaging is analogous to integration, it is logical to conclude that
sharpening can be accomplished by spatial differentiation.

CV
54
PR
MEDIAN FILTERING

(a) X-ray image of a circuit board, corrupted by salt-and-pepper noise. (b) Noise reduction using a
19 × 19 Gaussian lowpass filter kernel with s = 3. (c) Noise reduction using a 7 × 7 median filter.
(Original image courtesy of Mr. Joseph E. Pascente, Lixi, Inc.)

CV
55
PR
USING THE SECOND DERIVATIVE FOR IMAGE
SHARPENING—THE LAPLACIAN

(a) Laplacian kernel used to implement Eq. (3-53). (b) Kernel used to implement
an extension of this equation that includes the diagonal terms. (c) and (d) Two other
Laplacian kernels.

CV
56
PR
IMAGE SHARPENING
USING THE LAPLACIAN
(a) Blurred image of the North
Pole of the moon.
(b) Laplacian image obtained
using the kernel in Fig. 3.45(a).
(c) Image sharpened using Eq.
with c = −1.
(d) Image sharpened using
the same procedure, but
with the kernel .
(Original image courtesy of
NASA.)

CV
57
PR
UNSHARP MASKING AND HIGHBOOST FILTERING

(a) Original image of size 600 × 259 pixels. (b) Image blurred using a 31 × 31
Gaussian lowpass filter with s = 5. (c) Mask. (d) Result of unsharp masking using
Eq. (3-56) with k = 1. (e) Result of highboost filtering with k = 4 5 . .

CV
58
PR
FIRST-ORDER
DERIVATIVES FOR IMAGE
SHARPENING—THE
GRADIENT
(a) A 3 × 3 region of an image,
where the zs are intensity values.
(b)–(c) Roberts cross-gradient
operators.
(d)–(e) Sobel operators. All the
Kernel coefficients sum to zero, as
expected of a derivative operator.

CV
59
PR
USING THE GRADIENT FOR EDGE ENHANCEMENT

(a) Image of a contact

lens (note defects on
the boundary at 4 and
5 o’clock).
(b) Sobel gradient.
(Original image courtesy of
Perceptics
Corporation.)

CV
60
PR
COMBINING SPATIAL ENHANCEMENT METHODS

Spatial filtering of
the zone plate
image. (a) Lowpass
result; (b) Highpass
result. (c) Image (b)
with intensities
scaled. (d) andreject
result. (e) Bandpass
result. (f) Image (e)
with intensities
scaled.

CV
61
PR
COMBINING SPATIAL
ENHANCEMENT
METHODS

(a) Image of whole

body bone scan.
(b) Laplacian of (a).
(c) Sharpened image
obtained by adding
(a) and (b).
(d) Sobel gradient of
image (a). (Original
image courtesy of
G.E. Medical Systems.)
CV
62
PR
COMBINING SPATIAL
ENHANCEMENT
METHODS

(e) Sobel image smoothed with a

5 × 5 box filter.
(f) Mask image formed by the
product of (b) and (e).
(g) Sharpened image obtained
by the adding images (a) and (f).
(h) Final result obtained by
applying a power law transformation
to (g). Compare images (g) and (h)
with (a).
(Original image courtesy of G.E.
Medical Systems.)
CV
63
PR