Fundamentals of Image and

Video Processing
Module 4
Color Image processing
 Syllabus
Color fundamentals, color models, pseudo
color image processing, basics of fullcolor
image processing, color transforms,
smoothing and sharpening, color
segmentation
 Module 5 Image Compression
Image compression fundamentals coding
Redundancy, spatial and temporal redundancy
Compression models : Lossy and Lossless,
Huffmann coding, Arithmetic coding, LZW
coding, run length coding, Bit Plane coding,
transform coding, predictive coding , wavelet
coding, JPEG standards
 Module 6 Video Processing & 2-D Motion
Estimation
Analog video, Digital Video, Time varying Image Formation
models : 3D motion models, Geometric Image formation ,
Photometric Image formation, sampling of video signals,
filtering operations.
Optical flow, general methodologies, pixel based motion
estimation, Block matching algorithm, Mesh based motion
Estimation, global Motion Estimation, Region based
motion estimation, multi resolution motion estimation.
Waveform based coding, Block based transform coding,
predictive coding, Application of motion estimation in
video coding
Physical Background
Visible light: a narrow band of
electromagnetic radiation
380nm (blue) - 780nm (red)
Wavelength: Each physically
distinct colour corresponds to at
least one wavelength in this band.
Pure Colours: Pure or
monochromatic colours do not
exist in nature.
 Spectrum: Intensity as
a function of wavelength.

From http://fuse.pha.jhu.edu/~wpb/spectroscopy/basics.html

The colour of an object: is the product of the

spectrum of the incident light with the light
absorption and/or reflection properties of the
object.
 Color Fundamentals
In 1666 Sir Isaac Newton discovered that when a
beam of sunlight passes through a glass prism, the
emerging beam is split into a spectrum of colours
 Color Fundamentals
Chromatic light spans the electromagnetic spectrum from
approximately 400 to 700 nm
As we mentioned before human colour vision is achieved through 6 to
7 million cones in each eye
 Color Fundamentals

6 to 7 million cones in the human eye can be divided into

three principal sensing categories, corresponding roughly to
red, green, and blue.
65%: red 33%: green 2%: blue (blue cones are the most
sensitive)
 Problems with Processing Colour Images
When processing colour images, the following
problems (amongst others) have to be dealt
with:
The images are vectorial 3 numbers are
associated with each pixel.
The colours recorded by a camera are heavily
dependent on the lighting conditions.
 Lighting conditions
The lighting conditions of the scene have a
large effect on the colours recorded.

Image taken lit by a flash. Image taken lit by a tungsten

lamp.
 The following four images of the same scene were
acquired under different lighting conditions:
 Color Fundamentals
Color Fundamentals

The human eye does not perceive individual light wavelengths.

It contains three types of colour receptor (cones) which integrate over
parts of the spectrum:
 Color Fundamentals

3 basic qualities are used to describe the quality of

a chromatic light source:
Radiance: the total amount of energy that flows from
the light source (measured in watts)
Luminance: the amount of energy an observer
perceives from the light source (measured in lumens)
Note we can have high radiance, but low luminance
Brightness: a subjective (practically unmeasurable)
notion that embodies the intensity of light
Well return to these later on
 Color Fundamentals

Specifying colours systematically can be

achieved using the CIE chromacity diagram
On this diagram the x-axis represents the
proportion of red and the y-axis represents the
proportion of red used
The proportion of blue used in a colour is
calculated as:
z = 1 (x + y)
 RGB
It is therefore possible to characterise a psycho-visual
colour by specifying the amounts of three primary
colours: red, green and blue, mixed together.
This leads to the standard RGB space used in
television, computer monitors, etc.
We specify the levels of R, G and B in the range [0, 1],
but they can easily be extended to other ranges (8-bit
integers for example). (1,1,1)

RGB

(0,0,0)
 RGB
In the RGB model each colour appears in its primary
spectral components of red, green and blue
The model is based on a Cartesian coordinate system
RGB values are at 3 corners
Cyan magenta and yellow are at three other corners
Black is at the origin
White is the corner furthest from the origin
Different colours are points on or inside the cube
represented by RGB vectors
 RGB
Images represented in the RGB colour model
consist of three component images one for each
primary colour
When fed into a monitor these images are
combined to create a composite colour image
The number of bits used to represent each pixel is
referred to as the colour depth
A 24-bit image is often referred to as a full-colour
image as it allows = 16,777,216 colours
 Dealing with Lighting Changes
Knowing just the RGB values is not enough to
know everything about the image.
The R, G and B primaries used by different devices are
usually different.
For scientific work, the camera and lighting
should be calibrated.
For multimedia applications, this is more difficult
to organise:
Algorithms exist for estimating the illumination colour.
 Processing Vectorial Images
A vectorial image has a vector at each pixel. For colour
images, these vectors each have 3 components.
Vectorial images with larger numbers of components
also exist, e.g. in satellite imagery.
There are two ways one can process vectorial images:
Marginal processing.
Vectorial processing.
Greyscale

Colour

f (x, y) {0,1,, N} f (x, y) [ {0,,N}, {0,,N}, {0,,N} ]

 Marginal Processing

Each channel is processed separately:

Red Red
Scalar process.

Green Green
Scalar process.

Blue Blue
Scalar process.
 Vectorial Processing
The colour triplets are processed as single units:

Red Red

Green Vectorial Green

process.
Blue Blue
The Problem of False Colours
False
Colours !!
Marginal
Median

Vectorial
Median
 Alternative Colour Spaces
Various other colour representations can be
calculated from the RGB representation.
This can be done for:
Decorrelating the colour channels:
principal components.
Bringing colour information to the fore:
Hue, saturation and brightness.
Perceptual uniformity:
CIELuv, CIELab,
 Processing Strategy

Red Red

Processing
Green Green
T T-1
Blue Blue
 Colour spaces
RGB (CIE), RnGnBn (TV - National Television Standard Comittee)
XYZ (CIE)
UVW (UCS de la CIE), U*V*W* (UCS modified by the CIE)
YUV, YIQ, YCbCr
YDbDr
DSH, HSV, HLS, IHS
Munsel colour space (cylindrical representation)
CIELuv
CIELab
SMPTE-C RGB
YES (Xerox)
Kodak Photo CD, YCC, YPbPr, ...
 Color Fundamentals

The characteristics generally used to distinguish one color

from another are brightness, hue, and saturation

brightness: the achromatic notion of intensity.

hue: dominant wavelength in a mixture of light waves,

represents dominant color as perceived by an observer.

saturation: relative purity or the amount of white light mixed

with its hue.
 Color Fundamentals

Tristimulus
Red, green, and blue are denoted X, Y, and Z, respectively. A
color is defined by its trichromatic coefficients, defined as

X
x=
X +Y + Z
Y
y=
X +Y + Z
Z
z=
X +Y + Z
 CIE Chromaticity Diagram

It shows color
composition as
a function of x
(red) and y
(green)
RGB Color Model
RGB Color Model

Pixel depth

The total number of colors

in a 24-bit RGB image is (28)3
= 16,777,216
Safe RGB colors (or safe
Web colors) are
reproduced faithfully,
reasonably
independently of viewer
hardware capabilities
 The CMY and CMYK Color Models

C 1 R
M =
1-
G


Y
1
B

Equal amounts of the pigment primaries, cyan, magenta, and yellow
should produce black. In practice, combining these colors for
printing produces a muddy-looking black.

To produce true black, the predominant color in printing, the fourth

color, black, is added, giving rise to the CMYK color model.
 CMY vs. CMYK

http://en.wikipedia.org/wiki/CMYK
 HIS Color Model

RGB is useful for hardware implementations and is

serendipitously related to the way in which the
human visual system works
However, RGB is not a particularly intuitive way in
which to describe colours
Rather when people describe colours they tend to
use hue, saturation and brightness
RGB is great for colour generation, but HSI is great
for colour description
 HIS Color Model

The HSI model uses three measures to describe

colours:
Hue: A colour attribute that describes a pure colour
(pure yellow, orange or red)
Saturation: Gives a measure of how much a pure
colour is diluted with white light
Intensity: Brightness is nearly impossible to measure
because it is so subjective. Instead we use intensity.
Intensity is the same achromatic notion that we have
seen in grey level images
 HIS Color Model

Intensity can be extracted from RGB images

which is not surprising if we stop to think about
it
Remember the diagonal on the RGB colour cube
that we saw previously ran from black to white
Now consider if we stand this cube on the black
vertex and position the white vertex directly
above it
 HSI, Hue & RGB
Now the intensity component
of any colour can be
determined by passing a
plane perpendicular to
the intenisty axis and
containing the colour
point
The intersection of the plane
with the intensity axis gives us
the intensity component of the
colour
 HSI, Hue & RGB
In a similar way we can extract
the hue from the RGB colour
cube
Consider a plane defined by
the three points cyan, black
and white
All points contained in
this plane must have the
same hue (cyan) as black
and white cannot contribute
hue information to a colour
 HSI, Hue & RGB
Consider if we look straight down at
the RGB cube as it was arranged
previously
We would see a hexagonal
shape with each primary
colour separated by 120
and secondary colours
at 60 from the primaries
So the HSI model is
composed of a vertical
intensity axis and the locus of colour
points that lie on planes perpendicular
to that axis
 The HSI Colour Model
To the right we see a hexagonal
shape and an arbitrary colour
point
The hue is determined by an
angle from a reference point,
usually red
The saturation is the distance from the origin to the point
The intensity is determined by how far up the vertical
intenisty axis this hexagonal plane sits (not apparent
from this diagram
Because the only important things are the angle
and the length of the saturation vector this plane
is also often represented as a circle or a triangle
HIS Color Model
HIS Color Model
HIS Color Model
 Converting Colors from RGB to HSI

Given an image in RGB color format, the H component of each

RGB pixel is obtained using the equation

q
if B G
H =
360 - q if B>G

1
-1
[ ( R - G ) + ( R - B ) ]
q = cos 2
1/ 2
(R - G ) + ( R - B)(G - B)

2


 Converting Colors from RGB to HSI

Given an image in RGB color format, the saturation

component is given by

3
S = 1- [ min( R, G, B)]
( R + G + B)
 Converting Colors from RGB to HSI

Given an image in RGB color format, the intensity component

is given by

1
I = ( R + G + B)
3
 Converting Colors from HSI to RGB

RG sector (0o H < 120o)

B = I (1 - S )
S cos H
R=I 1+
cos(60 - H )
o

and
G = 3I - ( R + B )
 Converting Colors from HSI to RGB

RG sector (120o H < 240o)

H = H - 120o
R = I (1 - S )
S cos H
G=I1+
cos(60 - H )
o

and
B = 3I - ( R + G )
 Converting Colors from HSI to RGB

RG sector (240o H 360o)

H = H - 240o
G = I (1 - S )
S cos H
B=I1+
cos(60 - H )
o

and
R = 3I - (G + B )
 Pseudocolor Image Processing

The process of assigning colors to gray values based on a

specified criterion.

Intensity Slicing

f ( x, y ) = ck if f ( x, y ) Vk
 Pseudocolor Image Processing

Intensity to Color Transformation

The images are obtained from
an airport X-ray scanning
system.
The left contains ordinary
articles and the right contains
the same articles as well as a
block of simulated plastic
explosives.
Pseudocolor by combining
several of the sensor images
from the Galileo spacecraft,
some of which are in spectral
regions not visible to the eye.

Bright red depicts materials

newly ejected from an active
volcano on Io, and the
surrounding yellow materials
are older sulfur deposits.
 Basics of Full-Color Image Processing

Let c represent an arbitrary vector in RGB color space:

cR R
c=
c =
G
G


cB
B

At coordinates (x, y ),
cR ( x, y ) R ( x, y )
c ( x, y ) =
c
G ( x, y ) =
G ( x, y )


cB ( x, y )
B ( x, y )

Basics of Full-Color Image Processing
 Color Transformations

g ( x, y ) = T [ f ( x, y ) ]

si = Ti (r1 , r2 ,..., rn ), i = 1, 2,..., n.

 g ( x, y ) = kf ( x, y )

si = kri , si = kri + (1 - k ), s3 = kr3

i = 1, 2,3. i = 1, 2,3.
 Color slicing

Highlighting a specific range of colors in an image

If the colors of interest are enclosed by a cube of width W

and centered at a protypical color with components
(a1 , a2 ,..., an ), the necessary set of transformations is

0.5
| rj - a j |> W / 2
if
si =
any 1j n

ri otherwise
 Color slicing

If a sphere is used to specify the colors of interest,

R 0 is the radius of the enclosing of its center.
The transformations is
n

( j j ) 0
2
0.5 if r - a > R 2

si = j =1
ri otherwise

Color slicing
 Tone and
Color
Corrections
 Color Image Smoothing

Let S xy denote the set of coordinates defining a neighborhood

centered at (x, y ) in an RGB color image. The average of the
RGB component vectors in this neighborhood is
1
R ( s, t )
K ( s ,t )S xy
1 1
c ( x, y ) =
K ( s ,t )S xy
c ( s, t ) = G ( s, t )
K ( s ,t )S xy

1 B ( s, t )

K ( s ,t )S xy

 Color Image Sharpening

The Laplacian of vector c is

R ( x, y )
2

2
[ c ( x, y ) ] =
2
G ( x, y )

2
B ( x , y )

 Image
Segmentation
Based on Color:

Segmentation in
HIS Color Space
 Segmentation in RGB Vector Space

Let the average color of interest is denoted by the

RGB vector a. Let z denote an arbitrary point in
RGB space.
1/2
D( z, a) = z - a =
( z - a) ( z - a)

T

2 1/2
=
( z R - aR ) + ( zG - aG ) + ( z B - aB )

2 2