Computer Vision, Image Processing and

Computational Photography
Lecture 2: Digital image processing contd

Ayelet Heimowitz

October, 2022

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image filtering - idea

The linear filter will replace each pixel with a weighted sum
over the pixel values in some (small) neighborhood around the
pixel.
The weights are called the filter coefficients.
We define the kernel h as the matrix whose entries are the
filter coefficients. For example,
 
1/9 1/9 1/9
h = 1/9 1/9 1/9 .
1/9 1/9 1/9

The filtering operation is

J=I∗h

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Image filtering - idea

That is,
k
X k
X
J(x, y ) = I (x − m, y − n)h(m, n)
m=−k n=−k

Alternatively, the filtering can be done with a correlation

operator (that is, via cross-correlation)

k
X k
X
J(x, y ) = I (x + m, y + n)h(m, n)
m=−k n=−k

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Averaging filter

In the averaging filter, the filter coefficients are uniform and

sum to one.
That is, the kernels can be
 
1/9 1/9 1/9
h = 1/9 1/9 1/9 .
1/9 1/9 1/9

 
1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25
 
 /25
h=1 1/25 1/25 1/25 1/25 .

1/25 1/25 1/25 1/25 1/25
1/25 1/25 1/25 1/25 1/25

etc.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Averaging filter

The operation of the averaging filter is

k k
1 X X
J(x, y ) = I (x − m, y − n)
k2
m=−k n=−k

What happens at J(0, 0)? Border effects.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Gaussian filter
In a Gaussian filter, the filter coefficients are determined by
the 2-D Gaussian distribution
1 −(x 2 +y 2 )/2σ2
h(x, y ) = e
2πσ 2

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Gaussian filter
A digital image contains discrete spatial coordinates.

Need to approximate the Gaussian distribution at discrete

spatial coordinates.

How large should the kernel be? Beyond µ ± 3σ most values

are negligible.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Gaussian vs. Averaging filter
Averaging filter, k=5 Averaging filter, k=5

Gaussian filter, k=5, s=1 Gaussian filter, k=5, s=3

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Gaussian vs. Averaging filter

Both filters have positive values that sum to 1.

The Gaussian filter gives more weight to the center of the

kernel.

The operation done by these filters is smoothing, that is,

removing sudden changes.

This is actually low pass filtering.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Sharpening filters

To sharpen an image I, we can remove a Gaussian (or

averaging) filter h as such

J = I + α(I − I ∗ h)

Figure: Original image Figure: Sharp image

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Review: properties of convolution
convolution

M
X N
X
J(x, y ) = I (x − m, y − n)h(m, n)
m=−M n=−N

The delta function is the identity for convolution, that is

I = I∗

(with proper treatment of the boundary).

Convolution with a shifted delta will result in a shift in image

pixels.

Commutative property (after padding with zeros)

I∗h=h∗I
Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra
Review: properties of convolution

convolution

M
X N
X
J(x, y ) = I (x − m, y − n)h(m, n)
m=−M n=−N

Distributive property

(I + J) ∗ h = I ∗ h + J ∗ h

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Separable filtering

convolution

M
X N
X
J(x, y ) = I (x − m, y − n)h(m, n)
m=−M n=−N

A separable filter has a kernel that can be expressed as a

multiplication of a column vector by a row vector.

Separable filters allow us to reduce computational overhead by

convolving the image with vectors, since

I ∗ (f ∗ h) = (I ∗ f) ∗ h.

This leads to M + N multiply-add operations per pixel instead

of MN.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Separable filtering

When is a kernel separable?

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Separable filtering

When is a kernel separable?

Idea: singular value decomposition.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Separable filtering

When is a kernel separable?

Idea: singular value decomposition.
Any (real-valued) m × n matrix can be expressed as a
multiplication of three matrices
 
σ0 0 · · · 0  T 
v
T
   0 σ1 · · ·
 0   0. 

h = USV = u0 · · · up−1  ..   ..  ,
.
vp−1 T
 
0 0 · · · σp−1

where U is an orthonormal matrix (that is, UT U = Ip×p ) of

size m × p, S is a diagonal matrix of size p × p and V is an
orthonormal matrix of size p × n.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Separable filtering

When is a kernel separable?

Idea: singular value decomposition.
Any (real-valued) m × n matrix can be expressed as a
multiplication of three matrices
 
σ0 0 · · · 0  T 
v
T
   0 σ1 · · ·
 0   0. 

h = USV = u0 · · · up−1  ..   ..  ,
.
vp−1 T
 
0 0 · · · σp−1

where U is an orthonormal matrix (that is, UT U = Ip×p ) of

size m × p, S is a diagonal matrix of size p × p and V is an
orthonormal matrix of size p × n.
If S has a single non-zero element then the matrix h is
separable.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Non-linear filtering: median filter
A median is a robust estimator of the mean.
The median filter replaces each pixel’s value with the median
of its neighborhood pixel values.
J(m, n) = median(I (m−K , n−K ), . . . , I (m, n), . . . , I (m+K , n+K )).
input image average filtering

Gaussian filtering median filtering

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra

Non-linear filtering: Bilateral filtering

The Bilateral filter is a weighted average of the neighborhood

pixel values. The weights are determined according to two
kernels.
The first kernel depends on the location, while the second
kernel depends on pixel similarity.
P
I (i, j)w (x, y , i, j)
i,j
J(x, y ) = P
i,j w (x, y , i, j)

(x − i)2 + (y − j)2 kI (x, y ) − I (i, j)k2

w (x, y , i, j) = exp(− − )
2σ12 2σ22

Limits the influence of far-away pixels and near-by edges.

Ayelet Heimowitz Computer Vision, Image Processing and Computational Photogra