Exercises from Linear algebra, signal

processing, and wavelets. A unified

approach.
Python version

Øyvind Ryan

Sep 26, 2018

Contents

1 Sound and Fourier series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Digital sound and Discrete Fourier analysis . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Discrete time filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4 Motivation for wavelets and some simple examples . . . . . . . . . . . . . . . . 31

5 The filter representation of wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

6 Constructing interesting wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

7 The polyphase representation of filter bank transforms . . . . . . . . . . . . 51

8 Digital images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

9 Using tensor products to apply wavelets to images . . . . . . . . . . . . . . . . . 67

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3
Chapter 1
Sound and Fourier series

Exercise 1.9: Playing sound with added noise

Removing noise from recorded sound can be very challenging, but adding noise is simple.
There are many kinds of noise, but one kind is easily obtained by adding random
numbers to the samples of a sound. For this we can use the function random.random as
follows.

z = x + c*(2*random.random(shape(x))-1)

This adds noise to all channels. The function for returning random numbers returns
numbers between 0 and 1, and above we have adjusted these so that they are between −1
and 1 instead, as for other sound which can be played by the computer. c is a constant
(usually smaller than 1) that dampens the noise.
Write code which adds noise to the audio sample file, and listen to the result for
damping constants c=0.4 and c=0.1. Remember to scale the sound values after you
have added noise, since they may be outside [−1, 1].

Solution.

The following code can be used.

z = x + c*(2*random.random(shape(x))-1)
z /= abs(z).max()
play(z, fs)

With c = 0.4 the resulting sound can be found in the file castanetsnoisehigh.wav, while
with c = 0.1 it can be found in the file castanetsnoiselow.wav.

Exercise 1.11: Playing the notes in an octave

In music theory, an octave is a set of pure tones at frequencies f0 , ..., f11 , f12 so that the
ratio of neighboring tones are the same, and so that f12 is double the frequency of f0 ,
i.e. so that

5
6 1 Sound and Fourier series

f1 f2 f12
= = ··· = = 21/12 .
f0 f1 f11
Make a program which plays all the pure tones in an octave, and listen to it with
f0 = 440Hz.

Solution.

The following code can be used.

f_s = 44100
num_sec = 1
k = 2**(1/12)
f = 440
t = linspace(0, num_sec, f_s*num_sec)
for s in range(13):
play(sin(2*pi*f*t), f_s)
time.sleep(num_sec)
f *= k

Exercise 1.17: Listening to the Fourier series of the triangle

wave

a) Plot the Fourier series of the triangle wave.

Solution.

The following code can be used.

t = linspace(0, T, 100)
x = zeros(len(t))
for k in range(1, 20, 2):
x -= (8/(k*pi)**2)*cos(2*pi*k*t/T)
plt.figure()
plt.plot(t, x, ’k-’)

b) Write code so that you can listen to the Fourier series of the triangle wave. How
high must you choose N for the Fourier series to be indistinguishable from the triangle
wave itself?

Solution.

The following code can be used.

x = tile(x, antsec/T)
play(x, fs)
1 Sound and Fourier series 7

Exercise 1.23: Fourier series of polynomials

Write down difference equations for finding the Fourier coefficients of f (t) = tk from
those of f (t) = tk−1 , and write a program which uses this recursion to compute the
Fourier coefficients of f (t) = tk . Use the program to verify what you computed in
Exercise 1.22 in the book.

Solution.

Let us define an,k , bn,k as the Fourier coefficients of tk . When k > 0 and n > 0, integration
by parts gives us the following difference equations:

Z T
2
an,k = tk cos(2πnt/T )dt
T 0
 T Z T !
2 T k kT k−1
= t sin(2πnt/T ) − t sin(2πnt/T )dt
T 2πn 0 2πn 0
kT
=− bn,k−1
2πn
Z T
2
bn,k = tk sin(2πnt/T )dt
T 0
 T Z T !
2 T k kT k−1
= − t cos(2πnt/T ) + t cos(2πnt/T )dt
T 2πn 0 2πn 0
Tk kT
=− + an,k−1 .
πn 2πn
When n > 0, these can be used to express an,k , bn,k in terms of an,0 , bn,0 , for which we
Tk
clearly have an,0 = bn,0 = 0. For n = 0 we have that a0,k = k+1 for all k.
The following program computes an,k , bn,k recursively when n > 0.

def find_fourier_coeffs(n, k, T):

ank, bnk = 0, 0
if n==0:
ank = T**k/(k+1)
elif k > 0:
ankprev, bnkprev = find_fourier_coeffs(n, k-1, T)
ank = -k*T*bnkprev/(2*pi*n)
bnk = -T**k/(pi*n) + k*T*ankprev/(2*pi*n)
return ank, bnk

Exercise 1.24: Fourier series of a given polynomial

Use the previous exercise to find the Fourier series for f (x) = − 13 x3 + 12 x2 − 16
3
x + 1 on
the interval [0, 1]. Plot the 9th order Fourier series for this function. You should obtain
the plots from Figure 1.5 in the book.
8 1 Sound and Fourier series

Solution.

The following code generates the left plot of Figure 1.5 in the book.

plt.figure()
polycoeffs = matrix([[1, -3/16, 1/2, -1/3]])
num_terms = 9
ank = matrix(zeros(( shape(polycoeffs)[1], num_terms + 1)))
bnk = ank.copy()
for k in range(shape(polycoeffs)[1]):
for n in range(0, num_terms + 1):
ank[k, n], bnk[k, n] = find_fourier_coeffs(n, k, 1)
t = arange(0, 1, 0.01); t = t.reshape((1,len(t))); t = matrix(t)
n = arange(num_terms + 1); n = n.reshape((len(n), 1)); n = matrix(n)
vals = polycoeffs*(ank*cos(2*pi*n*t) + bnk*sin(2*pi*n*t))
t_plot = array( t.reshape((1, shape(t)[1])) )
powervals = matrix(zeros(( shape(polycoeffs)[1], shape(t)[1] )))
for k in range(shape(polycoeffs)[1]):
powervals[k,:] = t_plot**k
t_plot = array( t.reshape(( shape(t)[1], 1 )) )
polyvals = polycoeffs*powervals
plt.plot(t_plot, polyvals.reshape((shape(polyvals)[1],1)),’r’)
plt.hold(’on’)
plt.axis([0, 1, 0.975, 1])
plt.plot(t.reshape((shape(t)[1],1)), vals.reshape((shape(vals)[1],1)), ’b’)

The polynomial coefficients are stored in the row vector polycoeffs at the top. The
code is written so that it will work for any polynomial: You just need to replace the
vector of polynomial coefficients. The Fourier coefficients are stored in the matrices
ank, bnk, using the function implemented in the previous exercise. Each row in those
matrices correspond to Fourier coefficients of a fixed polynomial. We also multiplied
with matrices containing cosine and sine values at all instances in time.

Exercise 1.29: Complex Fourier series of f (t) = sin2 (2πt/T )

Compute the complex Fourier series of the function f (t) = sin2 (2πt/T ).

Solution.

We have that

 2
1 2πit/T
f (t) = sin2 (2πt/T ) = (e − e−2πit/T
2i
1 1 1 1
= − (e2πi2t/T − 2 + e−2πi2t/T ) = − e2πi2t/T + − e−2πi2t/T .
4 4 2 4
This gives the Fourier series of the function (with y2 = y−2 = −1/4, y0 = 1/2). This
could also have been shown by using the trigonometric identity sin2 x = 12 (1 − cos(2x))
RT
first, or by computing the integral T1 0 f (t)e−2πint/T dt (but this is rather cumbersome).
1 Sound and Fourier series 9

Exercise 1.32: Complex Fourier coefficients of the square wave

Compute the complex Fourier coefficients of the square wave using equation (1.15) in
the book, i.e. repeat the calculations from Example 1.13 in the book for the complex
case. Use Theorem 15 in the book to verify your result.

Solution.

We obtain that

Z T /2 Z T
1 −2πint/T 1
yn = e dt − e−2πint/T dt
T 0 T T /2
 T /2  T
1 T −2πint/T T −2πint/T 1
=− e e +
T 2πin 0 2πin T T /2
1
−e−πin + 1 + 1 − e−πin +


=
2πin (
1 −πin

0, if n is even;
= 1−e = .
πin 2/(πin), if n is odd.

2(1−cos(nπ)
Instead using Theorem 15 in the book together with the coefficients bn = nπ we
computed in Example 1.13 in the book, we obtain

( (
1 1 0, if n is even; 0, if n is even;
yn = (an − ibn ) = − i =
2 2 4/(nπ), if n is odd. 2/(πin), if n is odd.

when n > 0. The case n < 0 follows similarly.

Exercise 1.37: Fourier series of a delayed square wave

Define the function f with period T on [−T /2, T /2) by

(
1, if −T /4 ≤ t < T /4;
f (t) =
−1, if T /4 ≤ |t| < T /2.
f is just the square wave, delayed with d = −T /4. Compute the Fourier coefficients of f
directly, and use Property 4 in Theorem 17 in the book to verify your result.

Solution.

We obtain that
10 1 Sound and Fourier series

T /4
1 −T /4 −2πint/T 1 T /2 −2πint/T
Z Z Z
1 −2πint/T
yn = e dt − e dt − e dt
T −T /4 T −T /2 T T /4
 T /4  −T /4  T /2
1 −2πint/T 1 −2πint/T 1 −2πint/T
=− e + e + e
2πin −T /4 2πin −T /2 2πin T /4
1  −πin/2 
= −e + eπin/2 + eπin/2 − eπin + e−πin − e−πin/2
2πin
1 2
= (2 sin(πn/2) − sin(πn)) = sin(πn/2).
πn πn
The square wave defined in this exercise can be obtained by delaying our original square
wave with −T /4. Using Property 3 in Theorem 17 in the book with d = −T /4 on the
complex Fourier coefficients
(
0, if n is even;
yn =
2/(πin), if n is odd,
which we obtained for the square wave in Exercise 1.32, we obtain the Fourier coefficients

( (
2πin(T /4)/T 0, if n is even; 0, if n is even;
e = 2i sin(πn/2)
2/(πin), if n is odd. πin , if n is odd.
(
0, if n is even;
= 2 .
πn sin(πn/2), if n is odd.

This verifies the result.

Exercise 1.40: Symmetric filters preserve sine- and

cosine-series

An analog filter where λs (ν) = λs (−ν) is also called a symmetric filter.

a) Prove that, if the input to a symmetric filter is a Fourier series which is a cosine- or
sine series, then the output also is a cosine- or sine series.

Solution.

We have that

 
1 2πint/T
s(cos(2πnt/T )) = s (e + e−2πint/T )
2
1 1
= λs (n/T )e2πint/T + λs (−n/T )e−2πint/T
2  2 
1 2πint/T
= λs (n/T ) (e + e−2πint/T ) = λs (n/T ) cos(2πnt/T ),
2

so that s preserves cosine-series. A similar computation holds for sine-series.

1 Sound and Fourier series 11
Ra
b) Show that s(f ) = −a g(s)f (t − s)ds is a symmetric filter whenever g is symmetric
around 0, and zero outside [−a, a].

Solution.

We have that
Z a Z a
−2πiνs
λs (ν) = g(s)e ds = g(s)e2πiνs ds = λs (−ν),
−a −a

so that s is symmetric.

Exercise 1.44: The Fejer summability kernel

The Fejer kernel is defined as

N  
X |n|
FN (t) = 1− e2πint/N .
N +1
n=−N

Clearly FN ∈ VN,T , and of degree N .

 2
a) Show that FN (t) = N1+1 sin(π(N +1)t/T )
sin(πt/T ) , and conclude from this that 0 ≤ FN (t) ≤
T2
4(N +1)t2 .

Hint.

2
Use that π |u| ≤ | sin u| when u ∈ [−π/2, π/2].

Solution.

We compute

N  
2
X |n|
sin (πt/T ) 1− e2πint/N
N +1
n=−N
N  
1 X |n|
= (1 − cos(2πt/T )) 1− e2πint/N
2 N +1
n=−N
 N
 X  
1 1 1 |n|
= − e−2πint/T + − e2πint/T 1− e2πint/N .
4 2 4 N +1
n=−N

When one here multiplies out, one sees that most terms cancel (since −1/4 + 1/2 − 1/4 =
0), and we are left with
12 1 Sound and Fourier series
 
1 1 1 1 1
− e−2πi(N +1)t/T + − e2πi(N +1)t/T = sin2 (π(N + 1)t/T ).
N +1 4 2 4 N +1
2
This proves the claim. Finally, using that π |u| ≤ | sin u| when u ∈ [−π/2, π/2], we obtain
2
1 π2 T2

1 sin(π(N + 1)t/T ) 1
≤ = .
N +1 sin(πt/T ) N + 1 4 π 2 t2 /T 2 4(N + 1)t2

b) Show that FN (t) satisfies the three properties of a summability kernel.

Solution.

The first property of summability is easily seen to hold. Since from a) FN (t) ≥ 0, the
the second property follows from the first. To prove the third, due to a) we only need to
RT
compare with the integral δ t−2 dt, which exists for any δ.
RT PN
c) Show that T1 0 FN (u)f (t − u)du = N1+1 n=0 fn (t).

Hint.

1
PN
Show that FN (t) = N +1 n=0 Dn (t), and use Exercise 1.43 b).

Solution.

We have that

N N
!
1 X 1 X
−2πint/T 2πint/T
Dn (t) = N +1+ (N − n + 1)(e +e )
N + 1 n=0 N +1 n=0
N
X N + 1 − n −2πint/T
=1− (e + e2πint/T )
n=0
N +1
N  
X n
=1− 1− (e−2πint/T + e2πint/T )
n=0
N + 1
N
X  
|n|
= 1− e2πint/T = FN (t).
N +1
n=−N

Using this we obtain that

T N N
1 X 1 T
Z Z
1 1 X
FN (u)f (t − u)du = DN (u)f (t − u)du = fn (t).
T 0 N + 1 n=0 T 0 N + 1 n=0
Chapter 2
Digital sound and Discrete Fourier analysis

Exercise 2.12: DFT of cos2 (2πk/N )

Let x be the vector of length N where xk = cos2 (2πk/N ). What is then FN x?

Solution.

The idea is to express x as a linear combination of the Fourier basis vectors φn , and use
that FN φn = en . We have that

 
1 2πik/N  2
cos2 (2πk/N ) = e + e−2πikn/N
2
1 2πi2k/N 1 1 −2πi2k/N 1 1 1
= e + + e = e2πi2k/N + + e2πi(N −2)k/N
4  2 4  4 2 4
√ 1 1 1
= N φ2 + φ0 + φN −2 .
4 2 4

We here used the periodicity of e2πikn/N , i.e. that e−2πi2k/N = e2πi(N −2)k/N . Since FN
is linear and FN (φn ) = en , we have that

√ √
 
1 1 1
FN (x) = N e2 + e0 + eN −2 = N (1/2, 0, 1/4, 0, . . . , 0, 1/4, 0) .
4 2 4

Exercise 2.14: DFT implementation

Extend the code for the function dft_impl in Example 2.4 in the book so that
1. The function also takes a second parameter called forward. If this is true the DFT is
applied. If it is false, the IDFT is applied. If this parameter is not present, then the
forward transform should be assumed.
2. If the input x is two-dimensional (i.e. a matrix), the DFT/IDFT should be applied
to each column of x. This ensures that, in the case of sound, the FFT is applied to
each channel in the sound when the entire sound is used as input.

13
14 2 Digital sound and Discrete Fourier analysis

Also, write documentation for the code.

Solution.

The library version of this function looks as follows.

def dft_impl(x, forward=True):

"""
Compute the DFT of the vector x using standard matrix
multiplication. To avoid out of memory situations, we do not
allocate the entire DFT matrix, only one row of it at a time.
Note that this function differs from the FFT in that it includes
the normalizing factor 1/sqrt(N). The DFT is computed along axis
0. If there is another axis, the DFT is computed for each element
in this as well.
x: a vector
forward: Whether or not this is forward (i.e. DFT)
or reverse (i.e. IDFT)
"""
y = zeros_like(x).astype(complex)
N = len(x)
sign = -(2*forward - 1)
if ndim(x) == 1:
for n in range(N):
D = exp(sign*2*pi*n*1j*arange(float(N))/N)
y[n] = dot(D, x)
else:
for n in range(N):
D = exp(sign*2*pi*n*1j*arange(float(N))/N)
for s2 in range(shape(x)[1]):
y[n,s2] = dot(D,x[:, s2])
if sign == 1:
y /= float(N)
return y

Exercise 2.16: DFT on complex and real data

Let x1 , x2 be real vectors, and set x = x1 + ix2 . Use Theorem 7 in the book to show
that

1 
(FN (x1 ))k = (FN (x))k + (FN (x))N −k
2
1  
(FN (x2 ))k = (FN (x))k − (FN (x))N −k
2i
This shows that we can compute two DFT’s on real data from one DFT on complex
data, and 2N extra additions.

Solution.

We have that
2 Digital sound and Discrete Fourier analysis 15

(FN (x))k = (FN (x1 + ix2 ))k = (FN (x1 ))k + i(FN (x2 ))k
(FN (x))N −k = (FN (x1 ))N −k + i(FN (x2 ))N −k = (FN (x1 ))k + i(FN (x2 ))k ,

where we have used Property 1 of Theorem 7 in the book. If we take the complex
conjugate in the last equation, we are left with the two equations

(FN (x))k = (FN (x1 ))k + i(FN (x2 ))k

(FN (x))N −k = (FN (x1 ))k − i(FN (x2 ))k .

If we add these we get

1 
(FN (x1 ))k = (FN (x))k + (FN (x))N −k ,
2
which is the first equation. If we instead subtract the equations we get
1  
(FN (x2 ))k = (FN (x))k − (FN (x))N −k ,
2i
which is the second equation

Exercise 2.21: Plotting the interpolant

Implement code where you do the following:

• at the top you define the function f (x) = cos6 (x), and M = 3,
• compute the unique interpolant from VM,T (i.e. by taking N = 2M + 1 samples over
one period), as guaranteed by Proposition 9 in the book,
• plot the interpolant against f over one period.
Finally run the code also for M = 4, M = 5, and M = 6. Explain why the plots coincide
for M = 6, but not for M < 6. Does increasing M above M = 6 have any effect on the
plots?

Solution.

The code can look as follows.

import matplotlib.pyplot as plt

from numpy import *
f = lambda t:cos(t)**6
M = 5
T = 2*pi
N = 2*M + 1
t = linspace(0, T, 100)
x = f(linspace(0, T - T/float(N), N))
y = fft.fft(x, axis=0)/N
s = real(y[0])*ones(len(t))
for k in range(1,int((N+1)/2)):
s += 2*real(y[k]*exp(2*pi*1j*k*t/float(T)))
plt.plot(t, s, ’r’, t, f(t), ’g’)
plt.legend([’Interpolant from V_{M,T}’,’f’])
16 2 Digital sound and Discrete Fourier analysis

Exercise 2.25: Execution times for the FFT

Let us compare execution times for the different methods for computing the DFT.
a) Write code which compares the execution times for an N -point DFT for the following
three cases: Direct implementation of the DFT (as in Example 2.4 in the book), the FFT
implementation used in this chapter, and the built-in fft-function. Your code should
use the sample audio file castanets.wav, apply the different DFT implementations to
the first N = 2r samples of the file for r = 3 to r = 15, store the execution times in a
vector, and plot these. You can use the function time() in the time module to measure
the execution time.

Solution.

The code can look as follows.

x0, fs = audioread(’sounds/castanets.wav’)
kvals = arange(3,16)
slowtime = zeros(len(kvals))
fasttime = zeros(len(kvals))
fastesttime = zeros(len(kvals))
N = 2**kvals
for k in kvals:
x = x0[0:2**k].astype(complex)
start = time()
dft_impl(x)
slowtime[k - kvals[0]] = time() - start
start = time()
fft_impl(x, fft_kernel_standard)
fasttime[k - kvals[0]] = time() - start
start = time()
fft.fft(x, axis=0)
fastesttime[k - kvals[0]] = time() - start
# a.
plt.plot(kvals, slowtime, ’ro-’, \
kvals, fasttime, ’go-’, \
kvals, fastesttime, ’bo-’)
plt.grid(’on’)
plt.title(’time usage of the DFT methods’)
plt.legend([’DFT’, ’Standard FFT’, ’Built-in FFT’])
plt.xlabel(’log_2 N’)
plt.ylabel(’time used [s]’)
plt.show()
plt.figure()
# b.
plt.loglog(N, slowtime, ’ro-’, N, fasttime, ’go-’, N, fastesttime, ’bo-’)
plt.axis(’equal’)
plt.legend([’DFT’, ’Standard FFT’, ’Built-in FFT’])

b) A problem for large N is that there is such a big difference in the execution times
between the two implementations. We can address this by using a loglog-plot. Plot N
against execution times using the function loglog. How should the fact that the number
of arithmetic operations are 8N 2 and 5N log2 N be reflected in the plot?
2 Digital sound and Discrete Fourier analysis 17

Solution.

The two different curves you see should have a derivative approximately equal to one
and two, respectively.
c) It seems that the built-in FFT is much faster than our own FFT implementation,
even though they may use similar algorithms. Try to explain what can be the cause of
this.

Solution.

There may be several reasons for this. One is that Python code runs slowly when
compared to native code, which is much used in the built-in FFT. Also, the built-in fft
has been subject to much more optimization than we have covered here.

Exercise 2.28: FFT algorithm adapted to real data

It is possible to adapt an FFT algorithm to real input. Since yN −n = yn for real input,
there is no additional complexity in computing the second half of the DFT coefficients,
once the first half has been computed. We will now rewrite equation (2.11) in the book
for indices n and N/2 − n as

yn = (DFTN/2 x(e) )n + e−2πin/N (DFTN/2 x(o) )n

yN/2−n = (DFTN/2 x(e) )N/2−n + e−2πi(N/2−n)/N (DFTN/2 x(o) )N/2−n
= (DFTN/2 x(e) )n − e2πin/N (DFTN/2 x(o) )n
= (DFTN/2 x(e) )n − e−2πin/N (DFTN/2 x(o) )n .

We see here that, if we already have computed DFTN/2 x(e) and DFTN/2 x(o) , we need
one additional complex multiplication for each yn with 0 ≤ n < N/4 (since e−2πin/N
and (DFTN/2 x(o) )n are complex). No further multiplications are needed in order to
compute yN/2−n , since we simply conjugate terms before adding them. Again yN/2 must
be handled explicitly with this approach. For this we can use the formula

yN/2 = (DFTN/2 x(e) )0 − (DN/2 DFTN/2 x(o) )0

instead.
a) Conclude from this that an FFT algorithm adapted to real data at each step requires
N/4 complex additions and N/2 complex additions. Conclude from this as before that
an algorithm based
on real data requires MN = O(N log2 N ) real multiplications and
AN = O 32 N log2 N real additions (i.e. half the operation count for complex input).

Solution.

The formulas at the top show that we need only compute the e−2πin/N (DFTN/2 x(o) )n
for 0 ≤ n < N/4, i.e. N/4 complex additions. Since N/2 formulas are listed, N/2 complex
18 2 Digital sound and Discrete Fourier analysis

additions are needed. This translates to N real multiplications, and N/2 + N = 3N/2
real additions. These are exactly half the numbers we obtained for the complex version of
the algorithm, for which we obtained MN = O(2N log2 (N )) and AN = O(3N log2 (N )).

version will require MN = O(N log2 N ) real
It is then not difficult to see that the real
multiplications and AN = O 32 N log2 N real additions.
b) Find an IFFT algorithm adapted to vectors with conjugate symmetry, which has
the same operation count as this FFT algorithm adapted to real data.

Hint.

Consider the vectors z, w with entries zn = yn + yN/2−n ∈ CN/2 and wn = e2πin/N (yn −
yN/2−n ) ∈ CN/2 . From the equations above, how can these be used in an IFFT?

Solution.

Note that

zN/2−n = yN/2−n + yn = yn + yN/2−n = zn

wN/2−n = e2πi(N/2−n)/N (yN/2−n − yn ) = −e−2πin/N (yN/2−n − yn )
= e2πin/N (yn − yN/2−n ) = wn ,

showing that the vectors z and w also have conjugate symmetry. Note also that, if we
add and subtract the equations at the top, we obtain

2(DFTN/2 x(e) )n = yn + yN/2−n = zn

2(DFTN/2 x(o) )n = e2πin/N (yn − yN/2−n ) = wn .

so that

1 1
x(e) = IDFTN/2 z x(o) = IDFTN/2 w.
2 2
The point is that the number of arithmetic operations in computing the vectors z and
w is the same as the FFT in a), so that the overall complexity is the same.

Exercise 2.29: Non-recursive FFT algorithm

Use the factorization in (2.16) in the book to write a kernel function for a non-recursive
FFT implementation. In your code, perform the matrix multiplications in equation
(2.16) in the book from right to left in an (outer) for-loop. For each matrix loop through
the different blocks on the diagonal in an (inner) for-loop. Make sure you have the right
number of blocks on the diagonal, each block being on the form
2 Digital sound and Discrete Fourier analysis 19
 
I DN/2k
.
I −DN/2k
It may be a good idea to start by implementing multiplication with such a simple matrix
first as these are the building blocks in the algorithm. Do this so that everything is
computed in-place. Also compare the execution times with our original FFT algorithm,
as we did in Exercise 2.25, and try to explain what you see in this comparison.

Solution.

The library version of the non-recursive FFT is as follows

def fft_kernel_nonrec(x, forward):

"""
Compute the FFT of x, using a non-recursive FFT algorithm.
x: a bit-reversed version of the input. Should have only one axis
forward: Whether the FFT or the IFFT is applied
"""
N = len(x)
sign = -1
if not forward:
sign = 1
D = exp(sign*2*pi*1j*arange(float(N/2))/N)
nextN = 1
while nextN < N:
k = 0
while k < N:
xe, xo = x[k:(k + nextN)], x[(k + nextN):(k + 2*nextN)]
xo *= D[0::(int(N/(2*nextN)))]
x[k:(k+2*nextN)] = concatenate([xe + xo, xe - xo])
k += 2*nextN
nextN *= 2

If you add the non-recursive algorithm to the code from Exercise 2.25, you will see
that the non-recursive algorithm performs much better. There may be several reasons
for this. First of all, there are no recursive function calls. Secondly, the values in the
matrices DN/2 are constructed once and for all with the non-recursive algorithm. Code
which compares execution times for the original FFT algorithm, our non-recursive
implementation, and the split-radix algorithm of the next exercise, can look as follows:

x0, fs = audioread(’sounds/castanets.wav’)
kvals = arange(3,16)
slowtime = zeros(len(kvals))
fasttime = zeros(len(kvals))
fastesttime = zeros(len(kvals))
N = 2**kvals
for k in kvals:
x = x0[0:2**k].astype(complex)
start = time()
fft_impl(x, fft_kernel_standard)
slowtime[k - kvals[0]] = time() - start
start = time()
fft_impl(x, fft_kernel_nonrec)
fasttime[k - kvals[0]] = time() - start
start = time()
fft_impl(x, fft_kernel_splitradix)
fastesttime[k - kvals[0]] = time() - start
20 2 Digital sound and Discrete Fourier analysis

plt.plot(kvals, slowtime, ’ro-’, \

kvals, fasttime, ’bo-’, \
kvals, fastesttime, ’go-’)
plt.grid(’on’)
plt.title(’time usage of the DFT methods’)
plt.legend([’Standard FFT algorithm’, \
’Non-recursive FFT’, \
’Split radix FFT’])
plt.xlabel(’log_2 N’)
plt.ylabel(’time used [s]’)
plt.show()

Exercise 2.34: Type-IV DCT

N −1
Show that the vectors {cos(2π(n + 1/2)(k + 1/2)/(2N ))}n=0qin RN are orthogonal,
p
with lengths N/2. This means that the matrix with entries N2 cos(2π(n + 1/2)(k +
1/2)/(2N )) is orthogonal. Since this matrix also is symmetric, it is its own inverse. This
(IV)
is the matrix DCTN , which can be used for type IV cosine matrices.

Hint.

Compare with the orthogonal vectors dn , used in the DCT.

Solution.

We can write

n + 21
     
1 2n + 1 1
cos 2π k+ = cos 2π k+ .
2N 2 4N 2
If we consider these as vectors of length 2N , we recognize thesep as the unit
p vectors
d2n+1 in the 2N -dimensional DCT,
√ divided by the factor dn = 2/(2N ) = 1/N , so
that these vectors have length N . To see that these vectors are orthogonal when we
restrict to the first N elements using the property

cos(2π(n + 1/2)(2N − k)/(2N )) = − cos(2π(n + 1/2)k/(2N ))

as follows:
2 Digital sound and Discrete Fourier analysis 21

2N −1
n1 + 21 n2 + 21
     
X 1 1
N δn1 ,n2 = cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
N −1
n1 + 21 n2 + 21
     
X 1 1
= cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
2N −1
n1 + 21 n2 + 21
     
X 1 1
+ cos 2π k+ cos 2π k+
2N 2 2N 2
k=N
N −1
n1 + 12 n2 + 21
     
X 1 1
= cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
N −1
n1 + 21 n2 + 21
     
X 1 1
+ cos 2π k+ cos 2π k+
2N 2 2N 2
k=0
N −1
n1 + 21 n2 + 21
     
X 1 1
=2 cos 2π k+ cos 2π k+ .
2N 2 2N 2
k=0

This shows that

N −1 1
n2 + 21
     
X n1 + 2 1 1 N
cos 2π k+ cos 2π k+ = δn1 ,n2 ,
2N 2 2N 2 2
k=0
p
so that the vectors are orthogonal with lengths N/2.
Chapter 3
Discrete time filters

Exercise 3.2: Some linear operations which are not filters

a) In Example 1.2 in the book we looked at time reversal as an operation on digital

sound. In RN this can be defined as the linear mapping which sends the vector ek to
eN −1−k for all 0 ≤ k ≤ N − 1. Write down the matrix for time reversal, and explain
from this why time reversal is not a filter.

Solution.

The matrix for time reversal is the matrix

 
0 0 ··· 0 1
0 0 · · · 1 0
 
 .. .. .. .... 
. . . . .
 
0 1 · · · 0 0
1 0 ··· 00
This is not a circulant Toeplitz matrix, since all diagonals assume the values 0 and 1, so
that they are not constant on each diagonal. Time reversal is thus not a filter.
b) Consider the linear mapping S which keeps every second component in RN ,
i.e. S(e2k ) = e2k , and S(e2k−1 ) = 0. Write down the matrix for this, and explain
why this operation is not a filter.

Solution.

The matrix for the operation which keeps every second component is
 
1 0 ··· 0 0
0 0 · · · 0 0
 
 .. .. .. .. .. 
. . . . . ,
 
0 0 · · · 1 0
0 0 ··· 0 0

23
24 3 Discrete time filters

where 1 and 0 are repeated in alternating order along the main diagonal. Since the
matrix is not constant on the main diagonal, it is not a circulant Toeplitz matrix, and
hence not a filter.

Exercise 3.5: Replacing a circular convolution with a longer

one

Let s, x ∈ RN . For M ≥ 2N − 1 define the vectors

s̃ = (s0 , s1 , ..., sN −1 , 0, ..., 0, s1 , ..., sN −1 )T

x̃ = (x0 , x1 , ..., xN −1 , 0, ..., 0)T

in RM (in s̃, M − (2N − 1) zeros have been inserted in the middle). Show that the first
N elements of s̃ ~ x̃ equal s ~ x, so that a circular convolution can be computed as part
of a longer circular convolution. A good strategy may now be to choose M as a power
of two, since circular convolution can be computed with the help of the DFT, and since
there are efficient DFT algorithms when the number of points is a power of two.

Solution.

Circular convolution with s and s̃ correspond to N × N - and M × M circulant Toeplitz

matrices. Since x̃ has zero entries after the first N entries, the result would follow if
those two circulant Toeplitz matrices have equal upper left N × N corners. That this is
the case is straightforward to see.

Exercise 3.6: Replacing a sparse circular convolution with

shorter ones

Let s, x ∈ RN , and assume that s has at most k nonzero entries, gathered in one
segment.
a) Show that there exists a s̃ ∈ RM +k−1 and a number a so that, for any r,

(s ~ x)r,r+1...,r+M −1 = (s̃ ~ x̃)0,...,M −1 ,

where

x̃ = (xa+r , xa+r+1 , ..., xa+r+M +k−2 ).

In other words, any set of M consecutive entries in s ~ x can be obtained from a circular
convolution of size M + k − 1. Thus, a circular convolution of size N can be computed
N
as M circular convolutions of size M + k − 1.
3 Discrete time filters 25

Solution.

Denote by S the N × N circulant Toeplitz matrix with first column equal to s. Assume
that the nonzero entries in the first row of S occur in the interval [a, a + k − 1]. Then
the nonzero entries in rows r, ..., r + M − 1 of S are in the columns from a + r to
a + r + M − 1 + k − 1 = a + r + M + k − 2, i.e. the interval a + [r, r + M + k − 2] (there
are M + k − 1 such columns). With

x̃ = (xa+r , xr+1 , ..., xa+r+M +k−2 ),

and viewing the submatrix of S from rows r, ..., r + M − 1 and columns from a + [r, r +
M + k − 2] as the first M rows of a (M + k − 1) × (M + k − 1)-circulant Toeplitz matrix
S̃ with first column s̃, we obtain immediately that

(s ~ x)r,r+1...,r+M −1 = (s̃ ~ x̃)0,...,M −1 .

b) It is natural to implement the shorter circular convolutions in terms of the FFT. Use
this strategy with M + k − 1 = 2r to find the number of operations needed to compute
N
the M circular convolutions from a), and write a program which finds the r which gives
the lowest number of operations. Create a plot where k is plotted against the number of
operations corresponding to the optimal r, and also against the number of operations
required by a direct implementation of circular convolution.

Solution.

Recall that an N -point split-radix FFT requires 4N log2 N −6N +8 arithmetic operations.
With M + k − 1 = 2r for N this gives (4r − 6)2r + 8 operations for each of the shorter
convolutions. A 2r -point convolution implemented in terms of the FFT (or rather FFT,
multiplication in frequency, and IFFT) thus requires

2((4r − 6)2r + 8) + 6 · 2r = (8r − 6)2r + 16

operations. The factor 2 comes from computing both an FFT and an IFFT, and the
summand 6 · 2r comes from the multiplications in frequency (each complex multiplication
requires 4 real multiplications and 2 real additions, a total of 6 operations for each of
the 2r entries). In total forl all the mN/M (or rather dN/M e) shorter convolutions we
N
thus need ((8r − 6)2r + 16) 2r −k+1 operations. In a direct implementation, for each of
the N entries we need to make k complex multiplications and k − 1 complex additions.
Since each complex addition requires 2 real additions, this totals 6k + 2(k − 1) = 8k − 2
operations for each entry, so that the overall total is (8k − 2)N operations.
The following code finds the r which gives the minimum of ((8r −6)2r +16)/(2r −k +1)
for a fixed k, and then stores that minimum value. This is then repeated for all the k,
and compared with 8k − 2, as we deduced for a direct implementation.

kvals = arange(1, 51)

opt_vals = kvals.copy()
for k in kvals:
rvals = arange(ceil(log2(k)),50)
num_ops = ((8*rvals-6)*2**rvals + 16)/(2**rvals-k+1)
opt_vals[k-1] = num_ops.min()
plt.plot(kvals, 8*kvals-2,kvals,opt_vals)
26 3 Discrete time filters

Although this code does not plot the optimal value for r, this value is very much needed
as a guideline for implementations on how to split convolutions into shorter convolutions.
The reason that we iterate r from log2 k in the code is that k ≤ 2r secures that there is
room for all the filter coefficients inside the vector s̃ used in the smaller convolution.
From the resulting plot we see that when the number of filter coefficients gets above 5,
the FFT-based approach works better. When the number of filter coefficients is below
this, a direct implementation of the filter should be chosen.
When the number of points in an FFT gets high, one usually not cares about the
terms −6N + 8 from the exact operation count of the Split radix FFT, since they
are not dominant terms. In this exercise these terms really mattered, however, since
short FFT’s are in focus. If you had dropped the terms −6N + 8, you can verify that
((8r − 6)2r + 16)/(2r − k + 1) should be replaced by (8r + 6)2r /(2r − k + 1), and that a
new plot shows that at least 8 filter coefficients are needed in order for the FFT-based
approach to outperform the direct implementation.

Exercise 3.7: Dualities between circular convolution and

multiplication

Show that

IDF TN (λ1 ◦ λ2 ) = (IDF TN λ1 ) ~ (IDF TN λ2 )

1
DF TN (s1 ◦ s2 ) = (DF TN s1 ) ~ (DF TN s2 )
N
IDF TN (λ1 ~ λ2 ) = N (IDF TN λ1 ) ◦ (IDF TN λ2 )

Note that there is a factor in front of the last two equations, so that one must be a bit
careful here.

Solution.

Setting λ1 = DF TN s1 , λ1 = DF TN s2 , and taking the IDFT on both sides in

DF TN (s1 ~ s2 ) = (DF TN s1 ) ◦ (DF TN s2 ), it follows that

(IDF TN λ1 ) ~ (IDF TN λ2 ) = IDF TN (λ1 ◦ λ2 ),

which is the first equation. Conjugating both sides in this equation, and using that
IDF TN = N1 DF TN , we obtain that
1 1
2
(DF TN λ1 ) ~ (DF TN λ2 ) = DF TN (λ1 ◦ λ2 ).
N N
Conjugating again gives the second equation. The last equation follows by taking the
DFT on both sides of this.
3 Discrete time filters 27

Exercise 3.18: Passing between compact filter notation and

matrix form

a) Assume that a filter is defined by the formula

1 1 1 1
zn =xn+1 + xn + xn−1 + xn−2 .
4 4 4 4
Write down the corresponding compact filter notation, and the corresponding circulant
Toeplitz matrix when N = 8.

Solution.

Here we have that t−1 = 1/4, t0 = 1/4, t1 = 1/4, and t2 = 1/4. We now get that
s0 = t0 = 1/4, s1 = t1 = 1/4, and s2 = t2 = 1/4 (first formula), and sN −1 = s7 =
t−1 = 1/4 (second formula). This means that S = {= 1/4, 1/4, 1/4, 1/4}, and that the
corresponding circulant Toeplitz matrix is
 
11000011
1 1 1 0 0 0 0 1
 
1 1 1 1 0 0 0 0
 
1 0 1 1 1 1 0 0 0
S=   .
4 0 0 1 1 1 1 0 0
0 0 0 1 1 1 1 0
 
0 0 0 0 1 1 1 1
10000111
b) Given the circulant Toeplitz matrix
 
1 2 0 0
0 1 2 0
S=
0
.
0 1 2
2 0 0 1
Write down the corresponding compact filter notation.

Solution.

We obtain that S = {2, 1}.

Exercise 3.23: Filters with coefficients being powers

Assume that S = {1, c, c2 , . . . , ck }. Compute and plot λS (ω) when k = 4 and k = 8, for
c = 0.5. How do the choices of k and c influence the frequency response?

Solution.

The frequency response is

28 3 Discrete time filters

k
X 1 − ck+1 e−i(k+1)ω
cs e−isω = .
s=0
1 − ce−iω

It is straightforward to compute the limit as ω → 0 as ck (k + 1). This means that as we

increase k or c, this limit also increases. The value of k also dictates oscillations in the
frequency response, since the numerator oscillates fastest. When c = 1, k dictates how
often the frequency response hits 0.

Exercise 3.25: Execution times for convolution

Implement code which computes t ∗ x in the vectorized- and non-vectorized ways

described in Section 3.2.1 in the book (i.e. as a single loop over k or n with the other
variable vectorized, or a double loop). As your t, take k randomly generated numbers.
Compare execution times with the convolve function, for different values of k. Present
the result as a plot where k runs along the x-axis, and execution times run along the
y-axis. Your result will depend on how vectorization is performed by the computing
environment.

Solution.

The code can look as follows.

x, fs = audioread(’sounds/castanets.wav’)
x = x[:,0]
N= len(x)
kmax=100
vals1 = zeros(int(kmax/10))
vals2 = zeros(int(kmax/10))
vals3 = zeros(int(kmax/10))
ind = 0
for k in range(10, kmax+1,10):
t = random.random(k)
start = time()
convolve(t, x)
vals1[ind] = time() - start
z = zeros(N)
start = time()
for s in range(k):
z[(k-1):N] += t[s]*x[(k-1-s):(N-s)]
vals2[ind] = time() - start
z = zeros(N)
start = time()
for n in range(k-1, N):
for s in range(k):
z[n] += t[s]*x[n-s]
vals3[ind] = time() - start
ind += 1
plt.plot( range(10, kmax+1,10),log(vals1), ’r-’, \
range(10, kmax+1,10), log(vals2), ’g-’, \
range(10, kmax+1,10), log(vals3), ’b-’, )
plt.legend([’conv’,’simple for’,’double for’])
3 Discrete time filters 29

Exercise 3.26: Implementing convolution in-place

Filters are often implemented in hardware, and hardware implementations have strong
limitations on the number of local variables. Assume that a filter t has n nonzero
coefficients, all known to the hardware, and that the vector x is input. Show that t ∗ x
can be implemented in-place (i.e. that t ∗ x can be computed and stored directly in x)
by using n − 1 local variables. You can assume that two local variables can be swapped
without using an additional local variable. It follows that, since any real convolution
can be split into real convolutions with three filter coefficients, only 2 local variables are
needed in order to compute any convolution in-place.

Solution.

Assume for simplicity that the nonzero filter coefficients are t0 , ..., tn−1 . We have that
x0 is needed in order to compute z0 , ..., zn−1 . Therefore, z0 can’t be stored into the
location of x0 until zn−1 has been computed. When we start the computation of zn−1 ,
z0 , ..., zn−2 must be stored in n − 1 local variables. The final contribution of x0 is with
x0 tn−1 in zn−1 . Now, multiply x0 in-place with tn−1 , and compute zn−1 in the same
memory location. Finally, swap the result with the local variable holding z0 . As a result,
z0 has replaced x0 in the input memory, and the local variables hold z1 , ..., zn−1 instead.
This is continued incrementally.
When the first nonzero filter coefficient is not t0 , it is clear that this procedure will
still work, but that the output will be a delayed version.

Exercise 3.36: Applying low-pass- and high-pass filters

deduced from Pascal’s triangle to the audio sample file

a) Write code where you apply the low-pass and high-pass filters in examples 3.34
and 3.35 in the book to the audio sample file, and verify that the sounds you get are
the same as in these examples. How high must k be in order for you to hear difference
from the actual sound? How high can you choose k and still recognize the sound at all?
If you solved Exercise 3.28 you can also use the function filter_impl to perform the
filtering, rather than using convolution (which, as mentioned, discards circularity).

Solution.

The code can look like this, when we apply low-pass filters.

z = x[:, 0]
for kval in range(k):
z = convolve(z, [1/2., 1/2.])

play(z, fs)

If we apply high-pass filters instead, the code can look like this.
30 3 Discrete time filters

z = x[:, 0]
for kval in range(k):
z = convolve(z, [1/2., -1/2.])

play(z, fs)

b) In your code, it is not necessary to scale the values after applying the low-pass or
high-pass filters so that values fit inside [−1, 1]. Explain why this is the case.

Solution.

The low-pass filters compute weighted averages of the input samples. Therefore, as long
as the input values are inside the legal range[−1, 1], the output values will as well. Since
the filter coefficients sum to one it is easy to see that also the high-pass filters produce
values in the legal range.

Exercise 3.39: Filters in the MP3 standard

Assume that tk are the filter coefficients of a filter S1 , and that S2 is the filter with filter
coefficients cos(kωc )tk , where ωc ∈ [0, π). Show that
1
λS2 (ω) = (λS1 (ω − ωc ) + λS1 (ω + ωc )).
2
In other words, when we multiply (modulate) the filter coefficients with a cosine, the
new frequency response can be obtained by shifting the old frequency response with ωc
in both directions, and taking the average of the two. We saw that the filters in the MP3
standard were constructed in this way. The collection of these filters are also called a
cosine modulated filter bank. It is also possible to construct DFT-modulated filter banks
where the shift in frequency is obtained by multiplying with a complex exponential
instead, but this produces filters with complex coefficients.

Solution.

We have that

X 1 X ikωc
λS2 (ω) = cos(kωc )tk e−ikω = (e + e−ikωc )tk e−ikω
2
k k
!
1 X
−ik(ω−ωc )
X
−ik(ω+ωc )
= tk e + tk e
2
k k
1
= (λS1 (ω − ωc ) + λS1 (ω + ωc )).
2
Chapter 4
Motivation for wavelets and some simple
examples

Exercise 4.2: Realizing the DWT in terms of averages and

differences

Let f (t) ∈ V1 , and let fn,1 be the value f attains on [n, n + 1/2), and fn,2 the value f
attains on [n + 1/2, n + 1). Show that
• projV0 (f ) is the function in V0 which equals (fn,1 + fn,2 )/2 on the interval [n, n + 1).
• projW0 (f ) is the function in W0 which is (fn,1 − fn,2 )/2 on [n, n + 1/2), and −(fn,1 −
fn,2 )/2 on [n + 1/2, n + 1).

Hint.

Apply Lemma 10 in the book.

Solution.

Since f is constant and equal to f (n) on [n, n+1/2), and constant and equal to f (n+1/2)
on [n + 1/2, n + 1), we get that

Z N Z n+1
hf, φ0,n i = f (t)φ0,n (t)dt = f (t)dt
0 n
Z n+1/2 Z n+1
= f (t)dt + f (t)dt
n n+1/2
Z n+1/2 Z n+1
= f (n)dt + f (n + 1/2)dt
n n+1/2

= f (n)/2 + f (n + 1/2)/2 = (f (n) + f (n + 1/2))/2.

The orthogonal decomposition theorem gives that

N −1 N −1
X X f (n) + f (n + 1/2)
projV0 f = hf, φ0,n iφ0,n = φ0,n .
n=0 n=0
2

31
32 4 Motivation for wavelets and some simple examples

Since φ0,n is 1 on [n, n + 1) and 0 elsewhere, projV0 f is the piecewise constant function
which is equal to (f (n) + f (n + 1/2))/2 on [n, n + 1).

Exercise 4.6: Properties of block diagonal matrices

Assume that A, B, C, and D are square matrices of the same dimensions.

a) Assume that the eigenvalues of A are equal to those of B. What are the eigenvalues
of diag(A, B)? Can you express the eigenvectors of diag(A, B) in terms of those of A
and B?

Solution.

Assume that λ is an eigenvalue common to both A and B. Then there exists a vector v1
so that Av1 = λv1 , and a vector v2 so that Bv2 = λv2 . We now have that

          
v A 0 v1 Av1 λv1 v
diag(A, B) 1 = = = =λ 1 .
v2 0 B v2 Bv2 λv2 v2
 
v1
This shows that λ is an eigenvalue for diag(A, B) also, with an eigenvector.
v2
b) Assume that A and B also are non-singular. Show that diag(A, B) is non-singular,
and that (diag(A, B))−1 = diag(A−1 , B −1 ).

Solution.

We have that

   −1 
−1 −1 A 0 A 0
diag(A, B)diag(A ,B )=
0 B 0 B −1
AA−1 0
   
I0
= = =I
0 BB −1 0I

where we have multiplied as block matrices. This proves that diag(A, B) is non-singular,
with inverse diag(A−1 , B −1 ).
c) Show that diag(A, B)diag(C, D) = diag(AC, BD).

Solution.

We have that

    
A 0 C 0 AC 0
diag(A, B)diag(C, D) = = = diag(AC, BD)
0 B 0 D 0 BD
4 Motivation for wavelets and some simple examples 33

where we again have multiplied as block matrices.

Exercise 4.17: Computing the DWT of a simple vector

Suppose that we have the vector x with length 210 = 1024, defined by xn = 1 for n
even, xn = −1 for n odd. What will be the result if you run a 10-level DWT on x? Use
the function dwt_impl to verify what you have found.

Hint.
√
We defined ψ by ψ(t) = (φ √1,0 (t) − φ1,1 (t))/ 2. From this connection
√ it follows that
ψ9,n = (φ10,2n − φ10,2n+1 )/ 2, and thus φ10,2n − φ10,2n+1 = 2ψ9,n . Try to couple this
identity with the alternating sign you see in x.

Solution.
P1023
The vector x is the coordinate vector of the function f (t) = n=0 (−1)n φ10,n in the
√ P511 √
basis φ10 for V10 . Since φ10,2n − φ10,2n+1 = 2ψ9,n , we can write f (t) = n=0 2ψ9,n .
Since a 10-level-DWT gives as a result the coordinate vector of f in

(φ0 , ψ0 , ψ1 , ψ2 , ψ3 , ψ4 , ψ5 , ψ6 , ψ7 , ψ8 , ψ9 ),
(the DWT is √ nothing but the change of coordinates from φ10 to this basis), and√since
P511
f (t) = n=0 2ψ9,n , it is clear that the coordinate vector of f in this basis has 2 in
the second part (the ψ9 -coordinates), and 0 elsewhere. The 10-level DWT of x√therefore
gives the vector of length 1024 which is 0 on the first half, and equal to 2 on the
second half. m = 10 is here arbitrarily chosen: The result would have been the same for
m = 1, m = 2, and so on. The following code verifies the result:

x = tile([1.,-1.], 512)
dwt_impl(x, ’Haar’, m=10)
print(x)

Exercise 4.22: The vector of samples is the coordinate vector

Show that, for f ∈ V0 we have that [f ]φ0 = (f (0), f (1), . . . , f (N − 1)). This shows that,
also for the piecewise linear wavelet, there is no loss in working with the samples of f
rather than f itself.

Solution.
PN −1
Let us write f (t) = n=0 cn φ0,n (t). If k is an integer we have that
34 4 Motivation for wavelets and some simple examples

N
X −1 N
X −1
f (k) = cn φ0,n (k) = cn φ(k − n).
n=0 n=0

Clearly the only integer for which φ(s) = 6 0 is s = 0 (since φ(0) = 1), so that the
only n which contributes in the sum is n = k. This means that f (k) = ck , so that
[f ]φ0 = (f (0), f (1), . . . , f (N − 1)).

Exercise 4.24: Implementation of elementary lifting of odd

type

Write a function

lifting_odd_symm(lambda, x, bd_mode)

which applies an elementary lifting matrix of odd type (equation (4.28) in the book)
to x. Assume that N is even. The parameter bd_mode should do nothing, as we will
return to this parameter later. The function should not allocate the full matrix, and
apply as few multiplications as possible. How many local variables are needed in your
code? Compare with the corresponding number you found for filters in Exercise 3.26.

Solution.

The library version of this function is as follows.

def lifting_odd_symm(lmbda, x, bd_mode):

if (bd_mode.lower() == ’per’) and mod(len(x), 2) != 0:
raise AssertionError()
x[1:-1:2] += lmbda*(x[0:-2:2] + x[2::2])
if mod(len(x), 2) == 0:
if bd_mode.lower() == ’symm’:
x[-1] += 2*lmbda*x[-2] # With symmetric extension
else:
x[-1] += lmbda*(x[0]+x[-2])

Since λ appears twice in each row, note that we have added entries before multiplication,
to avoid extra multiplication. We also see that there is no need for local variables in the
code.

Exercise 4.30: Two vanishing moments

In this exercise we will show that there is a unique function on the form given by
equation (4.30) in the book which has two vanishing moments.
a) Show that, when ψ̂ is defined by equation (4.30) in the book, we have that
4 Motivation for wavelets and some simple examples 35



−αt − α for − 1 ≤ t < 0
(2 + α − β)t − α for 0 ≤ t < 1/2



ψ̂(t) = (α − β − 2)t − α + 2 for 1/2 ≤ t < 1

βt − 2β for 1 ≤ t < 2





0 for all other t

Solution.

The function ψ̂ is a sum of the functions ψ = √12 φ1,1 , φ, and φ0,1 (i.e. we have set n = 0
in equation (4.30) in the book). All these are continuous and piecewise linear, and we
can write


2t
 0 ≤ t < 1/2
ψ(t) = 2 − 2t 1/2 ≤ t < 1

0 elsewhere


1 + t
 −1 ≤ t < 0
φ0,0 (t) = 1 − t 0≤t<1

0 elsewhere


t
 0≤t<1
φ0,1 (t) = 2 − t 1≤t<2 .

0 elsewhere


It follows that ψ̂(t) = ψ(t) − αφ(t) − βφ1,1 is piecewise linear, and linear on the segments
[−1, 0], [0, 1/2], [1/2, 1], [1, 2].
On the segment [−1, 0] only the function φ is seen to be nonzero, and since φ(t) = 1+t
here, we have that ψ̂(t) = −α(1 + t) = −α − αt here.
On the segment [0, 1/2] all three functions are nonzero, and

φ1,1 (t) = 2t
φ0,0 (t) = 1 − t
φ0,1 (t) = t

on this interval. This means that ψ̂(t) = 2t − α(1 − t) − βt = (2 + α − β)t − α on [0, 1/2].
On the segment [0, 1/2] all three functions are nonzero, and

φ1,1 (t) = 2 − 2t
φ0,0 (t) = 1 − t
φ0,1 (t) = t

on this interval. This means that ψ̂(t) = 2 − 2t − α(1 − t) − βt = (α − β − 2)t − α + 2

on [1/2, 1].
36 4 Motivation for wavelets and some simple examples

On the segment [1, 2] only the function φ0,1 is seen to be nonzero, and since φ0,1 (t) =
2 − t here, we have that ψ̂(t) = −β(2 − t) = βt − 2β here. For all other values of t, ψ̂ is
zero. This proves the formulas for ψ̂ on the different intervals.
b) Show that

Z N Z N
1 1
ψ̂(t)dt = − α − β, tψ̂(t)dt = − β.
0 2 0 4

Solution.

We can write

Z N Z 2 Z 0 Z 1/2 Z 1 Z 2
ψ̂(t)dt = ψ̂(t)dt = ψ̂(t)dt + ψ̂(t)dt + ψ̂(t)dt + ψ̂(t)dt
0 −1 −1 0 1/2 1
Z 0 Z 1/2
= (−α − αt)dt + (2 + α − β)t − α)dt
−1 0
Z 1 Z 2
+ ((α − β − 2)t − α + 2)dt + (βt − 2β)dt
1/2 1
 0  1/2
1 2 1 2
= −αt − αt + (2 + α − β)t − αt
2 −1 2 0
 1  2
1 2 1 2
+ (α − β − 2)t + (2 − α)t + βt − 2βt
2 1/2 2 1
1 1 1 3 1 3
= −α + α + (2 + α − β) − α + (α − β − 2) + (2 − α) + β − 2β
2 8 2 8 2 2
1
= − α − β,
2
RN 1
0
tψ̂(t)dt is computed similarly, so that we in the end arrive at 4 − β.
c) Explain why there is a unique function on the form given by equation (4.30) in the
book which has two vanishing moments, and that this function is given by equation
(4.32) in the book.

Solution.

The equation system

1
−α−β =0
2
1
−β =0
4
has the unique solution α = β = 14 , which we already have found.
Chapter 5
The filter representation of wavelets

Exercise 5.5: Passing between the MRA-matrices and the

filters

a) Assume that we have the forward filter bank transform

 
1/5 1/5 1/5 0 0 0 · · · 0 1/5 1/5
−1/3 1/3 −1/3 0 0 0 ··· 0 0 0 
 
 1/5 1/5 1/5 1/5 1/5 0 · · · 0 0 0 
H= 
 0 0 −1/3 1/3 −1/3 0 · · · 0 0 0 
.. .. .. .. .. .. .. .. .. ..
 
. . . . . . . . . .
Write down the corresponding filters H0 , H1 , and compute and plot the frequency
responses. Are the filters symmetric?

Solution.

We have that H0 = 15 {1, 1, 1, 1, 1}, and H1 = 13 {−1, 1, −1}. The frequency responses are

1 2iω 1 iω 1 1 −iω 1 −2iω

λH0 (ω) = e + e + + e + e
5 5 5 5 5
2 2 1
= cos(2ω) + cos ω +
5 5 5
1 iω 1 1 −iω 2 1
λH1 (ω) = − e + − e = − cos ω + .
3 3 3 3 3
Both filters are symmetric.
b) Assume that we have the reverse filter bank transform

37
38 5 The filter representation of wavelets
 
1/2 −1/4 0 0 ···
1/4 3/8
 1/4 1/16 · · ·

 0 −1/4 1/2 −1/4 · · ·
 
 0 1/16
 1/4 3/8 · · ·
 0 0 0 −1/4 · · ·
G= 0
 
 0 0 1/16 · · ·
 0
 0 0 0 · · ·
 .. .. .. .. .. 
 . . . . . 
 
 0 0 0 0 · · ·
1/4 1/16 0 0 ···
Write down the filters G0 , G1 , and compute and plot the frequency responses. Are the
filters symmetric?

Solution.

We have that G0 = {1/4, 1/2, 1/4}, and G1 = {1/16, −1/4, 3/8, −1/4, 1/16}. The
frequency responses are

1 iω 1 1 −iω
λG0 (ω) = e + + e
4 2 4
1 1
= cos(ω) +
2 2
1 2iω 1 iω 3 1 −iω 1 −2iω
λG1 (ω) = e − e + − e e
16 4 8 4 16
1 1 3
= cos(2ω) − cos ω + .
8 2 8
Both filters are symmetric.
c) Assume that H0 = {1/16, 1/4, 3/8, 1/4, 1/16}, and H1 = {−1/4, 1/2, −1/4}. Plot
the frequency responses of H0 and H1 , and verify that H0 is a low-pass filter, and that
H1 is a high-pass filter. Also write down the corresponding forward filter bank transform
H.

Solution.

The frequency responses are

1 2iω 1 iω 3 1 −iω 1 −2iω

λH0 (ω) = e + e + + e e
16 4 8 4 16
1 1 3
= cos(2ω) + cos ω +
8 2 8
1 iω 1 1 −iω
λH1 (ω) = − e + − e
4 2 4
1 1
= − cos(ω) + .
2 2
The two first rows in H are
5 The filter representation of wavelets 39

 
3/8 1/4 1/16 0 · · · 1/16 1/4
−1/4 1/2 −1/4 0 · · · 0 0
The remaining rows are obtained by translating these in alternating order.
d) Assume that G0 = 13 {1, 1, 1}, and G1 = 15 {1, −1, 1, −1, 1}. Plot the frequency
responses of G0 and G1 , and verify that G0 is a low-pass filter, and that G1 is a
high-pass filter. Also write down the corresponding reverse filter bank transform G.

Solution.

The frequency responses are

1 iω 1 1 −iω 2 1
λG0 (ω) = e + + e = cos ω +
3 3 3 3 3
1 2iω 1 iω 1 1 −iω 1 −2iω
λG1 (ω) = e − e + − e e
5 5 5 5 5
2 2 1
= cos(2ω) − cos ω +
5 5 5
The two first columns in G are
 
1/3 −1/5
1/3 1/5 
 
 0
 −1/5
 0 1/5 
 
 0 0 
 
 .. .. 
 . . 
 
 0 0 
1/3 1/5
The remaining columns are obtained by translating these in alternating order.

Exercise 5.9: Perfect reconstruction for forward and reverse

filter bank transforms

Combine equations (5.4) and (5.5) in Theorem 3 in the book to show that

1
GHφr = (λH0 ,r λG0 ,r + λH1 ,r λG1 ,r )φr
2
1
+ (λH0 ,r λG0 ,r+N/2 − λH1 ,r λG1 ,r+N/2 )φr+N/2 .
2
Conclude that, in terms of continuous frequency responses, we have alias cancellation if
and only if

λH0 (ω)λG0 (ω + π) = λH1 (ω)λG1 (ω + π),

40 5 The filter representation of wavelets

and perfect reconstruction if and only if in addition

λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω) = 2.

These are also called the alias cancellation- and perfect reconstruction conditions, re-
spectively.

Solution.

We have that

GHφr
1 1
= (λH0 ,r + λH1 ,r )Gφr + (λH0 ,r − λH1 ,r )Gφr+N/2
2  2 
1 1 1
= (λH0 ,r + λH1 ,r ) (λG0 ,r + λG1 ,r )φr + (λG0 ,r+N/2 − λG1 ,r+N/2 )φr+N/2
2 2 2
 
1 1 1
+ (λH0 ,r − λH1 ,r ) (λG0 ,r+N/2 + λG1 ,r+N/2 )φr+N/2 + (λG0 ,r − λG1 ,r )φr )
2 2 2
1
= ((λH0 ,r + λH1 ,r )(λG0 ,r + λG1 ,r ) + (λH0 ,r − λH1 ,r )(λG0 ,r − λG1 ,r )) φr
4
1

+ (λH0 ,r + λH1 ,r )(λG0 ,r+N/2 − λG1 ,r+N/2 ) + (λH0 ,r − λH1 ,r )(λG0 ,r+N/2 + λG1 ,r+N/2 ) φr+N/2
4
1 1
= (λH0 ,r λG0 ,r + λH1 ,r λG1 ,r )φr + (λH0 ,r λG0 ,r+N/2 − λH1 ,r λG1 ,r+N/2 )φr+N/2 .
2 2

Exercise 5.13: Alternative QMF filter bank transforms

An alternative QMF filter bank is one where G0 = (H0 )T and G1 = (H1 )T (i.e. the filter
coefficients in the forward and reverse transforms are reverse of one-another), and

λH1 (ω) = λH0 (ω + π).

There is some confusion in the literature between classical and alternative QMF filter
banks.
a) Show that, for an alternative QMF filter bank, the two alias cancellation conditions
in Exercise 5.10 a) are satisfied with α = 1, d = 0.

Solution.

If we in the first alias cancellation condition substitute G0 = (H0 )T we get

λH1 (ω) = α−1 e−2idω λG0 (ω + π) = α−1 e−2idω λH0 (ω + π).

If we set α = 1, d = 0, we get equality here. A similar computation follows for the second
alias cancellation condition.
5 The filter representation of wavelets 41

b) Show that the perfect reconstruction condition for alternative QMF filter banks can
be written as

|λH0 (ω)|2 + |λH0 (ω + π)|2 = 2.

Solution.

We have that

2 = λH0 (ω)λG0 (ω) + λH1 (ω)λG1 (ω) = λH0 (ω)λH0 (ω) + λH0 (ω + π)λH0 (ω + π)
= |λH0 (ω)|2 + |λH0 (ω + π)|2 .

We see that the perfect reconstruction conditions of the two definitions of QMF filter
banks only differ in that the latter takes absolute values. It turns out that the latter
also has many interesting solutions, as we will see in Chapter 6 in the book.

Exercise 5.16: The support of the scaling function and the

mother wavelet

The scaling functions and mother wavelets we encounter will always turn out to be
functions with compact support (for a general proof of this, see [1]). An interesting
consequence of equations (5.2) and (5.3) in the book is that the corresponding filters
then are FIR, and that one can find a connection between the supports of the scaling
function and mother wavelet, and the number of filter coefficients. In the following
we will say that the support of a filter is [E, F ] if tE , ..., tF are the only nonzero filter
coefficients.
a) Assume that G0 has support [M0 , M1 ], G1 has support [N0 , N1 ]. Show that Supp(φ) =
[M0 , M1 ], and Supp(ψ) = [(M0 + N0 + 1)/2, (M1 + N1 + 1)/2].

Solution.

Let [m0 , m1 ] be the support of φ. Then φ1,n clearly has support [(m0 + n)/2, (m1 + n)/2].
In the equation
M1
X
φ(t) = (G0 )n φ1,n ,
n=M0

the function with the leftmost support is φ1,M0 , while the one with the rightmost one is
φ1,M1 . This gives the support [(m0 + M0 )/2, (m1 + M1 )/2] for the sum. In order for the
supports of the two sides to match we clearly must have

m0 = (m0 + M0 )/2 m1 = (m1 + M1 )/2.

42 5 The filter representation of wavelets

It follows that m0 = M0 and m1 = M1 , so that the support of φ is [M0 , M1 ]. Similarly,

let [n0 , n1 ] be the support of ψ. We have that
NX
1 +1

ψ(t) = (G1 )n−1 φ1,n ,

n=N0 +1

and we get in the same way

n0 = (m0 + N0 + 1)/2 n1 = (m1 + N1 + 1)/2.

It follows that n0 = (M0 + N0 + 1)/2. n1 = (M1 + N1 + 1)/2, so that the support of ψ

is ((M0 + N0 + 1)/2, (M1 + N1 + 1)/2).
b) Assume that all the filters are symmetric. Show that Supp(φ) is on the form
[−M1 , M1 ] (i.e. symmetric around 0), and that Supp(ψ) is on the form 1/2 + [−(M1 +
N1 )/2, (M1 + N1 )/2], i.e. symmetric around 1/2. Verify that the alternative piecewise
linear wavelet has Supp(φ) = [−1, 1], and Supp(ψ) = [−1, 2] (as we already know from
Figure 4.18 in the book), and that the piecewise linear wavelet has Supp(ψ) = [0, 1].

Solution.

Supp(φ) is on the stated form due to a), and since the support of a symmetric G0 has
the form [−M1 , M1 ]. Since N0 = −N1 as well when G1 is symmetric, we have that

Supp(ψ) = [(M0 + N0 + 1)/2, (M1 + N1 + 1)/2]

= [(−M1 − N1 + 1)/2, (M1 + N1 + 1)/2]
= 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2].

For both piecewise linear wavelets M1 = 1 so that Supp(φ) = [−1, 1]. For the first
piecewise linear wavelet we had N0 = N1 = 0 so that

Supp(ψ) = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2] = 1/2 + [−1/2, 1/2] = [0, 1].

For the alternative piecewise linear wavelet we had N0 = −2, N1 = 2, so that

Supp(ψ) = 1/2 + [−(M1 + N1 )/2, (M1 + N1 )/2] = 1/2 + [−3/2, 3/2] = [−1, 2].

c) The wavelet with symmetric filters with most filter coefficients we will consider has
7 and 9 filter coefficients, respectively. Explain why plotting over [−4, 4] captures the
entire support of both the scaling function and the mother wavelet when there are at
most this number of filter coefficients. This explains why [−4, 4] has been used as the
plotting interval in many of the plots.
5 The filter representation of wavelets 43

Solution.

With 7 and 9 filter coefficients it is easily seen that the support of φ is [−3, 3], and the
support of ψ is [−3, 4]. Both these are contained in [−4, 4].
d) Verify that, for the Haar wavelet, G0 has filter coefficients evenly distributed around
1/2, G1 has equally many, and evenly distributed around −1/2, and the supports of
both φ and ψ are symmetric around 1/2. The Haar wavelet is the simplest orthonormal
wavelet, and these properties will be used as a template for the other orthonormal
wavelets.

Solution.

We know that both φ and ψ for the Haar wavelet are supported on [0, 1), so that 1/2
is the midpoint of the support. The symmetries around 1/2 and −1/2 follows from
G0 = √12 {1, 1} and G1 = √12 {1, −1}.

e) We will only consider orthonormal wavelets with at most 8 filter coefficients. Explain
again why [−4, 4] as plotting interval is desirable.

Solution.

This number of filter coefficients is easily seen to give the support [−3, 4], which again is
contained within [−4, 4].

Exercise 5.23: Implementing symmetric extensions

a) The function you implemented in Exercise 3.28 computed circular convolution, i.e. it
assumed a periodic extension of the input. Reimplement that function so that it also
takes a third parameter bd_mode. If this equals "per" a periodic extension should be
assumed. If it equals "symm" a symmetric extension should be assumed, as described
above. Again, the function filter_impl in the library can serve as a template for your
code.

Solution.

The implementation of filter_impl in the library looks as follows, and handles other
boundary modes which we do not cover here as well:

def filter_impl(t, x, bd_mode):

szx = shape(x)
N = szx[0]
n = int(prod(szx[1:]))
y = reshape(x, (N, n))
tlen = len(t); N0 = int((tlen - 1)/2)
w = 0
if bd_mode.lower() == ’symm’:
w = concatenate([ y[N0:0:(-1)], y, y[(N-2):(N - N0 - 2):(-1)] ])
elif bd_mode.lower() == ’per’:
44 5 The filter representation of wavelets

w = concatenate([ y[(N - N0):], y, y[:N0]])

elif bd_mode.lower() == ’none’ or bd_mode.lower() == ’bd’:
w = concatenate( (zeros(N0, n), y, zeros(N0, n)) )
for k in range(n):
z = convolve(t, w[:, k])
y[:, k] = z[(2*N0):(len(z)-2*N0)]
x[:] = reshape(y, szx)

We see that the function numpy.convolve is called

b) Implement functions

dwt_kernel_filters(x, bd_mode, wav_props)

idwt_kernel_filters(x, bd_mode, wav_props)

which return DWT and IDWT kernels that apply the function filter_impl from a)
together with Theorem 2 in the book. The parameter wav_props should be an object
which gives access to the filters by means of writing wav_props.h0, wav_props.h1,
wav_props.g0, and wav_props.g1.

Solution.

The implementation of these functions in the library is as follows:

def _dwt_kernel_filters(x, bd_mode, dual_wav_props):

x0 = x.copy()
x1 = x.copy()
filter_impl(dual_wav_props.g0, x0, bd_mode)
filter_impl(dual_wav_props.g1, x1, bd_mode)
x[::2] = x0[::2]
x[1::2] = x1[1::2]

def _idwt_kernel_filters(x, bd_mode, wav_props):

x0 = x.copy(); x0[1::2] = 0
x1 = x.copy(); x1[::2] = 0
filter_impl(wav_props.g0, x0, bd_mode)
filter_impl(wav_props.g1, x1, bd_mode)
x[:] = x0 + x1

Using these functions one can define standard DWT and IDWT kernels in the following
way, assuming that the wav_props has been computed:

dwt_kernel = lambda x, bd_mode: dwt_kernel_filters(x, bd_mode, wav_props)

idwt_kernel = lambda x, bd_mode: idwt_kernel_filters(x, bd_mode, wav_props)

Exercise 5.25: Restricted matrices for elementary lifting

Show that
5 The filter representation of wavelets 45
 
1 2λ 0 0 · · · 0 0 0
0 1 0 0 · · · 0 0 0 
 
0 λ 1 λ · · · 0 0 0 
(Aλ )r =  . . . . . . . . 
 
 .. .. .. .. .. .. .. .. 
 
0 0 0 0 · · · λ 1 λ
0 0 0 0 ··· 0 0 1
 
1 0 0 0 ··· 0 0 0
λ 1 λ 0 · · · 0 0 0
 
 0 0 1 0 · · · 0 0 0
(Bλ )r =  . . . . . . . .  .
 
 .. .. .. .. .. .. .. .. 
 
 0 0 0 0 · · · 0 1 0
0 0 0 0 · · · 0 2λ 1

Also, change the implementations of lifting_even_symm and lifting_odd_symm so

that these expressions are used (rather than Aλ , Bλ ) when the bd_mode parameter
equals "symm".

Solution.

Elementary lifting matrices can be viewed as wavelet transforms where the filters are
{λ, 1, λ} and {1}. Since these are symmetric, Theorem 11 in the book guarantees that
they preserve symmetric extensions.

Exercise 5.30: Implementing forward and reverse filter bank

transforms

It is not too difficult to implement the encoding and decoding steps as explained in the
MP3 standard. Write a function which implements the encoding steps, and one which
implements the decoding steps. In your code, assume for simplicity that the input and
output vectors to your methods all have lengths which are multiples of 32, and use the
functions mp3_ctable, mp3_dtable.
In the library the functions mp3_forward_fbt and mp3_reverse_fbt implement these
encoding- and decoding steps. Use these as a template for your code.

Solution.

The library versions of mp3_forward_fbt and mp3_reverse_fbt are as follows.

def mp3_forward_fbt(x):
N = len(x)
z = mat(zeros((N,1)))
C = mp3_ctable() # The analysis window;
x = x[(N-1)::(-1)]
x = concatenate([x, zeros(512 - 32)])
# The 32x64 matrix M
yvec = arange(0, 32, 1, float)
yvec = yvec.reshape((32, 1))
xvec = arange(-16, 48, 1, float)
46 5 The filter representation of wavelets

xvec = xvec.reshape((1, 64))

M = cos((2*yvec+1)*xvec*pi/64)
start = len(x) - 512;
for n in range(1, int(N/32) + 1):
X = x[start:(start + 512)]
Z = C*X # Pointwise multiplication
Y = zeros(64)
for j in range(8):
Y += Z[(64*j):(64*(j+1))]
Y = Y.reshape((64, 1))
z[((n-1)*32):(n*32), 0] = mat(M)*mat(Y)
start -= 32
z = array(z).flatten()
return z

def mp3_reverse_fbt(z):
N = len(z)
z = z. reshape((N,1))
z = mat(z)
Ns = int(N/32)
x = zeros(32*Ns)
D = mp3_dtable() # The reconstruction window.
V = mat(zeros((1024,1)))
# The 64x32 matrix N
yvec = arange(16, 80, 1, float)
yvec = yvec.reshape((64, 1))
xvec = arange(0, 32, 1, float)
xvec = xvec.reshape((1, 32))
Nmatr = mat(cos(yvec*(2*xvec + 1)*pi/64))
U = zeros(512)
for n in range(1, Ns+1):
V[64:1024, 0] = V[0:(1024-64), 0]
V[0:64, 0] = Nmatr*z[((n-1)*32):(n*32), 0]
for i in range(8):
U[(i*64):(i*64 + 32)] = \
array(V[(i*128):(i*128 + 32), 0]).flatten()
U[(i*64 + 32):((i + 1)*64)] = \
array(V[(i*128 + 96):((i+1)*128), 0]).flatten()
W = U*D
for i in range(16):
x[((n-1)*32):(n*32)] += W[32*i:(32*(i + 1))]
return x
Chapter 6
Constructing interesting wavelets

Exercise 6.3: Mixed continuous-discrete expressions

Let f (t) = k xk h(t − nTs ). Show that fˆ(ν) = x̂(νTs )ĥ(ν). This can be considered a
P
mixed continuous-discrete expression for that convolution in time to corresponds to
multiplication in frequency.

Solution.

We have that

Z ∞X Z ∞
ˆ 1 −iνt
X 1
f (ν) = √ xk h(t − nTs )e dt = xk √ h(t − nTs )e−iνt dt
2π −∞ k k
2π −∞
Z ∞
−iνnTs 1
X
−iνt
= xk e √ h(t)e dt = x̂(νTs )ĥ(ν).
k
2π −∞

Exercise 6.5: Finding a dual frame

Assume that U U T is non-singular, so that {un }n is a frame. From the previous exercise
it follows from this that σm > 0. From Appendix A in the book it also follows that
the rank of U is m, i.e. U has full column rank. Show that Ũ = (U T )† satisfies the
requirements of a dual frame, where (U T )† is the generalized inverse of U T , defined in
the appendix. The appendix then says that Ũ = (U T )† = (U U T )−1 U . It follows that
(U U T )−1 maps the frame vectors uk to the dual frame vectors ũk , and that the frame
operator U U T maps the dual frame vectors to the frame vectors.

Solution.

First we have that

Ũ U T = (U T )† U T = Im ,

47
48 6 Constructing interesting wavelets

where we used Appendix A in the book. Also, if σi are the singular values of U T , 1/σi
are the singular values of (U T )† . Its smallest singular value is thus 1/σ1 , and its largest
singular value is 1/σm . Again with U T = U0 Σ(V0 )H a singular value decomposition of
U T , we have that Ũ T = U0 Σ −1 (V0 )H . Following the deductions in the previous exercises
we get that
1 1
2 kf k2 ≤ kŨ T f k2 =≤ 2 kf k2 .
σ1 σm
Since 1/σ1 = 1/B and 1/σm = 1/A the result follows.

Exercise 6.8: Computing filters

Compute the filters H0 , G0 in Theorem 8 in the book when N = N1 = N2 = 4, and

Q1 = Q(4) , Q2 = 1. Compute also filters H1 , G1 so that we have perfect reconstruction
(note that these are not unique).

Solution.

We have that

 N1 /2  
1 1
λH0 (ω) = (1 + cos ω) Q1 (1 − cos ω)
2 2
 2  
1 1
= (1 + cos ω) Q(4) (1 − cos ω)
2 2
 N2 /2    2
1 1 1
λG0 (ω) = (1 + cos ω) Q2 (1 − cos ω) = (1 + cos ω)
2 2 2
 2
1 1 1 1 2iω
= 1 + eiω + e−iω = (e + 4eiω + 6 + 4e−iω + e−2iω ).
4 2 2 16
1
Therefore G0 = 16 {1, 4, 6, 4, 1}. We do not recommend to compute H0 by hand. With
the package sympy in Python you can do as follows to compute H0 .

from sympy import *

x = Symbol(’x’)
z = expand( ((1+x/2+1/(2*x))/2)**2* \
(2+8*((1-x/2-1/(2*x))/2)+20*((1-x/2-1/(2*x))/2)**2\
+40*((1-x/2-1/(2*x))/2)**3) )

2
Here we have substituted x for eiω , 1/x for e−iω . The first part represents 12 (1 + cos ω) ,
the second part represents Q(4) (u) = 2 + 8u + 20u2 + 40u3 with u = 12 (1 − cos ω) =
1 1 iω 1 −iω


2 1 − 2e − 2e . This gives
1
H0 = {−5, 20, −1, −96, 70, 280, 70, −96, −1, 20, −5}.
128
Using Exercise 5.10 in the book with α = 1, d = 0, we get
6 Constructing interesting wavelets 49

1
H1 = {1, −4, 6, −4, 1}
16
1
G1 = {5, 20, 1, −96, −70, 280, −70, −96, 1, 20, 5}
128

Exercise 6.9: Plotting frequency responses of Spline wavelets

The library represents a Spline wavelet with x and y vanishing moments (i.e. N1 and
N2 ) with the name "splinex.y".
a) Listen to the low-resolution approximations and detail components in sound for the
"spline4.4" wavelet.

Solution.

The following code can be used for listening to the low-resolution approximations for a
given value of m.

m = 1
x, fs = forw_comp_rev_dwt1(m, ’spline4.4’)
play(x, fs)

x, fs = forw_comp_rev_dwt1(m, ’spline4.4’, False)

play(x, fs)

b) Plot all scaling functions and mother wavelets (using the cascade algorithm), and
frequency responses for the "spline4.4" wavelet.

Solution.

The code to plot the functions can look as follows.

m=10;
cascade_alg(m, -4, 4, ’spline4.4’, True, False)
cascade_alg(m, -4, 4, ’spline4.4’, False, False)

The frequency response can be plotted as follows

freqresp_alg(’spline4.4’, lowpass, dual)

Chapter 7
The polyphase representation of filter bank
transforms

Exercise 7.1: The frequency responses of the polyphase

components

Let H and G be 2-channel forward and reverse filter bank transforms, with filters H0 ,
H1 , G0 , G1 , and polyphase components H (i,j) , G(i,j) .
a) Show that

λH0 (ω) = λH (0,0) (2ω) + eiω λH (0,1) (2ω)

λH1 (ω) = λH (1,1) (2ω) + e−iω λH (1,0) (2ω)
λG0 (ω) = λG(0,0) (2ω) + e−iω λG(1,0) (2ω)
λG1 (ω) = λG(1,1) (2ω) + eiω λG(0,1) (2ω).

Solution.

G(0,0) , G(1,1) are the even-indexed filter coefficients of G0 , G1 , respectively, so that

λG(0,0) (2ω), λG(1,1) (2ω) represents the half of λG0 (ω), λG1 (ω), respectively, from the even
filter coefficients. G(1,0) are the odd-indexed filter coefficients of G0 . Since coefficient 0
in G(1,0) equals coefficient 1 in G0 , it is clear that e−iω λG(1,0) (2ω) represents the half of
λG0 (ω) from the odd filter coefficients. This proves the first formula. The second formula
follows from the same kind of reasoning.
If we transpose H (also in polyphase form), we get an MRA-matrix where the columns
are given by the filters (H0 )T , (H1 )T . Inserting these in the formulas we just proved we
get that

λ(H0 )T (ω) = λ(H (0,0) )T (2ω) + e−iω λ(H (0,1) )T (2ω)

λ(H1 )T (ω) = λ(H (1,1) )T (2ω) + eiω λ(H (1,0) )T (2ω).

If we conjugate these expressions we get

51
52 7 The polyphase representation of filter bank transforms

λH0 (ω) = λH (0,0) (2ω) + eiω λH (0,1) (2ω)

λH1 (ω) = λH (1,1) (2ω) + e−iω λH (1,0) (2ω),

and the proof is done.

b) In the proof of the last part of Exercise 5.10 in the book, we deferred the last part,
namely that

λH1 (ω) = α−1 e−2idω λG0 (ω + π)

λG1 (ω) = αe2idω λH0 (ω + π).

follow from

G(0,0) G(0,1) αE−d H (1,1) −αE−d H (0,1)

   
= .
G(1,0) G(1,1) −αE−d H (1,0) αE−d H (0,0)

Prove this based on the result from a).

Solution.

The first column in the matrix on the left hand side gives the filter G0 . On the right hand
side, a) states that the even-indexed columns are taken from the filter with frequency
response

λαE−d H (1,1) (2ω) + e−iω λ−αE−d H (1,0) (2ω)

= αλE−d (2ω) λH (1,1) (2ω) − e−iω λH (1,0) (2ω)


 
= αe2idω λH (1,1) (2(ω + π)) + e−i(ω+π) λH (1,0) (2(ω + π)) = αe2idω λH1 (ω + π).

This shows that λG0 (ω) = αe2idω λH1 (ω+π). We obtain the first equation easily from this.
Now, the second column in the matrix on the left hand side gives the filter coefficients
of G1 . On the right hand side, a) states that the odd-indexed columns are taken from
the filter with frequency response

λαE−d H (0,0) (2ω) + eiω λ−αE−d H (0,1) (2ω)

= αλE−d (2ω) λH (0,0) (2ω) − eiω λH (0,1) (2ω)


 
= αe2idω λH (0,0) (2(ω + π)) + ei(ω+π) λH (0,1) (2(ω + π)) = αe2idω λH0 (ω + π).

This shows that λG1 (ω) = αe2idω λH0 (ω + π), which is the second equation.
7 The polyphase representation of filter bank transforms 53

Exercise 7.3: Polyphase view of alias cancellation

How we can spot alias cancellation in terms of the polyphase representation will be
useful to us in the next section when we explain why the filter bank transform of
the MP3 standard provides such cancellation. We will there end up with a polyphase
representation on the form given in a) of this exercise.
a) Assume that the polyphase representation of S is a k × k-block diagonal matrix
where the diagonal elements are M × M -filters which are all equal. Show that S is
a (kM ) × (kM )-filter. Thus, if GH has this form for forward and reverse filter bank
transforms H and G, we have alias cancellation.

Solution.

Since S has a k × k-block diagonal polyphase representation with equal diagonal blocks,
the only nonzero elements in the filter representation are at indices (i + M r1 , i + M r2 )
with 0 ≤ i < M , 0 ≤ r1 , r2 < k. Increasing i with 1 just switches to the next block
on the diagonal, and we get the same value since the blocks are assumed equal. Also,
increasing both r1 and r2 with one clearly gives a new element on the same diagonal in
the same block, which has the same value by the assumption that each block entry is
a filter. It follows that S is constant on diagonals 0, ±M , ±2M , and so on, and zero
elsewhere. It follows that S is a (kM ) × (kM )-filter.
b) If S is a general (kM ) × (kM )-filter, give a general description of its polyphase
representation.

Solution.

Each block in the polyphase representation is a k × k-filter. Also, the filters S (i,j) and
S (i+k,j+k) must be equal for any i, j, k.

Exercise 7.8: Lifting-based implementations of the Spline 5/3

and CDF 9/7 wavelets

Let us use the different lifting factorizations obtained in this chapter to implement the
corresponding wavelet kernels. Your functions should call the functions from Exercise 4.24
and 4.29 in the book. You will need equations (7.8)-(7.11) here, in order to complete
the kernels.
a) Write functions

dwt_kernel_53(x, bd_mode)
idwt_kernel_53(x, bd_mode)

which implement the DWT and IDWT kernels for the Spline 5/3 wavelet. Use the lifting
factorization obtained in Example 7.2.3 in the book.
54 7 The polyphase representation of filter bank transforms

Solution.

The code can look like this:

def dwt_kernel_53(x, bd_mode):

x[0::2] *= 2
x[1::2] *= 0.5
lifting_odd_symm(-0.125, x, bd_mode)
lifting_even_symm(1, x, bd_mode)

def idwt_kernel_53(x, bd_mode):

lifting_even_symm(-1, x, bd_mode)
lifting_odd_symm(0.125, x, bd_mode)
x[0::2] *= 0.5
x[1::2] *= 2

b) Write functions

dwt_kernel_97(x, bd_mode)
idwt_kernel_97(x, bd_mode)

which implement the DWT and IDWT kernels for the CDF 9/7 wavelet. Use the lifting
factorization obtained in Example 7.2.4 in the book.

Solution.

The code can look like this:

def dwt_kernel_97(x, bd_mode):

lambda1=-0.586134342059950
lambda2=-0.668067171029734
lambda3=0.070018009414994
lambda4=1.200171016244178
alpha=-1.149604398860250
beta=-0.869864451624777
x[0::2] *= alpha
x[1::2] *= beta
lifting_odd_symm(-lambda4, x, bd_mode)
lifting_even_symm(-lambda3, x, bd_mode)
lifting_odd_symm(-lambda2, x, bd_mode)
lifting_even_symm(-lambda1, x, bd_mode)

def idwt_kernel_97(x, bd_mode):

lambda1=-0.586134342059950
lambda2=-0.668067171029734
lambda3=0.070018009414994
lambda4=1.200171016244178
alpha=-1.149604398860250
beta=-0.869864451624777
lifting_even_symm(lambda1, x, bd_mode)
lifting_odd_symm(lambda2, x, bd_mode)
lifting_even_symm(lambda3, x, bd_mode)
lifting_odd_symm(lambda4, x, bd_mode)
x[0::2] /= alpha
x[1::2] /= beta
7 The polyphase representation of filter bank transforms 55

c) In Chapter 4 in the book, we listened to the low-resolution approximations and detail

components in sound for three different wavelets. Repeat these experiments with the
Spline 5/3 and the CDF 9/7 wavelet, using the new kernels you implemented in a) and
b).

Solution.

The following code can be used for listening to the low-resolution approximations for a
given value of m.
m = 1
x, fs = forw_comp_rev_dwt1(m, ’cdf53’)
play(x, fs)

m = 1
x, fs = forw_comp_rev_dwt1(m, ’cdf97’)
play(x, fs)

d) Plot all scaling functions and mother wavelets for the Spline 5/3 and the CDF 9/7
wavelets, using the cascade algorithm and the kernels you have implemented.

Solution.

The code can look as follows.

m=10
cascade_alg(m, -4, 4, ’cdf53’, True, False)
cascade_alg(m, -4, 4, ’cdf53’, False, False)
cascade_alg(m, -4, 4, ’cdf97’, True, False)
cascade_alg(m, -4, 4, ’cdf97’, False, False)

In the plot for the CDF 9/7 wavelet, it is seen that the functions and their dual
counterparts are close to being equal. This reflects the fact that this wavelet is close to
being orthogonal.

Exercise 7.11: Wavelet based on piecewise quadratic scaling

function

In Exercise 6.8 you should have found the filters

1
H0 = {−5, 20, −1, −96, 70, 280, 70, −96, −1, 20, −5}
128
1
H1 = {1, −4, 6, −4, 1}
16
1
G0 = {1, 4, 6, 4, 1}
16
1
G1 = {5, 20, 1, −96, −70, 280, −70, −96, 1, 20, 5}.
128
56 7 The polyphase representation of filter bank transforms

Show that
1
I − 41 {1, 1}
    1 
I − 128 {5, −29, −29, 5} I 0 0
G= 4 ,
0 I −{1, 1} I 0 I 04
and derive a lifting factorization of G from this.

Solution.

The polyphase representation of the IDWT is

1 1

16 {1, 6, 1} 128 {5, 1, −70, −70, 1, 5} .
1 1
16 {4, 4} 128 {20, −96, 280, −96, 20}

We can first apply an even lifting step:

 1 1
I − 14 {1, 1}
 
16 {1, 6, 1} 128 {5, 1, −70, −70, 1, 5}
1 1
0 I 16 {4, 4} 128 {20, −96, 280, −96, 20}
 1 1

16 {4} 128 {20, −116, −116, 20}
= 1 1 .
16 {4, 4} 128 {20, −96, 280, −96, 20}

We can now apply an odd lifting step

  1 1
 1 1

I 0 16 {4} 128 {20, −116, −116, 20} = 4 128 {20, −116, −116, 20}
1 1
−{1, 1} I 16 {4, 4} 128 {20, −96, 280, −96, 20} 0 4

Since
1
  1 1
 1 
I − 512 {20, −116, −116, 20} 4 128 {20, −116, −116, 20} 0
= 4 ,
0 I 0 4 04
it follows that
1
I − 41 {1, 1}
    1 
I − 128 {5, −29, −29, 5} I 0 0
G= 4 .
0 I −{1, 1} I 0 I 04

Exercise 7.13: Run forward and reverse transform

Run the forward and then the reverse transform from Exercise 5.30 on the vector
(1, 2, 3, . . . , 8192). Verify that there seems to be a delay on 481 elements, as promised in
Section 7.3.4 in the book. Do you get the exact same result back?

Solution.

The following code can be used:

x = arange(1,8193)
mp3_reverse_fbt(mp3_forward_fbt(x))
plt.plot(x)
7 The polyphase representation of filter bank transforms 57

There are some small errors from the original vector in the resulting vector, when one
compensates for the delay of 481 elements.

Exercise 7.14: Verify statement of filters

Use your computer to verify the following symmetries for the prototype filters:
(
−C512−i i 6= 64, 128, . . . , 448
Ci =
C512−i i = 64, 128, . . . , 448.
Explain also that this implies that hi = h512−i for i = 1, . . . , 511. Recall that the
modified version had the symmetry hi = h511−i . When the filters V (m) are defined as in
this section, explain why (7.26) in the book should be replaced by

V (64−k) = −E14 (V (k) )T

in the MP3 standard

Solution.

The following code can be used

ctable = mp3_ctable()
ctable = ctable[1::]
ctable[63:255:64] = -ctable[63:255:64]
res = ctable + ctable[-1::-1]
res[255]=0
print(abs(res).max())

Note that the middle element of the table is set to 0 in this verification. The reason is
that, for i = 256, there is nothing to check. So, since the verification works by producing
zero when there is symmetry, the element corresponding to i = 256 is set to zero. Also,
every 64’th element in one half of the table changed sign, in order to establish symmetry
for the corresponding entries, rather than anti-symmetry as for the other entries.
Chapter 8
Digital images

Exercise 8.8: Adjusting contrast with another function

a) Show that the function hn : R → R given by

hn (x) = xn ,
maps the interval [0, 1] → [0, 1] for any n, and that h0n (1) → ∞ as n → ∞.

Solution.

We have that h0n (x) = nxn−1 , so that h0n (1) = n → ∞.

b) The image secret.jpg, shown in Figure 8.1, contains some information that is
nearly invisible to the naked eye on most monitors. Use the function hn , to reveal the
secret message.

Fig. 8.1 Secret message.

59
60 8 Digital images

Hint.

Convert to a grayscale image before you adjust the contrast with hn .

Solution.

The following code can be used

n = 100
X = double(imread(’images/secret.jpg’, ’jpg’))
X = (X[:,:,0] + X[:,:,1]+ X[:,:,2])/3.
map_to_01(X)
X **= n
X *= 255
X = uint8(X)
imshow(X)

The resulting image is shown in Figure 8.2.

Fig. 8.2 Secret message revealed!

Exercise 8.13: Generating images

Write code which calls the function tensor2_impl with appropriate filters to produce
the following images:
a) The right image in Figure 8.10 in the book.

Solution.

The following code can be used

8 Digital images 61

X3 = excerpt.copy()
S2 = convolve((1/8.)*array([1, 3, 3, 1]),(1/8.)*array([1, 3, 3, 1]))
def longmolecule(x, bd_mode):
filter_impl(S2, x, bd_mode)
tensor2_impl(X3, longmolecule, longmolecule, ’symm’)
imshow(uint8(X3))

b) The right image in Figure 8.12 in the book.

Solution.

The following code can be used

X3 = excerpt.copy()
tensor2_impl(X3, donothing, diffxmolecule, ’per’)
map_to_01(X3)
X3 *= 255
contrast_adjust0(X3,50)
imshow(uint8(X3))

c) The images in Figure 8.14 in the book.

Solution.

The following code can be used

excerpt=create_excerpt()
X1 = excerpt.copy()
tensor2_impl(X1, donothing, diffxmolecule, ’per’)
X2 = excerpt.copy()
tensor2_impl(X2, diffymolecule, donothing, ’per’)
X1 = X1**2 + X2**2
imshow(uint8(X1))

X2[:] = X1
map_to_01(X2)
X2 *= 255
imshow(uint8(X2))

X3[:] = X2
contrast_adjust0(X3, 50)
imshow(uint8(X3))

d) The images in Figure 8.13 in the book.

Solution.

The following code can be used

62 8 Digital images

excerpt=create_excerpt()
X1 = excerpt.copy()
tensor2_impl(X1, donothing, diffxmolecule, ’per’)
map_to_01(X1)
X1 *= 255
contrast_adjust0(X1, 50)
imshow(uint8(X1))

def diffymolecule(x, bd_mode):

filter_impl(array([-1, 0, 1])/2., x, bd_mode)
X2 = excerpt.copy()
tensor2_impl(X2, diffymolecule, donothing, ’per’)
map_to_01(X2)
X2 *= 255
contrast_adjust0(X2, 50)
imshow(uint8(X2))

e) The images in Figure 8.15 in the book.

Solution.

The following code can be used

excerpt=create_excerpt()
X1 = excerpt.copy()
tensor2_impl(X1, donothing, diffxmolecule, ’per’)
tensor2_impl(X1, donothing, diffxmolecule, ’per’)
map_to_01(X1)
X1 *= 255
contrast_adjust0(X1, 100)
imshow(uint8(X1))

X2 = excerpt.copy()
tensor2_impl(X2, diffymolecule, diffxmolecule, ’per’)
map_to_01(X2)
X2 *= 255
contrast_adjust0(X2, 100)
imshow(uint8(X2))

X3 = excerpt.copy()
tensor2_impl(X3, diffymolecule, donothing, ’per’)
tensor2_impl(X3, diffymolecule, donothing, ’per’)
map_to_01(X3)
X3 *= 255
contrast_adjust0(X3, 100)
imshow(uint8(X3))

Exercise 8.18: Computational molecules

Let the filter S be defined by S = {1, 2, 1}.

a) Write down the computational molecule of S ⊗ S.
8 Digital images 63

Solution.

The computational molecule of S ⊗ S is

   
1
121
(1, 2, 1) ⊗ (1, 2, 1) = (1, 2, 1) ⊗ (1, 2, 1) = 2 1 2 1 = 2 4 2 .
1 121

b) Let us define x = (1, 2, 3), y = (3, 2, 1), z = (2, 2, 2), and w = (1, 4, 2). Compute the
matrix A = x ⊗ y + z ⊗ w.

Solution.

We get that

   
1
2

A = 2 3 2 1 + 2 1 4 2
3 2
     
321 284 5 10 5
= 6 4 2 + 2 8 4 =  8 12 6 .
963 284 11 14 7

c) Compute (S ⊗ S)A by applying the filter S to every row and column in the matrix
the way we have learned. If the matrix A was more generally an image, what can you
say about how the new image will look?

Solution.

We need to compute (S ⊗ S)A = SAS T , which corresponds to first applying S to every

column in the image, and then applying S to every row in the resulting image. If we
apply S to every column in the image we first get the matrix
 
29 46 23
SA = 32 48 24 .
35 50 25
If we apply the filter to the rows here we get
 
127 144 121
SAS T = 136 152 128 .
145 160 135
Since the filter which is applied is a low-pass filter, the new image should look a bit
more smooth than the original image.
64 8 Digital images

Exercise 8.20: Eigenvectors of tensor products

Let vA be an eigenvector of A with eigenvalue λA , and vB an eigenvector of B with

eigenvalue λB . Show that vA ⊗ vB is an eigenvector of A ⊗ B with eigenvalue λA λB .

Solution.

We have that

(A ⊗ B)(vA ⊗ vB ) = (AvA ) ⊗ (BvB ) = (λA vA ) ⊗ (λB vB ) = λA λB (vA ⊗ vB ).

Exercise 8.24: Implementing DFT and DCT on blocks

In this section we have used functions which apply the DCT and the DFT either to
subblocks of size 8 × 8, or to the full image.
a) Implement functions dft_impl8, idft_impl8, dct_impl8, and idct_impl8 which
apply the DFT, IDFT, DCT, and IDCT, to consecutive blocks of length 8.

Solution.

The following code can be used.

def dft_impl8(x, bd_mode):

N = shape(x)[0]
for n in range(0, N, 8):
x[n:(n+8), :] = fft.fft(x[n:(n+8), :], axis=0)

def idft_impl8(x, bd_mode):

N = shape(x)[0]
for n in range(0, N, 8):
x[n:(n+8), :] = fft.ifft(x[n:(n+8), :], axis=0)

def dct_impl8(x, bd_mode):

N = shape(x)[0]
for n in range(0, N, 8):
x[n:(n+8), :] = dct(x[n:(n+8), :], norm=’ortho’, axis=0)

def idct_impl8(x, bd_mode):

N = shape(x)[0]
for n in range(0, N, 8):
x[n:(n+8), :] = idct(x[n:(n+8), :], norm=’ortho’, axis=0)

b) Implement the two-dimensional FFT, IFFT, DCT, and IDCT on images, with the
help of their one-dimensional counterparts, as well as the function tensor2_impl.
8 Digital images 65

Solution.

The following code can be used.

tensor2_impl(X, dft_impl_full, dft_impl_full, ’symm’)

tensor2_impl(X, idft_impl_full, idft_impl_full, ’symm’)

tensor2_impl(X, dct_impl_full, dct_impl_full, ’symm’)

tensor2_impl(X, idct_impl_full, idct_impl_full, ’symm’)

c) The function forw_comp_rev_2d in the library applies given transforms to the rows
and columns of an input image, sets the coefficients below a certain threshold to zero,
and transforms back. Run forw_comp_rev_2d for different threshold parameters on the
sample image, and with the functions dct_impl8, idct_impl8, as well as DCT and
IDCT applied to the entire input, as parameters. Check that this reproduces the DCT
test images of this section, and that the correct numbers of values which have been
discarded (i.e. which are below the threshold) are printed on screen.

Solution.

Once

X1 = X.copy()
forw_comp_rev_2d(X1, dct_impl8, idct_impl8, 30)
imshow(uint8(X1))

X2 = X.copy()
forw_comp_rev_2d(X2, dct_impl8, idct_impl8, 50)
imshow(uint8(X2))

X3 = X.copy()
forw_comp_rev_2d(X3, dct_impl8, idct_impl8, 100)
imshow(uint8(X3))

X1 = X.copy()
forw_comp_rev_2d(X1, dct_impl_full, idct_impl_full, 30)
imshow(uint8(X1))

X2 = X.copy()
forw_comp_rev_2d(X2, dct_impl_full, idct_impl_full, 50)
imshow(uint8(X2))

X3 = X.copy()
forw_comp_rev_2d(X3, dct_impl_full, idct_impl_full, 100)
imshow(uint8(X3))
Chapter 9
Using tensor products to apply wavelets to
images

Exercise 9.4: Higher order tensor products

Tensor products can easily be generalized to higher dimensions. For 3D, and for x ∈ RM ,
y ∈ RN , z ∈ RK , we define x ⊗ y ⊗ z as the "3D matrix" with entries

(x ⊗ y ⊗ z)i,j,k = xi yj zk ,
and if S1 , S2 and S3 are linear transformations on RM , RN , and RK , respectively, their
tensor product is the unique linear transformation S1 ⊗ S2 ⊗ S3 defined on 3D matrices
by

(S1 ⊗ S2 ⊗ S3 )(x ⊗ y ⊗ z) = (S1 x) ⊗ (S2 y) ⊗ (S3 z),

for all x, y, and z.
a) Explain that S1 ⊗ S2 ⊗ S3 can be implemented by
• replacing x0:(M −1),j,k by S1 (x0:(M −1),j,k ) for all j, k,
• replacing xi,0:(N −1),k by S2 (xi,0:(N −1),k ) for all i, k,
• replacing xi,j,0:(K−1) by S3 (xi,j,0:(K−1) ) for all j, k,
and that the order of these three operations does not matter. In other words, higher
order tensor products can also be split into a series of one-dimensional operations.

Solution.

Similarly to the 2D case we can write

S1 ⊗ S2 ⊗ S3 = (S1 ⊗ I ⊗ I)(I ⊗ S2 ⊗ I)(I ⊗ I ⊗ S3 ),

where the order of the three right hand factors does not matter. Also, by adding terms,
we can write any 3D matrix as
X
(x0:(M −1),j,k ) ⊗ ej ⊗ ek ,
i,j

and applying (S1 ⊗ I ⊗ I) to this we obtain

67
68 9 Using tensor products to apply wavelets to images
X
(S1 x0:(M −1),j,k ) ⊗ ej ⊗ ek .
i,j

This proves the claim that we can apply S1 to the different “(:, j, k)-segments”. The
cases for I ⊗ S2 ⊗ I and I ⊗ I ⊗ S3 follow similarly.
b) In particular we can extend the tensor product to wavelets in 3D. How many detail
components will there be at each level in a 3D DWT, following the setup we did for the
2D DWT? How would you label the different components?

Solution.

Duplicating the theory for changes of coordinates in 2D, we see that the target basis for
a 1-level 3D DWT consists of 23 = 8 different types of basis functions:

φ0,n1 ⊗ φ0,n2 ⊗ φ0,n3 ψ0,n1 ⊗ φ0,n2 ⊗ φ0,n3

φ0,n1 ⊗ ψ0,n2 ⊗ φ0,n3 φ0,n1 ⊗ φ0,n2 ⊗ ψ0,n3
ψ0,n1 ⊗ ψ0,n2 ⊗ φ0,n3 ψ0,n1 ⊗ φ0,n2 ⊗ ψ0,n3
φ0,n1 ⊗ ψ0,n2 ⊗ ψ0,n3 ψ0,n1 ⊗ ψ0,n2 ⊗ ψ0,n3 .

Using as before L for φ, H for ψ, these would be labeled LLL, HLL, LHL, LLH, HHL,
HLH, LHH, and HHH, respectively.
dwt3_impl_internal and idwt3_impl_internal are the functions in the library
which compute the 3D DWT and the 3D IDWT. You are in particular encouraged to
review the function tensor3_impl, which is used in the functions, and is a straightforward
generalization of tensor2_impl. Also in the 3D case there is a simpler alternative
definition for the DWT, following the setup from the previous exercise.

Exercise 9.10: Applying the Haar wavelet to another chess

pattern image

The following code produces a chess pattern type image almost identical to that from
Example 9.5 in the book.

img=zeros((128,128))
for x in range(128):
for y in range(128):
img[x,y]=255*( (mod(x,64)>=32) == (mod(y,64)>=32) )
imshow(uint8(img))

Let us now apply a 2D DWT to this image as well with the Haar wavelet:

img2 = img.copy()
dwt_impl(img2, ’Haar’, bd_mode=’per’, dims=2)
map_to_01(img2)
img2 *= 255
imshow(uint8(img2))

The resulting images are shown in Figure 9.1

9 Using tensor products to apply wavelets to images 69

Fig. 9.1 A simple image before and after one level of the 2D DWT. The Haar wavelet was used.

There seem to be no detail components here, which is very different from what you
saw in Example 9.5 in the book, even though the images are very similar. Attempt to
explain what causes this to happen.

Hint.

Compare with Exercise 4.14 in the book.

Solution.

In Example 9.5 in the book, the borders in the chess pattern was chosen so that they
occur at odd numbers. This means that the image can not be represented exactly in
Vm−1 ⊗ Vm−1 , so that there is detail present in the image at all the borders in the
chess pattern. In the new image, the borders in the chess pattern was chosen so that
they occur at even numbers. This means that the image can be represented exactly in
Vm−1 ⊗ Vm−1 , so that there is no detail components present in the image.
References

[1] A. Cohen, I. Daubechies, and J-C. Feauveau. Biorthogonal bases of compactly

supported wavelets. Communications on Pure and Appl. Math., 45(5):485–560, June
1992.

71