Linear Transforms
Linear Transforms
import otter
grader = otter.Notebook("Linear transforms.ipynb")
The first week notebook (introduction to Python, Numpy and Matplotlib) can be
used as a help.
Important
You need to submit individually your answers on moodle before the next exercise
session. For the theoretical questions you can either fill the notebook or write it
on a separate sheet (if you are not comfortable with Markdown/TeX) and upload
a scanned version.
Objective
The end goal is to understand purely algebraic, matrix based, view of a few
linear transforms. You will use those linear transform to perform some basic
time-frequency analysis of signals.
Part I - Fourier
N
1. Prove that any set of orthogonal vectors vi ∈ C , i = 1, … , M ≤ N such
H
that v
i
vj = Cδi,j is linearly independent (where C is some constant).
H
v
i
vj = Cδi,j = C when i = j; = 0 when i ≠ j.
H N H N H H
v
i
⋅ (∑
j=1
αj v j ) = v
i
⋅ 0 ⇒ 0 = ∑
j=1
αj v j v
i
. Knowing that v
i
vj = Cδi,j ,
we can deduce that all terms of the previous sum are equal to 0 except when
i = j. Looking at this specific case, we have:
N H H H
0 = ∑
j=1
αj v j v
i
= αi v i v
i
. We know that vi v
i
= C (check first line of the
demo)
H
⇒ αi v i v
i
= 0 is only valid when αi = 0 .
n
N −1 j2πr
2. Compute ar = ∑
n=0
e N
, where r is an integer (discuss the result
depending on the value of r).
This sum is a geometric series. The general formula for a geometric series S is :
N
1−r
SN = a0 ( ),
1−r
with a0 being the first term, N being the number of terms, and r being the
n
th j2πr
common ratio. In this case, the n term is e N
⇒ the initial term is
0 1
j2πr j2πr
a0 = e N
= 1; the common ratio is r = e N
.
1
j2πr
N N
1−(e )
The sum becomes ar =
1
.
j2πr
1−e N
1
j2πr
Due to the periodicity of the exponential function (Euler's formula), e N
= 1
1
when the term r is an integer. We can then discuss of 2 potential results for
N
this expression.
n
N −1 j2πr
If r is equal to 0, we use the form ar = ∑
n=0
e N
to see that we are
N −1 j2π0 N −1
summing 1 N times, meaning ar = ∑
n=0
e = ∑
n=0
1 ⇒ ar = N .
1
j2πr
N
1−(e N ) j2πr
1−e
If r is not equal to 0, we use the form ar =
1
=
1
. Knowing
j2πr j2πr
1−e N 1−e N
j2πr
that r is an integer, we get e = 1, which cancels out the numerator. We
conclude that ar = 0 when r is not a multiple of N.
If r is a multiple of N, we can see that the previous point is not valid, as both
the numerator and the denominator go to 0 at the same time. Going back to
the case r = 0; we can see that this is equivalent to r being a multiple of N,
the sum thus results in N.
kn
N −j2π
3. Let vk ∈ C be such that vk [n] = e N , for k, n = 0, … , N − 1.
Prove that these vectors are mutually orthogonal, hence linearly
independent.
Compute the norm of vk .
To prove that these vectors are mutually orthogonal, we have to show that
vl = Cδk,l , with constant.
H
v C
k
n n n
H N −1 H N −1 j2πk −j2πl N −1 j2π (k−l)
v vl = ∑ vk [n] ⋅ vl [n] = ∑ e N ⋅ e N = ∑ e N
k n=0 n=0 n=0
n
N −1 j2π r
= ∑
n=0
e N , with r = k − l. We have the same situation as the previous
question.
For r = 0, we have v
H
k
vl = N , thus when k = l.
H
We can conclude that v
k
vl = N δk,l ; the vectors are mutually orthogonal and
thus linearly independant.
2
As for the norm of vk , we have ∥vk ∥ = v
H
k
vk = N ⇒ ∥vk ∥ = N .
return F
In [4]: grader.check("q4")
Out[4]: q4 passed!
q4 - 6 message: Good, you properly validated size before computing the result
The first one x1 (t) is made of four piecewise sinusoids, of different frequencies:
The second signal x2 (t) is the sum of the same sinusoids over the complete time
interval, with a scaling term s.t. the amplitude of both signals is identical.
5.
We can observe frequency spikes around the various component frequencies for
each signal. The x1 Fourier plot seems "fuller" compared to the x2 plot, the
frequency peaks are smoother and the signal has small and large comonents.
The x2 plot presents sharp precise peaks, with no sub-components or small
frequency contributions.
In [7]: # plot x1
plt.plot(x1)
In [8]: # plot x2
plt.plot(x2)
In order to have a better compromise between time and frequency, the input
signal will be split in smaller non-overlapping blocks of length p, and we will
perform the DFT of each block.
total_blocks = N//p
block_dft_matrix = np.zeros((N, N), dtype=complex)
F_p = get_fourier_matrix(p) #Fourier matrix for a single block
return block_dft_matrix
In [14]: grader.check("q6")
Out[14]:
q6 passed!
q6 - 2 message: Good, you properly validated size before computing the result
We will now use this block Fourier transform to how the frequencies of the signal
evolve through time.
7.
A larger block size p provides better frequency resolution, the Fourier transform
of a larger block can differentiate between closer frequencies. A smaller p
parameter allows for more time resolution, it gives a more precise localization of
changes in the signal's frequency content over time.
In [15]: # Compute the block DFT matrix Wb
N = 2048
p= 64
Wb = get_block_dft_matrix(N, p)
N
Let us consider a vector x ∈ R , with N being even. The single-level Haar
N
1
The coefficients of a are defined as follows:
1 1 N 1
an = (x2n−1 + x2n ), n = 1, . . . , . a is referred to as the average
√2 2
coefficients vector.
The coefficients of d
1
are defined as follows:
1 1 N 1
dn = (x2n−1 − x2n ), n = 1, . . . , . d is referred to as the detail
√2 2
coefficients vector.
9. Let us represent the single-level Haar transform by a matrix H1 s.t.
1
a
H1 x = ( )
1
d
th 1
The n row of the top half of the Haar matrix consists of at the 2n − 1 and
√2
1
The n
th
row of the bottom half of the Haar matrix consists of at the 2n − 1
√2
−1
and at the 2n position, and zeros elsewhere.
√2
For any 2 rows in the top half of H1 , the non-zero elements on each such row are
not shared between rows; each row averages a distinct pair of elements from x,
which leads to a zero dot product. The same reasoning applies for 2 bottom half
rows. As for one row in the top half and one row in the bottom half, the non-zero
elements are at the same position, but the product of these non-zero elements
1 1 1 −1
cancel out (coefficient , , and coefficient , ). The dot product of any
√2 √2 √2 √2
The sum of the squares of the coefficient in any rows always gives out
1 1
+ = 1, which demonstrates that each row is normalized. The H1 matrix is
2 2
thus orthonormal.
10. Write a function that returns the single-level Haar transform matrix H1 for a
given N. Raise a ValueError if N is invalid.
H1 = np.zeros((N, N))
return H1
In [19]: grader.check("q10")
Out[19]:
q10 passed!
q10 - 1 message: Good, you properly validated size before computing the result
For instance constructing 2-level Haar transform over N points start with the
N N
previously defined H1,N matrix of size N × N and the corresponding ×
2 2
version denoted by H
1,
N .
2
a
H
1,N
( ),
d
H
1,N
a d
where H
1,N
and H
1,N
are respectively the average and detail coefficient
N
matrices, both of size × N.
2
Following these notations, the 2-level Haar transform matrix H2,N can be written
as:
a
H N H
1, 1,N
2
( ),
d
H
1,N
11. Implement a function that returns the Hp,N matrix of size N × N that
performs a p-level haar transform. Raise a ValueError if the size and the
level are incompatible, or if the level is smaller than 1.
block_diagonal = np.eye(N)
block_diagonal[:size, :size] = H1
H_matr = np.dot(block_diagonal, H_matr)
return H_matr
In [21]: grader.check("q11")
Out[21]:
q11 passed!
q11 - 1 message: Good, you properly validated size/level before computing the
result
The noise is usually located in the higher frequencies. However, the signal we
created is a bit special as it has two discontinuities which also generate high
frequencies components (remember the Fourier transform of a rectangle function
is a sinc).
return denoised.real
return denoised.real
In [29]: # Perform denoising with the full Fourier transform and display the result.
# Make sure you choose a good threshold
N = len(angle1)+len(angle2)+len(angle3)
In [32]: grader.check("q14")
Out[32]:
q14 passed!
q14 - 1 message: Good, not denoising the noisy signal does not modify it.
15. Compare the three denoising methods (Fourier, block Fourier and Haar).
Which one performs better (in terms of noise removal but also in terms of
discontinuity preservation) ? Was that expected (justify) ?
The Full Fourier Transform method is good at overall noise removal, but
cannot properly preserve the hard edges and discontinuities of the signal.
The Block Fourier Transform does an initially better job when it comes to
overall noise removal, but after a few discontinuities the signal quality goes
down. Overall denoising quality is not uniform throughout the signal.
The Haar transform is the best at preserving the discontinuties as they are
almost perfectly represented. However the overall denoising quality is not as
good as the Fourier transforms, as this methods reaches a threshold beyond
which it cannot remove noise anymore.
Submission
Make sure you have run all cells in your notebook in order before running the cell
below, so that all images/graphs appear in the output. The cell below will
generate a zip file for you to submit. Please save before exporting!
Upload your notebook and separate pdf for theoretical questions if needed
In [ ]: # Save your notebook first, then run this cell to export your submission.
grader.export(pdf=False, run_tests=True)