Sampling Theory

with Applications using MATLAB

Anna Maeser
April 21, 2008
Abstract
Sampling Theory is the process of converting a function of continuous
space or time into a discrete sequence of numbers. This paper summa-
rizes the history behind sampling theory and shows how it can be used
along with the Discrete Fourier Transform and the Short-Time Fourier
Transform in applications. These transforms are used in various elds
such as physics, engineering, and mathematics. Applications to music are
discussed in terms of ltering, compression, and denoising. MATLAB is
used for the analysis.
1 Introduction
This paper considers the sampling theorem and applications to ltering,
compression, and denoising. E.T. Whittaker, Harold Nyquist, A. Ko-
telnikov, and Claude E. Shannon are the most prominent contributors to
the development of sampling. As a result, the sampling theorem is often
referred to as the Shannon sampling theorem, the Whittaker-Shannon
sampling theorem, the Whittaker-Shannon-Kotelnikov sampling theo-
rem, and the Nyquist-Shannon sampling theorem [5]. Whittaker rst
proved the sampling theorem from the point of view of interpolation the-
ory in 1915 through his study of Cardinal Functions. His second son, J.M.
Whittaker, further developed this idea and was the mathematician to coin
the term Cardinal Series in 1929 [6]. In 1928, Harry Nyquist published
a paper titled Certain Topics in Telegraph Transmission Theory reveal-
ing an important discovery known as the Nyquist rate that species the
lowest rate of discrete samples required to reproduce an original continu-
ous signal [4].
All of these advances laid the foundation for Claude Shannons dis-
covery of the sampling theorem in 1948 and its correlation to Fourier
transforms (5). Shannon is often referred to as the father of information
theory. An important application of Shannon sampling is that a band-
limited signal can be replaced by a discrete sequence of samples from the
signal without losing any information [4]. It is remarkable that a certain
function can be completely reconstructed from a discrete set of samples
1
taken at equal intervals. This is important in practice because it is often
necessary to convert analog models into digital to make use of modern
technology. In real-world applications, functions are usually given as se-
quences of values because they are measured over discrete sets of values.
Discrete Fourier transforms (7) and short-time Fourier transforms (8) will
be used to analyze these discrete functions throughout this paper [1]. This
includes the derivation of the sampling theorem with the use of Fourier
transforms and inverse Fourier transforms.
Joseph Fourier was a French mathematician who claimed that any
function dened on a nite interval has a Fourier series expansion (1). He
expanded on the works of Swiss mathematicians Daniel Bernoulli and
Leonhard Euler in the mid 1700s that was met with extreme doubt.
Fourier presented his theory to the French Academy in 1807 and was met
with similar doubts. He presented The Propogation of Heat in Solids
and the judges, mainly Lagrange, criticized his use of the Fourier series
as an expression of the initial temperature distribution. After his work
was rejected at this time, Fourier resubmitted his revised theory to the
Academy four years later and was met with great praise and acceptance.
He was then awarded the Academys Grand Prize in Mathematics [1]. His
outstanding accomplishments continue to accelerate advances in applied
mathematics. Several preliminary denitions having to do with the works
of Joseph Fourier must be mentioned before sampling theory can be fully
understood.
Fourier series deal with expanding a periodic function in terms of the
specic set of functions 1, cos x, cos 2x, cos 3x, ..., sin x, sin2x, sin3x, ...
Denition 1. A Fourier Series expansion of a function f can be ex-
pressed
f(x) = a0 +

n=1
(an cos nx + bn sin nx), (1)
where the coecients a0, an, and bn are called Fourier coecients of f
and are given by the following formulae:
a0 =
1
2

f(x) dx,
an =
1

f(x) cos nxdx (n = 1, 2, ...), (2)

bn =
1

f(x) sin nxdx (n = 1, 2, ...).

The functions cos x and sinx can be written using complex exponen-
tials as
cos x =
e
ix
+ e
ix
2
, sin x =
e
ix
e
ix
2i
,
which allows us to dene the Fourier series in the complex form.
2
Denition 2. Let f be a 2p-periodic piecewise smooth function. The
complex form of the Fourier series is
f(x) =

n=
cne
i
n
p
t
, (3)
where the Fourier coecients cn are given by
cn =
1
2p

p
p
f(t)e
i
n
p
t
dt, (n = 0, 1, 2, ...). (4)
The continuous Fourier Transform is denoted

f() and is a general-
ization of the complex Fourier series. It denes a relationship between a
signal in the time domain and its representation in the frequency domain.
Denition 3. The Fourier Transform of a function f is given by

f() =
1

f(x)e
ix
dx, (< < ). (5)
Denition 4. The Inverse Fourier Transform is dened
f(x) =
1

f()e
ix
d, (< x < ). (6)
As described previously, it is often necessary to sample a continuous
function to obtain a discrete representation. The next important de-
nition, the Discrete Fourier Transform, denoted X(k), can be viewed as
sampling from the continuous Fourier transform.
Denition 5. Given an N-sequence x=((x(0), x(1),...,x(N-1)), the N-
point Discrete Fourier Transform of x is dened by
X(k) =
1

N
N1

j=0
x(j)e
2ijk/N
, k = 0, 1, ..., N 1. (7)
The Fourier transform provides the intensity of a frequency; however,
in order to discover how long a specic frequency will occur, the Short-
Time Fourier Transform, also known as the Gabor transform, is necessary.
The short-time Fourier transform is a technique in which Fourier trans-
forms are taken of short consecutive segments of a signal. This transform
provides a clear representation of the signal and a frequency image of the
signal locally in time [2].
Denition 6. The Short-time Fourier Transform is dened as
Vgf(x, ) =
1

f(t)g(t x)e
it
dt, (8)
where t represents time and represents the bandwidth of the signal. g is
the lter window which is commonly represented by a Gaussian, f(x) =
e
x
2
, or an indicator function, f(x) = 1 if |x| < a and 0 otherwise. The
bar represents the complex conjugate.
3
Grasping these preliminary denitions and their purpose is important
for the understanding of sampling theory. This paper will introduce the
theory behind sampling in terms of band-limited functions, time-limited
functions, and the Fourier transform as well as provide an important ap-
plication to the theorem. Sampling plays a crucial role in communications,
signal processing, transmission, and information theory. This paper will
discuss sampling analysis on continuous sound les by transforming a con-
tinuous signal into a discrete sequence in which MATLAB can be utilized.
The purpose behind ltering will be discussed in terms of current appli-
cations with visual representations.
2 Theory Behind the Sampling Theorem
2.1 Band-Limited Functions and Band Width
Sampling analog signals is a prerequisite to digital signal processing. In
analog transmission, such as a telephone system, it is required that the
analog speech signal be sampled before becoming a part of the digital
telephone system. A digital signal processing device that converts a con-
tinuous signal to a discrete signal, known as a sampler, creates a sequence
of impulses by keeping track of the level of the signal over time at equal
intervals. These impulses are then coded as numbers and can be processed
digitally with the use of an analog-to-digital converter (ADC). This might
make it appear that information from the original signal is lost due to the
sampling. Although this is valid to some extent, it can be seen that the
lost information is unimportant as long as the sampling rate is high enough
[3].
This brings us to the idea of a band-limited functions and band width.
Denition 7. A function f(x) is band-limited if its Fourier transform
(5) vanishes outside a nite interval. Then there exists a positive number
W such that

f() = 0 for all || > W. The number W is called the band
width of f.
The majority of functions are not considered band-limited; however, most
functions can be approximated by band-limited functions [1].
The denition of band-limited functions leads to an important result.
Basically, every band-limited function with band width W can be written
as an innite sum. The coecients in this sum specically represent
function values at a discrete set of equally spaced sample points. The
sampling theorem for band-limited functions results.
Theorem 1. The Sampling Theorem for Band-Limited Functions
Suppose that f is band-limited with band width W. Then for all x,
f(x) =

n=
f(
n
W
)
sin(Wx n)
(Wx n)
, n = 0, 1, 2, ... (9)
We can see from this theorem that f is completely determined by its
sample values f(
n
W
). Harold Nyquist recognized that the larger the band
width, the more sample points per unit length are required for (9) to
4
hold. As a result, the smallest number of sample points per unit length
are required for (9) to be true is known as the Nyquist Sampling Rate.
This rate corresponds to the smallest W of f and is twice the bandwidth
of the signal or 2W[1].
2.2 Time-Limited Functions
A time-limited function is one in which the function itself vanishes outside
a nite time interval. This diers from the band-limited function where
the Fourier transform of the function vanishes outside a nite frequency
interval. The function and the Fourier transform in (9) can be exchanged
to obtain the sampling theorem for time-limited functions.
Theorem 2. The Sampling Theorem for Time-Limited Functions
Suppose t represents time and f(t) = 0 for all |t| > T. Then for all ,

f() =

n=

f(
n
T
)
sin(T n)
(T n)
, n = 0, 1, 2, ... (10)
The result is similar to the sampling theorem for band-limited func-
tions. In this case, the Fourier transform,

f of the function is completely
constructed by the coecients. In other words, sampling at the points
n
T
formulates

f [1].
2.3 Proof of the Sampling Theorem
In this section, the goal is to prove (9). That is, we will prove
f(x) =

n=
f(
n
W
)
sin(Wx n)
Wx n
for a bandlimited functionf(x).
Proof. By denition, we know that

f vanishes outside [W, W] [1]. By
the Inverse Fourier Transform (6), we have
f(x) =
1

f()e
ix
d
=
1

W
W

f()e
ix
d. (11)
On the -axis,

f can be extended periodically with period 2W due to
the fact that

f vanishes outside [W, W]. Using the complex form of the
Fourier series (3),

f can be represented

f() =

n=
cne
i
n
W

(|| < W) (12)
=

n=
cne
i
n
W

.
Using the complex Fourier coecients (4), we have
5
cn =
1
2W

W
W

f()e
i
n
W

d. (13)
Noticing the relationship between cn and f(x) in (11) and (13), we
can simplify and get
cn =

2
2W
f(
n
W
). (14)
One more integral must be solved before continuing the proof. The
solution (15) uses the identity e
ix
= cos x + i sinx.
1
2W

W
W
e
i
n
W

e
ix
d =
1
2W

W
W
e
i(Wxn)/W
d
=
1
2W

W
W
cos(
Wx n
W
) d
=
sin(Wx n)
Wx n
. (15)
We can now complete the proof using f(x) in Equation (11),

f() in
Equation (12), and cn in Equation (14).
f(x) =
1

W
W

f()

n=
cne
i
n
W

e
ix
d
=
1

W
W

f()

n=
c
n

2
2W
f(
n
W
) e
i
n
W

e
ix
d.
Interchanging the order of the integral and sum, we obtain
f(x) =

n=
f(
n
W
)
1
2W

W
W
e
i
n
W

e
ix
d
=

n=
f(
n
W
)
sin(Wx n)
Wx n
using (15).
3 Applications with MATLAB
In practice, it is usually necessary to work with both time- and band-
limited functions together. In reality, it is not possible to use the Fourier
transform to simultaneously obtain both exceptional time resolution and
frequency resolution. This is based on what is known as the Heisenberg
6
Uncertainty Principle. In this section, we will illustrate two ways of sam-
pling using MATLAB. The rst subsection will provide an example of
sampling a sound le using the Fourier transform. The second subsection
will provide a similar example using the short-time Fourier transform.
Both essentially consist of distorting a signal by adding a high frequency
tone to a sound clip and using transforms to lter out the high frequency
and reproduce the original signal.
3.1 Sampling with the Discrete Fourier Trans-
form
The rst application to sampling makes use of the discrete Fourier trans-
form (7). With the aid of MATLAB (code included in the Appendix), a
sound clip is read into the program and a high frequency tone is added to
half the signal in the form of a sine wave to create a constant pitch. The
formula used to create the sound is x = I sin(2n) where I is the intensity
of the tone. In this application, I is taken to be 0.2 and the length of n
is 67,337. An image of the original signal can be seen in Figure 1. The
signal with the added sine wave can be seen in Figure 2. The discrete
Fourier transform is taken of the signal and plotted in Figure 3.
Figure 1: This is what the original signal looks like.
Figure 2: This is the signal in Figure 1 with a constant tone added to half of it with
the formula x = .2 sin(2n).
7
Figure 3: This is the plot of the Fourier transform of the added noise. The spike
in the graph indicates the frequency component corresponding to the constant noise.
The modulus is taken to avoid complex numbers.
The next step is to get rid of the high frequency in the signal in order
to eliminate the constant tone. First, every frequency component from
the spike forward is set to zero as seen in Figure 4. Then, with the use
of the inverse Fourier transform, it is possible to lter the tone out of the
sound clip and reproduce the original signal (Figure 5).
Figure 4: The plot of the Fourier transform is reproduced after setting the freqency
components from the spike forward to zero.
8
Figure 5: This plot shows the original signal surrounded by the high frequency tone
that was ltered out. The rst half of the plot shows a much greater dierence between
the original le and the tone while the two are much closer in the second half. This is
a result of adding the tone to only the rst half.
3.2 Sampling with the Short-Time Fourier Trans-
form
One of the shortcomings of the discrete Fourier transform is that it does
not give information as to the time a frequency component occurs. This
is where the short-time Fourier transform comes into play. The process
starts out the same by reading a wav le into MATLAB. It is then neces-
sary to create a lter. Filter choices vary from application to application
and depend on the frequency domain required for analysis. Both the
Blackman and block lters will be used along with the short-time Fourier
transform.
3.2.1 Blackman Filter
Common lters include lowpass, highpass, bandpass, bandstop, and all-
pass lters. In this case of denoising, a lowpass lter known as the Black-
man lter is used. Lowpass lters reduce the amplitude of frequencies
above a certain cuto frequency and allows those below the cuto to
pass through. Every lowpass lter has a stopband associated with it. A
stopband is a band of frequencies in which the lter does not let signals
through. Any frequency between the upper and lower limits are not trans-
mitted. The minimum stopband associated with the Blackman lter is 74
dB.
The Blackman lter is created using the expression
w(n) = a0 a1 cos(
2n
N 1
) + a2 cos(
4n
N 1
)
9
The coecients are given by the following formulae:
a0 =
1
2
(16)
a1 =
1
2
a2 =

2
.
is typically taken to be 0.16 in the Blackman lter, which results in
w(n) = .42 .5 cos(
2n
N 1
) + .08 cos(
4n
N 1
), (17)
where N is the length of the lter and 0 n N 1. After adding
noise to a sound le, the Blackman lter is applied, resulting in a two-
dimensional graph that provides the time that the noisy frequency occurs.
It is then possible to eliminate the noise associated with the sound le and
reproduce the original le. The short-time Fourier transform is often more
benecial than the discrete Fourier transform in terms of frequency and
time detail. As previously mentioned, it is unfortunately not possible
to obtain exceptional time and frequency detail at the same time. The
extent to which either is obtained is dependent upon the ltering length,
or N in (17). N can either be considered thick or thin. The rst set
of gures depicts when N is set to a length of 256 samples and is thick.
This results in more detail in terms of frequency (Figure 6). On the other
hand, Figure 7 shows when N is set to a thin length of 32, exhibiting
more extreme time detail.
10
Figure 6: The top image is a picture of the Blackman lter of length 256. The
Gaussian form mentioned in the denition of the short-time Fourier transform (8) is
reiterated here. The second image shows excellent detail in terms of frequency on
the vertical axis. Time is on the horizontal axis. The coloring in the plot represents
intensity. The brighter the color, the higher the intensity. An image such as this is
helpful to audio engineers when deciding the location of frequencies of specic noises.
Figure 7: The image shows excellent time detail on the horizontal axis. The frequency
in this image remains the same as in Figure 6; however, the frequency detail is miniscule
in comparison.
11
3.2.2 Block Filter
Another type of lter called a block lter can be used to analyze the
data. It is obvious by looking at Figure 8 that the Blackman lter is more
appropriate for analysis. The reason for this is that it is benecial for
the lter and the Fourier transform of the lter to decay quickly in both
time and frequency. The Blackman lter meets this criterion. The Fourier
transform of the block function is the sinc function, which does not decay
quickly. Therefore, the resulting plot does not provide time or frequency
information as clearly as the Blackman lter (Figure 8).
Figure 8: The top image shows the block lter of length 32. The bottom image can
be compared with Figure 7 and it is obvious that the time axis is not as clear. The
block lter is illustrating poor time-frequency localization as seen by the artifacts and
is a direct result of the slow decay of its Fourier transform.
12
4 Aliasing
When discussing sampling, it is essential to understand the problem of
aliasing. This term is simply known as undersampling. Noise is nat-
urally going to contaminate any recorded physical signal. In addition to
the actual signal f, the recorded signal will primarily look like h = f +r,
where r contains relatively high frequencies but has relatively small ampli-
tude. As a result, the spectrum of f will be narrower than the spectrum of
h. This means that when one bases a sampling rate on what they assume
the band width of f to be, the sampling rate will have a good chance
of being too low. The calculated spectrum will not be useful when this
occurs. Often times, a frequency will masquerade as another resulting in
an undesirable sound. In order to avoid this problem, the signal must be
ltered with a lowpass lter before it is sampled. This is advantageous be-
cause the relevant sampling rate will be obtained and high-frequency noise
is eliminated. Since sampling of a signal is done discretely, its spectrum
becomes periodic. The phenomenon of aliasing occurs when the periods
overlap as seen in Figure 9. The Nyquist sampling rate mentioned in sec-
tion 2.1 comes into play here as it represents the minimun sampling rate
resulting in no aliasing. It can be seen in Figure 10, by increasing the sam-
pling rate, the components of the spectrum are separated and problems
associated with undersampling are avoided [3], [5].
Figure 9: (a) shows a signal with its Fourier transform in (b). In (c), the signal is
sampled and the transform becomes periodic and overlapping (d). This overlap is due
to undersampling, resulting in aliasing.
13
Figure 10: A high sampling rate results in separate, non-overlapping components of
the spectrum.
5 Conclusion
Many aspects of modern day applied mathematics are a result of the con-
tributions of Claude Shannon and Joseph Fourier. The use of the short-
time Fourier transform is common in many scientic elds. The dierence
between the discrete Fourier transform and the short-time Fourier trans-
form lies in the amount of detail provided by each. The discrete Fourier
transform gives information about frequencies and their intensities, while
the short-time Fourier transform illustrates when a certain frequency ac-
tually occurs. Unfortunately, it is not possible to obtain extreme time
and frequency detail at the same time. It is possible to obtain one or the
other depending on the ltering length used in analysis. The longer the
lter, the more detail provided in terms of frequency. The shorter the
lter, the better the time detail. Through analysis with the Blackman
and the block lter, it is obvious that a lter with fast decay, such as
the Blackman, results in a more useful outcome. It is important to avoid
the problem of undersampling in order to discretely sample a continuous
signal and accurately obtain a desirable approximation. Signal process-
ing and applications to electrical engineering are among the many elds
making use of these ideas and the observations and discoveries of Shannon
and Fourier.
References
[1] N.H. Asmar. Partial Dierential Equations with Fourier Series and
Boundary Value Problems. (Pearson Education, Inc.), 398:546-571,
2005.
[2] J.S. Byrnes and J.L. Byrnes. Recent Advances in Fourier Analysis and
Its Applications. (Kluwer Academic Publishers), 436:441-442, 1990.
[3] C. Gasquet and P. Witomski. Fourier Analysis and Applications: Fil-
tering, Numerical Computation, Wavelets. (Springer-Verlag New York
Inc.), 343-360, 1999.
[4] A.J. Jerri. The Shannon Sampling Theorem-Its Various Extensions
and Applications: A tutorial Review. (The Institute of Electrical and
Electronics Engineers, Inc.), 1565-1567, 1977.
14
[5] R.J. Marks II. Introduction to Shannon Sampling and Interpolation
Theory. (Springer-Verlag New York Inc.), 1-5:37, 1991.
[6] A.I. Zayed. Advances in Shannons Sampling Theory. (CRC Press,
Inc.), 1-5:19, 1993.
Appendices
MATLAB code from Section 3.1
y = wavread(boxachoc.wav); length(y);
L=2^16;
j=zeros(1,(L/2+1000));
n=[0:(L/2+1000),j];
x=.2*sin(2*n)
x=x+y(1:2000+L+1,1);
L=length(x)-1;
wavplay(x,11000)
zp=zeros(L,1);
xp=[x(1:L/2); zp; x(L/2+1:L)];
N=length(xp);
X=fft(xp)/sqrt(N);
colormap(hot)
figure(1);
plot(x);
figure(2);
plot(abs(X(1:L)));
X(30000:N)=0;
X(N-31000:N)=0;
figure(3);
plot(abs(X(1:L)));
XX=sqrt(N)*(ifft(X));
xlabel(Frequency)
ylabel(Intensity)
title(Fourier Transform without High Frequency Tone)
filtered=[XX(1:L/2);XX(3*L/2:2*L)];
filtered=real(filtered);
wavplay(filtered,11000)
figure(4);
plot(1:L+1,x,1:L+1,filtered,g)
MATLAB code from 3.2
N =256;
M=256;
w = .42 - .5*cos(2*pi*(0:M-1)/(M-1)) ...
+ .08*cos(4*pi*(0:M-1)/(M-1)); %blackman window i.e. tight frame
%w=ones(1,N); %To use block filter
w=w;
15
y = wavread(boxachoc.wav);
L=2^16;
j=zeros(1,(L/2)+1000); %adding a tone to half the signal
n=[0:(L/2)+1000,j];
x=.2*sin(2*n);
x=x+y(1:2000+L+1,1);
wavplay(x,11000)
nframes=67000;
Xtwz = zeros(N,nframes); % pre-allocate STFT output array
M = length(w); % M = window length, N = FFT length
zp =zeros(N-M,1) ; % zero padding (to be inserted)
R=1; % padding
M1=M/2;
xoff = 0;
% current offset in input signal x
for m=1:nframes
xt = x(xoff+1:xoff+M);
xtw = w .* xt; % apply window to current frame
xtwz = [xtw(M1+1:M); zp; xtw(1:M1)]; % windowed, z-padded
Xtwz(:,m) = fft(xtwz); % STFT for frame m
xoff = xoff +R; % advance in-pointer by hop-size R
end;
colormap(hot)
imagesc(abs(Xtwz));
plot(w);
figure(2)
colormap(hot)
subplot(211);plot(w)
subplot(212);
xlabel(Time)
ylabel(Frequency)
imagesc(abs(Xtwz(N/2:N,:)));
Xtwz(N/8:7*N/8,:)=0;
xlabel(Time)
ylabel(Frequency)
figure(3)
imagesc(abs(Xtwz(N/2:N,:)));
Xtwz(N/8:7*N/8,:)=0; % filter out tone
figure(4)
imagesc(abs(Xtwz(N/2:N,:)));
X=real(sum(Xtwz))/M;
figure(5)
plot(X);
wavplay(X,11000)
16