Digital Signal & Filtering

Filter Design, Applications in Adaptive Filter Design, Applications in Adaptive

Filtering & Communications
Pham Van Tuan
Electronic & Telecommunication Engineering
Danang University of Technology
Course Administration
Courses sequence: EE235, EE233, EE341
The prerequisite of this course
Discrete time signals
Linear time invariant (LTI) systems
Fourier transform (continuous/discrete time) Fourier transform (continuous/discrete time)
Laplace transform and Z transform
Credits: 3
Grading policies:
IIR: 50% (Final); FIR: 30% (Midterm); Writing exam:
20% (HW)
Goals
To provide students fundamental knowledge of
Digital filter characteristics
Design principles & design specifications
Applications to communications
Applications to adaptive filtering
Textbooks: Textbooks:
V.K. Ingle, J.G. Proakis, Digital Processing Using
Matlab, 2
nd
Ed., Thomson Learning, 2007
References:
S. S. Haykin, Adaptive Filter Theory, 4
th
Ed., Prentice Hall, 2002.
PP. Vaidyanathan, Multirate System and Filter Banks, Prentice Hall, 2004
Nguyn Quc Trung, X l s tn hiu, Tp 1,2.
Matlab Software 2007 and later versions
EE 442, Digital Signals & Filtering, EE Dept., University of Washington
Schedules
Reviews:
Discrete-Time Signals, DTFT: 1 week (chapter 2,3)
Z Transform, DFT, FFT: 1 week (chapter 4,5)
Digital Filter Structures: 1 week (chapter 6)
Finite Impulse Response (FIR) Filter Design: 2 weeks Finite Impulse Response (FIR) Filter Design: 2 weeks
(chapter 7)
Infinite Impulse Response (IIR) Filter Design: 2 weeks
(chapter 8)
Applications in Adaptive Filtering : 1 week (chapter 9)
Applications in Communications: 1 week (chapter 10)
Projects (midterm & final) will be announced on-site
Lecture 1
Fundamentals of Digital Signals
and Digital Filtering and Digital Filtering
(chapter 1)
Why Need Signal Processing ?
Signal processing is an operation designed for
extracting, enhancing, storing and transmitting expected
(useful) information.
Which are expected (useful) and unwanted (noisy)
information ? Very much depend on subjective and
objective of application. objective of application.
Teamwork and give examples
How to process signals? In practice, mostly deal with
analog signals. They are processed by analog devices
(active and passive elements ?): Analog Signal
Processing
Draw scheme ?
What Is DSP ?
Equivalent analog signal processor
Process signals based on digital hardware containing:
adders, multipliers, logic elements
Can be general-purpose computer or specific-purpose
microp. or digital hardware
PrF ADC DSP DAC PoF
Analog Analog
Equivalent analog signal processor
PrF: Pre-filter or antialiasing Filter
ADC/DAC: Analog-to-digital and Digital-to-Analog
converters
PoF: Post-filter to smooth out staircase waveform
Why DSP ?
It seems that ASP is simpler looking than DSP.
Advantages of DSP over ASP:
Relatively develop, test on general purpose computer
Extremely stable processing capability
Easily and flexibly be modified in real time Easily and flexibly be modified in real time
Normally with higher precision
Can be cost-effective (DSPs, VLSI, FPGA, ASIC)
Disadvantages ?
Lecture #1: Lecture #1: A big picture about A big picture about
Digital Signal Processing Digital Signal Processing
Duration: 1 hr
Outline:
1. Signals 1. Signals
2. Digital Signal Processing (DSP)
3. Why DSP?
Learning Digital Signal Processing is not something you accomplish; its a journey
you take.
R.G. Lyons, Understanding Digital Signal Processing
* * * * * * *
Given a continuous-time (CT) signal x(t), we can sample
it to generate the discrete-time (DT) signal x(n)
period sampling the is T where ), ( ) ( = T n x n x
Signals ?
1-D signals: speech, audio, biosensor, ECG, etc.
2-D signals: image (optical, X-ray, MRI), remote sensing
data, etc.
2.5-D signals: video, ultrasound images (2-D + time)
3-D signals: graphics and animation
Multi-D signals: multi-spectral data, sonar array etc.
period sampling the is T where ), ( ) ( = T n x n x
11--D signals D signals
ECG
EEG
Color image
Speech signal
22--D image signals D image signals
Binary image??? Color image Grey image
(indexed image)
ICA
ECA
Plaque
Imaging
2-D images (MRI)
CCA Bifurcation
ICA
Plaque
CCA
Imaging
location
2.5 2.5--D video signals D video signals
33--D animated signals D animated signals
Information Technologies Are Driving
Fast
Image Sensors: Digital Still Camera (DSC), PC
Camera (CCD and CMOS), Digital Camcorder,
portable scanners, etc.
Powerful Computers: faster and multi-core CPU, low
power embedded CPU, USB2 ports, CD/DVD RW, power embedded CPU, USB2 ports, CD/DVD RW,
flash memory & mini disk, etc.
Data Compression: audio/image/video coding (data
compression) standards. HDTV and digital radio.
2D/3D graphics/video cards: faster display and better
quality.
Communication: Internet, ATM, ADSL, wireless LAN,
WiMAX, WDM in optics, transceiver/modem, etc.
Two Main Categories of DSP
APPLICATIONS
Spectrum analysis
Feature extraction
APPLICATIONS
Noise removal
Interference separation
Signal compression
Analyze
Processed
Measures
Feature extraction
Signal detection
Signal estimation
Signal verification
Signal recognition
Signal modeling
Signal coding
Signal synthesis
Spectrum shaping
Filter
R
a
d
a
r
Biomedical Biomedical
Analysis of biomedical signals, diagnosis, patient
monitoring, preventive health care
Image processing Image processing
Image enhancement: processing an image to be more
suitable than the original image for a specific application
It makes all the difference whether one sees darkness through It makes all the difference whether one sees darkness through It makes all the difference whether one sees darkness through It makes all the difference whether one sees darkness through
the light or brightness through the shadows the light or brightness through the shadows the light or brightness through the shadows the light or brightness through the shadows
David Lindsay David Lindsay David Lindsay David Lindsay
Image processing Image processing
Image compression: reducing the redundancy in the
image data
UW Campus (bmp) 180 kb UW Campus (jpg) 13 kb
Music Music
Recording, encoding, storing
Playback
Manipulation/mixing
Communication Communication
Digital telephony: transmission of information in digital
form via telephone lines, modern technology, mobile
phone
Speech compression Speech compression
Speech Speech
recognition recognition
Fingerprint recognition Fingerprint recognition
Image Image restoration restoration
Image restoration: reconstruct a degraded image using a
priori knowledge of the degradation phenomenon
Noise removal Noise removal
Lecture #1: Lecture #1: A big picture about A big picture about
Digital Signal Processing Digital Signal Processing
Duration: 1 hr
Outline:
1. Signals
2. Digital Signal Processing (DSP)
3. Why DSP?
What is Digital Signal Processing? What is Digital Signal Processing?
Represent a signal by a sequence of numbers (called a
"discrete-time signal or "digital signal").
Modify this sequence of numbers by a computing process
to change or extract information from the original signal
The "computing process" is a system that converts one
digital signal into anotherit is a "discrete-time system or
"digital system.
Transforms are tools using in computing process
Discrete Discrete--time signal vs. time signal vs.
continuous continuous--time signal time signal
Continuous-time signal:
- define for a continuous duration of time
- sound, voice
Discrete-time signal:
- define only for discrete points in time (hourly, every
second, )
- an image in computer, a MP3 music file
- amplitude could be discrete or continuous
- if the amplitude is also discrete, the signal is digital.
CT signal vs. DT signal CT signal vs. DT signal
CT signal DT signal
00 10 00 10 11
Signal processing systems Signal processing systems
Processing
Analog signal
x(t)
Analog signal
y(t)
Analog signal processing
Processing
A/D
D/A
Analog signal
x(t)
Analog signal
y(t)
Digital signal processing
Digital signal processing Digital signal processing
implementation implementation
Performed by:
Special-purpose (custom) chips: application-specific integrated
circuits (ASIC) circuits (ASIC)
Field-programmable gate arrays (FPGA)
General-purpose microprocessors or microcontrollers (P/C)
General-purpose digital signal processors (DSP processors)
DSP processors with application-specific hardware (HW)
accelerators
Digital signal processing Digital signal processing
implementation implementation
Digital signal processing Digital signal processing
implementation implementation
Use basic operations of addition,
multiplication and delay
Combine these operations to accomplish
processing: a discrete-time input signal
another discrete-time output signal
An example of main step: An example of main step:
DT signal processing DT signal processing
From a discrete-time input signal:
{ 1 2 4 -9 5 3 }
Create a discrete-time output signal: Create a discrete-time output signal:
{ 1/3 1 7/3 -1 0 -1/3 8/3 1 }
What is the relation between input and output signal?
Lecture #1: Lecture #1: A big picture about A big picture about
Digital Signal Processing Digital Signal Processing
Duration: 1 hr
Outline:
1. Signals
2. Digital Signal Processing (DSP)
3. Why DSP?
Advantages of Digital Signal Advantages of Digital Signal
Processing Processing
Flexible: re-programming ability
More reliable
Smaller, lighter less power Smaller, lighter less power
Easy to use, to develop and test (by using the
assistant tools)
Suitable to sophisticated applications
Suitable to remote-control applications
Limitations of Digital Signal Limitations of Digital Signal
Processing Processing
A/D and D/A needed aliasing error and
quantization error quantization error
Not suitable to high-frequency signal
Require high technology
Lecture #2: Lecture #2: Analog Analog--to to--Digital and Digital and
Digital Digital--to to--Analog conversion Analog conversion
Duration: 2 hr
Outline: Outline:
1. A/D conversion
2. D/A conversion
A
D
C
A
D
C
Sampling Sampling
Continuous-time signal discrete-time signal
Analog
world
Digital
world
Sampling
Sampling Sampling
Taking samples at intervals and dont know what happens in
between cant distinguish higher and lower frequencies:
aliasing
How to avoid aliasing?
Nyquist sampling theory Nyquist sampling theory
To guarantee that an analog signal can be perfectly
recovered from its sample value
Theory: a signal with maximum of frequency of WHz must be
sampled at least 2Wtimes per second to make it possible to
reconstruct the original signal from the samples reconstruct the original signal from the samples
Nyquist sampling rate: minimum sampling frequency
Nyquist frequency: half the sampling rate
Nyquist range: 0 to Nyquist frequency range
To remove all signal elements above the Nyquist frequency
antialiasing filter
Anti Anti--aliasing filter aliasing filter
m
a
g
n
i
t
u
d
e
frequency
Analog signal
spectrum
Anti-aliasing
filter response
0 W 2W =f
s
3W 4W
frequency
0 W 2W =f
s
3W 4W
m
a
g
n
i
t
u
d
e
frequency
Filtered analog signal
spectrum
Some examples of sampling frequency Some examples of sampling frequency
Speech coding/compression ITU G.711, G.729, G.723.1:
fs = 8 kHz T = 1/8000 s = 125s
Broadband system ITU-T G.722:
fs = 16 kHz T = 1/16 000 s = 62.5s
Audio CDs:
fs = 44.1 kHz T = 1/44100 s = 22.676s
Audio hi-fi, e.g., MPEG-2 (moving picture experts group),
AAC (advanced audio coding), MP3 (MPEG layer 3):
fs = 48 kHz T = 1/48 000 s = 20.833s
Sampling and Hold Sampling and Hold
Sampling interval T
s
(sampling period): time between samples
Sampling frequency f
s
(sampling rate): # samples per second
Analog signal Sample-and-hold signal
0 1 2 3 4
Quantization Quantization
Continuous-amplitude signal discrete-amplitude signal
Quantization step
Quantization errors & coding Quantization errors & coding
Quantized sample N-bit code word
1.5V
0.0V
0.5V
1.0V
1.5V
0.82V
1.1V
1.25V
Example of quantization and coding Example of quantization and coding
Analog pressures
are recorded,
using a pressure
transducer, as
voltages between
0 and 3V. The
Digital code
000
001
010
Quantization
Level (V)
0.0
0.375
0.75
Range of
analog
inputs (V)
0.0-0.1875
0.1875-0.5625
0.5625-0.9375
signal must be
quantized using a
3-bit digital code.
Indicate how the
analog voltages
will be converted
to digital values.
010
011
100
101
110
111
0.75
1.125
1.5
1.875
2.25
2.625
0.5625-0.9375
0.9375-1.3125
1.3125-1.6875
1.6875-2.0625
2.0625-2.4375
2.4375-3.0
Example of quantization and coding Example of quantization and coding
An analog voltage
between -5V and 5V
must be quantized
using 3 bits. Quantize
Digital code
100
101
110
Quantization
Level (V)
-5.0
-3.75
-2.5
Range of analog
inputs (V)
-5.0 -4.375
-4.375-3.125
-3.125-1.875
each of the following
samples, and record
the quantization error
for each:
-3.4V; 0V; .625V
110
111
000
001
010
011
-2.5
-1.25
0.0
1.25
2.5
3.75
-3.125-1.875
-1.875-0.625
-0.6250.625
0.6251.875
1.8753.125
3.1255.0
Quantization parameters Quantization parameters
Number of bits: N
Full scale analog range: R
Resolution: the gap between levels Q = R/2
N
Quantization error = quantized value actual value
Dynamic range: number of levels, in decibel Dynamic range: number of levels, in decibel
Dynamic range = 20log(R/Q) = 20log(2
N
) = 6.02N dB
Signal-to-noise ratio SNR = 10log(signal power/noise power)
Or SNR = 10log(signal amplitude/noise amplitude)
Bit rate: the rate at which bits are generated
Bit rate = N.f
s
Noise removal by quantization Noise removal by quantization
Q/2
Noise
Error
Q
Quantized signal + noise After re-quantization
Non Non--uniform quantization uniform quantization
Quantization with variable quantization step Q value is
variable
Q value is directly
proportional to signal
Non-
uniform
Output
proportional to signal
amplitude SNR is
constant
Most used in speech
Input
Uniform
AA--law compression curve law compression curve

<
+
+

+
=
1 ) t ( s
A
1
,
A ln 1
) ) t ( s A ln( 1
A
1
) t ( s 0 ,
A ln 1
) t ( s A
) t ( s
1
1
1
1
2
1.0
s
2
(t)
- 1.0
- 1.0
1.0 0
A=87.6
A=1
A=5
s
1
(t)
ITU G.711 standard ITU G.711 standard
Input range Step size Part 1 Part 2 No. code
word
Decoding output
0-1
...
30-31
2 000 0000
...
1111
0
...
15
1
...
31
32-33
...
62-63
2 001 0000
...
1111
16
...
31
33
...
63
64-67
...
124-127
4 010 0000
...
1111
32
...
47
66
...
126
128-135 8 011 0000 48 132 128-135
...
248-255
8 011 0000
...
1111
48
...
63
132
...
252
256-271
...
496-511
16 100 0000
...
1111
64
...
7
264
...
504
512-543
...
992-1023
32 101 0000
...
1111
80
...
5
528
...
1008
1024-1087
...
1984-2047
64 110 0000
...
1111
6
...
111
1056
...
2016
2048-2175
...
3968-4095
128 111 0000
...
1111
112
...
127
2112
...
4032
ITU G.711 A ITU G.711 A--law curve law curve
2
1
7/8
6/8
1.0
Code-word format:
Sign bit
0/1
Part 1 (3bits)
000 111
Part 2 (16bits)
0000 1111
1.0 1/2 1/4 1/8 1/16
1/8
8
6
7
5
4
3
5/8
4/8
3/8
2/8
0
Example of G.711 code word Example of G.711 code word
A quantized-samples value is +121
Sign bit: 0
Part 1: 010
Part 2: 1110 Part 2: 1110
Code word: 00101110
Decoding value: +122
A quantized-samples value is -121
Code word: 10101110
Lecture #2: Lecture #2: Analog Analog--to to--Digital and Digital and
Digital Digital--to to--Analog conversion Analog conversion
Duration: 2 hr
Outline: Outline:
1. A/D conversion
2. D/A conversion
D
A
C
D
A
C
Anti Anti--imaging filter imaging filter
Images
Anti-imaging
filter
m
a
g
n
i
t
u
d
e
0 W 2W =f
s
4W = 2f
s
frequency
Original two-sided
analog signal spectrum
Prob.1. An analog signal is converted to digital and then
back to analog signal again, without intermediate DSP.
HW HW
In what ways will the analog signal at the output differ
from the one at the input?
Prob.2. An analog signal is sampled at its Nyquist rate
1/T
s
, and quantized using L quantization levels. The
derived signal is then transmitted on some channels.
HW HW
(a) Show that the time duration, T, of one bit of the
transmitted binary encoded signal must satisfy
(b) When is the equality sign valid?
) L /(log T T
2 s

Prob.3. A set of
analog samples,
listed in table 1,
is digitized using
HW HW
Digital code
000
Quantization
Level (V)
0.0
Range of analog
inputs (V)
0.0 0.3125
n 0 1 2 3 4 5 6 7 8
Sample(V) 0.5715 4.9575 0.6250 3.6125 4.0500 0.9555 2.8755 1.5625 2.7500
is digitized using
the quantization
table 2.
Determine the
digital codes, the
quantized level,
and the
quantization
error for each
sample.
000
001
010
011
100
101
110
111
0.0
0.625
1.250
1.875
2.500
3.125
3.750
4.375
0.0 0.3125
0.31250.9375
0.93751.5625
1.56252.1875
2.18752.8125
2.81253.4375
3.43754.0625
4.06255.0
Prob.4. Consider that you desire an A/D conversion system, such
that the quantization distortion does not exceed 2% of the full
scale range of analog signal.
(a) If the analog signals maximum frequency is 4000 Hz, and
HW HW
(a) If the analog signals maximum frequency is 4000 Hz, and
sampling takes place at the Nyquist rate, what value of sampling
frequency is required?
(b) How many quantization levels of the analog signal are needed?
(c) How many bits per sample are needed for the number of levels
found in part (b)?
(d) What is the data rate in bits/s?
Prob.5. An analog voice signal with voltage between
-5V and 5V must be quantized using ITU G.711
standard. Encode each of the following samples; and
HW HW
standard. Encode each of the following samples; and
record the quantization error for each:
(a) -3.45198 V
(b) 1.01119 V
Prob.6. A 3-bit D/A converter produces a 0 V output for
the code 000 and a 5 V output for the code 111, with
other codes distributed evenly between 0 and 5 V.
HW HW
other codes distributed evenly between 0 and 5 V.
Draw the zero order hold output from the converter for
the input below:
111 101 011 101 000 001 011 010 100 110
Lecture #2 Lecture #2
The concept of frequency in The concept of frequency in
CT & DT signals CT & DT signals
Duration: 2 hrs
Outline: Outline:
1. CT sinusoidal signals
2. DT sinusoidal signals
3. Relations among frequency variables
Functions:
+ < < + =
+ < < + =
t ), t f 2 cos( A
t ), t cos( A ) t ( x
a
Mathematical description of CT Mathematical description of CT
sinusoidal signals sinusoidal signals
Plot:
+ < < + = t ), t f 2 cos( A
t
x
a
(t)
Acos
T
p
= 1/f
Properties of CT sinusoidal signals Properties of CT sinusoidal signals
1. For every fixed value of the frequency f, x
a
(t) is
periodic: x
a
(t+T
p
) = x
a
(t)
T
p
= 1/f: fundamental period T
p
= 1/f: fundamental period
2. CT sinusoidal signals with different frequencies are
themselves different
3. Increasing the frequency f results in an increase in the
rate of oscillation of the signal (more periods in a
given time interval)
Properties of CT sinusoidal Properties of CT sinusoidal
signals (cont) signals (cont)
For f = 0 T
p
=
For f = T
p
= 0
Physical frequency: positive Physical frequency: positive
Mathematical frequency: positive and negative
The frequency range for CT signal:
- < f < +
) ( ) (
2 2
) cos( ) (

+ +
+ = + =
t j t j
a
e
A
e
A
t A t x
Functions:
+ < < + =
+ < < + =
n ), n F 2 cos( A
n ), n cos( A ) n ( x
Mathematical description of DT Mathematical description of DT
sinusoidal signals sinusoidal signals
Plot:
0 5 1 0 1 5 2 0 2 5 3 0 3 5 4 0
-2
-1 . 5
-1
-0 . 5
0
0 . 5
1
1 . 5
2
Ti m e i n d e x n
A
m
p
l
i
t
u
d
e
x(n N) x(n) n + =
Properties of DT sinusoidal signals Properties of DT sinusoidal signals
1. A DT sinusoidal signal x(n) is periodic only if its
frequency F is a rational number
x(n N) x(n) n + =
n ) n F 2 cos( A )] ) N n ( F 2 cos[ A
0 0
+ = + +
k 2 N F 2
0
=
N
k
F
0
=
2. DT sinusoidal signals whose frequencies are separated by
an integer multiple of are identical
2
) n cos( ) n 2 n cos( ] n ) 2 cos[( ) n ( x + = + + = + + =
Properties of DT sinusoidal signals Properties of DT sinusoidal signals
All
are identical
) n cos( ) n 2 n cos( ] n ) 2 cos[( ) n ( x
0 0 0
+ = + + = + + =

+ + =
= + =
0 0
, 2
... , 2 , 1 , 0 ), cos( ) (
k
k n A n x
k
k k
3. The highest rate of oscillation in a DT sinusoidal signal is
obtained when:
Properties of DT sinusoidal signals Properties of DT sinusoidal signals
or, equivalently,
) or ( = =
)
2
1
F or (
2
1
F = =
0 5 10 15 20 25 30
-1
-0.5
0
0.5
1
F = 3/24
0 5 10 15 20 25 30
-1
-0.5
0
0.5
1
F = 3/12
F
0
= 1/8 F
0
= 1/4
) 2 cos( ) (
0
n F n x =
Illustration for Illustration for
property 3 property 3
0 5 10 15 20 25 30 0 5 10 15 20 25 30
0 5 10 15 20 25 30
-1
-0.5
0
0.5
1
F = 3/6
0 5 10 15 20 25 30
-1
-0.5
0
0.5
1
F = 3/4
F
0
= 1/2 F
0
= 3/4
- or -1/2 F 1/2: fundamental range
CT signal Sampling DT signal
x
a
(t) x
a
(nT)
+ ) nT f 2 cos( A
Sampling of CT sinusoidal signals Sampling of CT sinusoidal signals
s
f
f
F =
) t f 2 cos( A +
|
|

\
|
+

=
+
S
f
n f 2
cos A
) nT f 2 cos( A
Normalized
frequency
CT signals DT signals
2 F =
2 f =
Relations among frequency variables Relations among frequency variables
+ < <
+ < <
f
2 / 1 F 2 / 1 +
+
2 / f f 2 / f
T / T /
s s
+
+
s
f
f
F =
s
f
T
1
=
Exercise Exercise
Consider the analog signal
a) Determine the minimum sampling rate required to avoid aliasing
] [ , 100 cos 3 ) ( s t t t x =
b) Suppose that the signal is sampled at the rate f
s
= 200 Hz. What is
the DT signal obtained after sampling?
c) Suppose that the signal is sampled at the rate f
s
= 75 Hz. What is
the DT signal obtained after sampling?
d) What is the frequency 0 < f < f
s
/2 of a sinusoidal signal that yields
samples identical to those obtained in part (c)?
Prob.7. Consider the analog signal
a
x (t) 3cos2000 t+5sin6000 t+10cos12000 t =
HW HW
a) Determine the minimum sampling rate required to avoid aliasing
b) Suppose that the signal is sampled at the rate f
s
= 5000
samples/sec. What is the DT signal obtained after sampling?
c) What is the analog signal we can reconstruct from the samples if
we use ideal interpolation?
Prob.8. Consider the analog signal
a) Sketch the signal for t from 0 to 30 ms
HW HW
] [ , 100 sin 3 ) ( s t t t x =
a) Sketch the signal for t from 0 to 30 ms
b) The signal is sampled at the rate f
s
= 300 samples/s. Determine
the frequency of the DT signal x(n) and show that it is periodic.
c) Compute the sample values in one period of x(n). Sketch x(n) on
the same diagram with x(t). What is the periodic of x(n) in ms?
d) Can you find a sampling rate so that x(n) reaches its peak value?