Support Vector Machine Techniques for Nonlinear Equalization

Support Vector Machine
Techniques for Nonlinear
Equalization
Bhaskar Raj Upadhyay
Shamman Noor Shoudha

Contents
I. Detection and Equalization
II. Support Vector Machine (SVM) technique
III. System Model
IV. Simulation Results – Decision Boundaries
V. BER Analysis
VI. Summary

Equalization – Non Linear
Equalization
Equalization
• Remove ISI and noise effects of the channel
• Located at the receiver
Severe channel effects, linear equalization
methods suffer – Noise enhancement
• Premise for Non-linear equalization
Non-linear equalization challenges
• Architectures maybe unmanageably complex
• Loss of information – nonlinear system maybe non-
invertible
• Computationally intensive
Why not think of it as a classification problem?

Why SVM
Train with small amounts of data
Training is straightforward
• Less ad hoc input from designer
Detection stage is efficient
Results Comparable to Volterra filters and
neural Networks
• Volterra filters – dimension grows quickly
• Neural networks – parameters of networks
determined in an ad-hoc fashion

Intro to SVM
Separate clouds of data using an optimal
hyperplane
Maximum margin classifiers don’t work well
with outliers
Outlier
Margin
Low Bias
High Variance

Intro to SVM
Soft Margins and Outliers
Separate clouds of data using an optimal
hyperplane
Maximum margin classifiers don’t work well
with outliers – Support vector classifiers do
Higher Bias
Low Variance
Allow for misclassifications
Soft Margin

Intro to SVM
Linear Classifier Limited
 Separate clouds of data using an optimal hyperplane
 In 2-Dimensions, the support vector classifier is a line
 Support vector machines – Deal with data with high amounts of
overlaps
No matter where you place
the margin, you will obtain a
lot of errors
2 categories, but no obvious
linear classifier to separate
them.

Intro to SVM
Non Linear Classifier
 Separate clouds of data using an optimal hyperplane
 In 2-Dimensions, the support vector classifier is a line
 Support vector machines – Deal with data with high amounts of
overlaps – non linear mapping from the pattern space to higher
dimensional feature space to create linearly separable clouds of
data
Move data to
higher dimension
Kernel Functions – Find
support vector classifiers in
higher dimensions

Support Vector Machines
Hyperplanes and decision criteria
 Objective – Find the weights (w) and bias (b) to define a hyperplane:
𝐰𝑇𝐱 + 𝑏 = 0
Optimal Hyperplane – A
hyperplane for which the
margin of separation is
maximized
Margin of separation is
maximum when norm of
the weight is minimized.
𝑑+
𝑑−

Support Vector Machines
Lagrangian Optimization Problem
Primal Problem
min 𝐿𝑝 =
1
2
𝐰 2 −
𝑖=1
𝑙
𝑎𝑖𝑦𝑖 𝐱𝑖 ∙ 𝐰 + 𝑏 +
𝑖=1
𝑙
𝑎𝑖
Dual Optimization problem
max 𝐿𝑑(𝑎𝑖) = 𝑎𝑖 −
1
2
𝑎𝑖𝑎𝑗𝑦𝑖𝑦𝑗𝐾(𝑥𝑖, 𝑥𝑗)
Under Constraints
𝑖=1
𝑙
𝑎𝑖𝑦𝑖 = 0 and 0 ≤ 𝑎𝑖 ≤ 𝐶
Why the dual?
Let’s us solve the problem by computing just the inner
products
𝐾(∙,∙) : Kernel
Polynomial
𝐾 𝑥, 𝑦 = 𝑥 ∙ 𝑦 + 1 𝑝
Radial Basis Function
𝐾 𝑥, 𝑦 = exp
− 𝑥 − 𝑦 2
2𝜎2
Sigmoid Function
𝐾 𝑥, 𝑦 = tanh 𝜅𝑥 ⋅ 𝑦 − 𝛿

SVM Classification
Equalization
𝑦 = 𝑠𝑖𝑔𝑛 𝑓 𝐱
 𝑦 : Estimate to the classification
 𝑓 𝐱 = i∈S αi yiΦ xi ∙ Φ x + b = i∈S αi yi𝐾 xi, x + b
• {𝛼i} – Lagrange Multipliers
• 𝑆 – Set of indices for which 𝑥𝑖 is a support vector
• 𝐾 ∙,∙ - Kernel satisfying conditions of Mercer’s theorem
• 𝑏 – Affine offset
 Training set consists of
 𝐱𝐢 ∈ 𝐑𝐌
 𝐲i ∈ {−1,1}, i = 1, … , L

System Model
NN {∙} SVMM
𝑧−𝐷
𝑢 𝑛 ∈ {±1}
𝒚𝒏
𝑓(𝐱)
𝑥(𝑛)
 NN {∙} – Nonlinear system
 𝑥(𝑛) – Nonlinear system output
 𝑢(𝑛) – Training sequence
 𝑦𝑛 – Desired output (delayed version of training sequence)

Nonlinear Transmission System
PAM
nonlinear
channel
SVM equalizer
𝑧−1
𝑢 𝑛
𝑒(𝑛)
 𝑥(𝑛) – Nonlinear channel output
 𝑒(𝑛) – Additive Noise
 (𝑀 − 1) – Feed Forward Delay (No of past channel outputs utilized)
 𝑢(𝑛 − 𝐷) – Equalizer detection output (goal to mimic 𝑢(𝑛 − 𝐷))
+ 𝑧−1
𝑥 𝑛 𝑥 𝑛 − 1 𝑥 𝑛 − 𝑀 + 1
𝑥 𝑛
𝑢 𝑛 − 𝐷

System Structure and Parameters
𝑥 𝑛 = 𝑥 𝑛 − 0.9𝑥3 𝑛
𝑥 𝑛 = 𝑢 𝑛 + 0.5𝑢 𝑛 − 1
𝑒 𝑛 ~𝑁 0, 𝜎𝑒
2 → 𝑁(0,0.2)
SVM Parameters
• C = 5 (constraint)
• d = 3 (equalizer kernel order)
• M = 2 (equalizer dimension)
• Kernel = Polynomial

Simulation Results
Typical classification regions of an SVM
𝐷 = 0 𝐷 = 1 𝐷 = 2
 𝑥 𝑛 = 𝑥 𝑛 − 0.9𝑥3 𝑛
 𝑥 𝑛 = 𝑢 𝑛 + 0.5𝑢 𝑛 − 1
 𝑒 𝑛 ~𝑁 0, 𝜎𝑒
2 → 𝑁(0,0.2)
 C = 5 (constraint)
 d = 3 (equalizer kernel order)
 M = 2 (equalizer dimension)
 Kernel = Polynomial

Results – Decision Boundaries
Colored Noise
Correlated
Noise – SVM
 𝐶𝑜𝑟𝑟𝑀𝑎𝑡 = 𝜎𝑒
2 1 𝜌
𝜌 1
 𝜌 = 0.48
 𝑀 = 2, 𝐷 = 0,𝑑 = 3
 𝑥 𝑛 = 𝑥 𝑛 + 0.1𝑥2
+ 0.05𝑥3
 𝑥 𝑛 = 0.5𝑢 𝑛 + 𝑢(𝑛 − 1)
 𝜎𝑒
2
= 0.2
Correlated Noise –
Optimum
Ref: Chen et al

Results – BER
Colored Noise Vs AWGN

Results – BER
For different values of D=0, 1, 2

Decision Boundaries and SNR
Polynomial Kernel
 K 𝐱, 𝐳 = 𝐱T
𝐳 + 1
d
 d = polynomial order
 All polynomials up to degree d
 For our simulation, d = 3
 𝐱T
𝐳 + 1
d
= O n computation
 Feature space might be non −
unique

RBF Kernel
 K 𝐱, 𝐳 = exp −γ 𝐱 − 𝐳 2
2
 Infinite dimensional space
 Parameter = γ
 As γ increases, the model
overfits
 As 𝛾 decreases, the model
underfits
 For our simulation, 𝛾 = 1

Sigmoid Kernel
 K 𝐱, 𝐳 = tanh k𝐱T
𝐳 − δ
 k = slope
 δ = intercept
 For our simulation,
• k = 10, δ = 10
 Sigmoidal kernels can be
thought of multi-layer
perceptron

Results – BER
For different SVM Kernels

Offline Training
Generalization over different channels
 𝑥𝑡𝑟𝑎𝑖𝑛 𝑛 = 𝑥 𝑛 − 0.9𝑥3
𝑛
 𝑥𝑡𝑟𝑎𝑖𝑛 𝑛 = 𝑢 𝑛 + 𝟎. 𝟗𝑢 𝑛 − 1
 𝑥𝑡𝑒𝑠𝑡 𝑛 = 𝑥 𝑛 − 0.9𝑥3
𝑛
 𝑥𝑡𝑒𝑠𝑡 𝑛 = 𝑢 𝑛 + 𝟎. 𝟓𝑢 𝑛 − 1
 𝑥𝑡𝑟𝑎𝑖𝑛 𝑛 = 𝑥 𝑛 − 𝟎. 𝟓𝑥3 𝑛
 𝑥𝑡𝑟𝑎𝑖𝑛 𝑛 = 𝑢 𝑛 + 0.6𝑢 𝑛 − 1
 𝑥𝑡𝑒𝑠𝑡 𝑛 = 𝑥 𝑛 − 𝟎. 𝟑𝑥3 𝑛
 𝑥𝑡𝑒𝑠𝑡 𝑛 = 𝑢 𝑛 + 0.6𝑢 𝑛 − 1

Offline Training
Generalization over different SNRs
 Training SNRs = 1: 20 dB
 Testing SNRs = 1: 20 dB
 Does not generalize well
over different SNR values
and multiple channels

SVM-Bank for Different SNR signals
SVM (SNR1)
SVM (SNR2)
SVM (SNR𝑁)
Noise
Variance
Estimator
NN {∙}
𝑢 𝑛 ∈ {±1} 𝑥(𝑛)
𝑢(𝑛)

Results – BER
For Bank of SVM

Summary
We looked at SVM as a Dual Lagrangian Optimization Problem
and how it fits in non-linear equalization problem.
We developed a non-linear channel communication system and
applied SVM equalizer.
For different values of Detector Delay (D) and SVM kernels, we
found different BER performance of the SVM equalizer.
For Unknown SNR, the SVM equalizer does not generalize well
to unknown channel and unknown SNR.
To solve the issue of SNR, we proposed a bank of SVM with SVM
models trained with different SNR values. After receiving the
signal, noise variance estimator block will select the desired
SVM model for equalization.

Support Vector Machine Techniques for Nonlinear Equalization

More Related Content

Support Vector Machine Techniques for Nonlinear Equalization

Editor's Notes