LDPC Optimization

The Role of Specialization in
LDPC Codes
Jeremy Thorpe
Pizza Meeting Talk
2/12/03
Talk Overview
LDPC Codes
Message Passing Decoding
Analysis of Message Passing Decoding
(Density Evolution)
Approximations to Density Evolution
Design of LDPC Codes using D.E.

The Channel Coding Strategy
Encoder chooses the m
th

codeword in codebook C
and transmits it across the
channel
Decoder observes the
channel output y and
generates m based on the
knowledge of the codebook
C and the channel
statistics.
Decoder
Encoder
Channel
k
} 1 , 0 { 'e m
n
Y e y
n
X C c e x
k
} 1 , 0 { e m
Linear Codes
A linear code C (over a finite field) can be
defined in terms of either a generator matrix
or parity-check matrix.
Generator matrix G (kn)

Parity-check matrix H (n-kn)

} {mG = C
} ' : { 0 cH c = = C
LDPC Codes
LDPC Codes -- linear codes defined in terms
of H.
H has a small average number of non-zero
elements per row
1 0 1 1 0 0 1 0 0 1
0 1 0 0 1 1 1 0 1 0
0 1 1 1 0 1 0 0 1 1
1 1 0 0 0 0 1 1 0 1
0 0 0 0 1 0 0 1 1 0
= H
Graph Representation of LDPC
Codes
H is represented by a
bipartite graph.
There is an edge from
v to c if and only if:

A codeword is an
assignment of v's s.t.:

c x
c v
v
=
0
|
.

.

.

.

.

.

Variable nodes
Check nodes
0 ) , ( = c v H
Regular (,) LDPC codes
Every variable node
has degree , every
check node has degree
.
Ensemble is defined by
matching left edge
"stubs" with right edge
"stubs via a random
permutation
.

.

.

.

.

.

Variable nodes
Check nodes

Message-Passing Decoding of
LDPC Codes
Message Passing (or Belief Propagation)
decoding is a low-complexity algorithm
which approximately answers the question
what is the most likely x given y?
MP recursively defines messages m
v,c
(i)
and
m
c,v
(i)
from each node variable node v to each
adjacent check node c, for iteration i=0,1,...
Two Types of Messages...
Liklihood Ratio

For y
1
,...y
n
independent
conditionally on x:
Probability Difference

For x
1
,...x
n

independent:
) 0 | (
) 1 | (
,
=
=
=
x y p
x y p
y x
) | 0 ( ) | 1 (
,
y x p y x p
y x
= = =
[
=
i
y x
y x i
n
,
,
1

[
=
i
y x y
i
x
i i
, ,

...Related by the Biliniear
Transform
Definition:

Properties:
x
x
x B
+
=
1
1
) (
y x
y x
y x p y x p
y p
y p y x p y p y x p
y p
x y p x y p
x y p x y p
x y p x y p
x y p
x y p
B B
,
,
) | 1 ( ) | 0 (
) ( 2
) ( ) | 1 ( 2 ) ( ) | 0 ( 2
) ( 2
) 1 | ( ) 0 | (
) 1 | ( ) 0 | (
) 1 | ( ) 0 | (
)
) 0 | (
) 1 | (
( ) (
=
= = =
= =
=
= =
=
= + =
= =
=
=
=
=
y x y x
y x y x
B
B
x x B B
, ,
, ,
) (
) (
)) ( (

=
=
=
Message Domains
Likelihood Ratio
Log Likelihood Ratio Log Prob. Difference
Probability Difference
) 1 | (
) 0 | (
=
=
x y P
x y P
) | 1 ( ) | 0 ( y x P y x P = =
) ( B
) ( ' B
Variable to Check Messages
On any iteration i, the
message from v to c is:

It is computed like:
) (
) 1 (
' |
, ' ,
) (
,

=
[
=
i
c v c
v c y x
i
c v
m B m
v v
.

.

.

.

.

.

v
c
) 0 | (
) 1 | (
=
=
x e P
x e P
Check to Variable Messages
On any iteration, the
message from c to v is:

It is computed like:

Assumption: Incoming
messages are indep.

) (
) (
' |
, '
) (
,
i
v c v
c v
i
v c
m B m
[
=
=
.

.

.

.

.

.

v
c ) | 1 ( ) | 0 ( e v P e v P = =
Decision Rule
After sufficiently many iterations, return the
likelihood ratio:
<
=

[
otherwise , 1
0 ) ( if , 0
) 1 (
|
, ,
i
v c
v c y x
m B
x
v v
Theorem about MP Algorithm

If the algorithm stops after r
iterations, then the algorithm
returns the maximum a
posteriori probability estimate
of x
v
given y within radius r of
v.
However, the variables within
a radius r of v must be
dependent only by the
equations within radius r of v,
v
r
.
.
.

.
.
.

.
.
.

Analysis of Message Passing
Decoding (Density Evolution)
in Density Evolution we keep track of
message densities, rather than the densities
themselves.
At each iteration, we average over all of the
edges which are connected by a permutation.
We assume that the all-zeros codeword was
transmitted (which requires that the channel
be symmetric).
D.E. Update Rule
The update rule for Density Evolution is
defined in the additive domain of each type
of node.
Whereas in B.P, we add (log) messages:

In D.E, we convolve message densities:
=
=
v c v
i
c v
i
v c
m B m
' |
) (
, '
) (
,
) ( '
[
=
=
v c v
i
c v
i
v c
m D B m D
' |
) (
, '
) (
,
)) ( ( '
*
) (
Familiar Example:
If one die has density function given by:

The density function for the sum of two dice
is given by the convolution:

1 3 6 5 4 2
2 4 7 6 5 3 8 10 12 11 9
D.E. Threshold
Fixing the channel message densities, the
message densities will either "converge" to
minus infinity, or they won't.
For the gaussian channel, the smallest
(infimum) SNR for which the densities
converge is called the density evolution
threshold.
Regular (,) LDPC codes
Every variable node has degree , every
check node has degree .
Best rate 1/2 code is (3,6), with threshold
1.09 dB.
This code had been invented by 1962 by
Robert Gallager.
D.E. Simulation of (3,6) codes
Set SNR to 1.12 dB
(.03 above threshold)
Watch fraction of
"erroneous messages"
from check to variable.
(note that this fraction
does not characterize
the distribution fully)
Improvement vs. current error
fraction for Regular (3,6)
Improvement per
iteration is plotted
against current error
fraction.
Note there is a single
bottleneck which took
most of the decoding
iterations.
Irregular (, ) LDPC codes
a fraction
i
of variable
nodes have degree i.
i

of check nodes have
degree i.
Edges are connected by
a single random
permutation.
Nodes have become
specialized.
.

.

.

.

.

.

Variable nodes
Check nodes

3
m
D.E. Simulation of Irregular
Codes (Maximum degree 10)
Set SNR to 0.42 dB
(~.03 above threshold)
Watch fraction of
erroneous check to
variable messages.
This Code was
designed by
Richardson et. al.
Comparison of Regular and
Irregular codes
Notice that the
Irregular graph is much
flatter.
Note: Capacity
achieving LDPC codes
for the erasure channel
were designed by
making this line
exactly flat.
Multi-edge-type construction
Edges of a particular
"color" are connected
through a permutation.
Edges become
specialized. Each edge
type has a different
message distribution
each iteration.
D.E. of MET codes.
For Multi-edge-type
codes, Density
evolution tracks the
density of each type of
message separately.
Comparison was made
to real decoding, next
slide (courtesy of Ken
Andrews).
MET D.E. vs. decoder
simulation
Regular, Irregular, MET
comparison
Multi-edge-type LDPC
codes improve
gradually through most
of the decoding.
MET codes have a
threshold below the
more "complex"
irregular codes.
Design of Codes using D.E.
Density evolution provides a moderately fast
way to evaluate single- and multi- edge type
degree distributions (hypothesis-testing).
Much faster approximations to Density
Evolution can easily be put into an outer loop
which performs function minimization.
Semi-Analytic techniques exist as well.
Review
Iterative Message Passing can be used to
decode LDPC codes.
Density Evolution can be used to predict the
performance of MP for infinitely large codes.
More sophisticated classes of codes can be
designed to close the gap to the Shannon
limit.
Beyond MET Codes
(future work)
Traditional LDPC codes are designed in two stages:
design of the degree distribution and design of the
specific graph.
Using very fast and accurate approximations to
density evolution, we can evaluate the effect of
placing or removing a single edge in the graph.
Using an evolutionary algorithm like Simulated
Annealing, we can optimize the graph directly,
changing the degree profile as needed.

LDPC Optimization

Uploaded by

Copyright:

Available Formats

LDPC Optimization

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

LDPC Optimization

Uploaded by

Copyright:

Available Formats

The Role of Specialization in

Theorem about MP Algorithm

You might also like