Lecture 18: Error Detection and Correction

John M Pauly

November 27, 2021

Error Protection: Detection and Correction
I Communication channels are subject to noise.

I Noise distorts analog signals.

I Noise can cause digital signals to be received as different values.
I Bits can be flipped
I Points in a signal constellation can be shifted.

I Changes in a digital signal are called errors.

Effects of Noise
Noise causes the constellation to blur out,

I QAM-4 I QAM-16
<latexit sha1_base64="ImRtkJYBliFBh5sSnnaqnjfIPAU=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE1GPBi94q2A9oQ9lsJ+3SzSbd3Qgl9E948aCIV/+ON/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj25nfekKleSwfzSRBP6IDyUPOqLFSO+syKsj9tFeuuFV3DrJKvJxUIEe9V/7q9mOWRigNE1Trjucmxs+oMpwJnJa6qcaEshEdYMdSSSPUfja/d0rOrNInYaxsSUPm6u+JjEZaT6LAdkbUDPWyNxP/8zqpCW/8jMskNSjZYlGYCmJiMnue9LlCZsTEEsoUt7cSNqSKMmMjKtkQvOWXV0nzoupdVb2Hy0rNzeMowgmcwjl4cA01uIM6NICBgGd4hTdn7Lw4787HorXg5DPH8AfO5w+pkY+r</latexit> <latexit sha1_base64="ImRtkJYBliFBh5sSnnaqnjfIPAU=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE1GPBi94q2A9oQ9lsJ+3SzSbd3Qgl9E948aCIV/+ON/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRU8epYthgsYhVO6AaBZfYMNwIbCcKaRQIbAWj25nfekKleSwfzSRBP6IDyUPOqLFSO+syKsj9tFeuuFV3DrJKvJxUIEe9V/7q9mOWRigNE1Trjucmxs+oMpwJnJa6qcaEshEdYMdSSSPUfja/d0rOrNInYaxsSUPm6u+JjEZaT6LAdkbUDPWyNxP/8zqpCW/8jMskNSjZYlGYCmJiMnue9LlCZsTEEsoUt7cSNqSKMmMjKtkQvOWXV0nzoupdVb2Hy0rNzeMowgmcwjl4cA01uIM6NICBgGd4hTdn7Lw4787HorXg5DPH8AfO5w+pkY+r</latexit>

R R
<latexit sha1_base64="aDzRWEBZk+mfsOepPjcYCAN20YI=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE1GPBi8cq9gPaUDbbSbt0s0l3N0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hodua3nlBpHstHM0nQj+hA8pAzaqzUzrqMCvIw7ZUrbtWdg6wSLycVyFHvlb+6/ZilEUrDBNW647mJ8TOqDGcCp6VuqjGhbEQH2LFU0gi1n83vnZIzq/RJGCtb0pC5+nsio5HWkyiwnRE1Q73szcT/vE5qwhs/4zJJDUq2WBSmgpiYzJ4nfa6QGTGxhDLF7a2EDamizNiISjYEb/nlVdK8qHpXVe/+slJz8ziKcAKncA4eXEMN7qAODWAg4Ble4c0ZOy/Ou/OxaC04+cwx/IHz+QO3Po+0</latexit> <latexit sha1_base64="aDzRWEBZk+mfsOepPjcYCAN20YI=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lE1GPBi8cq9gPaUDbbSbt0s0l3N0IJ/RNePCji1b/jzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hodua3nlBpHstHM0nQj+hA8pAzaqzUzrqMCvIw7ZUrbtWdg6wSLycVyFHvlb+6/ZilEUrDBNW647mJ8TOqDGcCp6VuqjGhbEQH2LFU0gi1n83vnZIzq/RJGCtb0pC5+nsio5HWkyiwnRE1Q73szcT/vE5qwhs/4zJJDUq2WBSmgpiYzJ4nfa6QGTGxhDLF7a2EDamizNiISjYEb/nlVdK8qHpXVe/+slJz8ziKcAKncA4eXEMN7qAODWAg4Ble4c0ZOy/Ou/OxaC04+cwx/IHz+QO3Po+0</latexit>

I With enough noise we will get decoding errors

I We’d like to be able to detect, and perhaps correct, these errors
I To do this, we’ll add some redundant information
Communication Systems
Recall the communication system block diagram:

* Source Channel
Source Encrypt Modulator
Encoder Encoder

Channel Noise

**
Source Channel
Sink Decrypt Demodulator
Decoder Decoder

We have concentrated on the modulator/demodulator blocks.

Error protection involves channel encoder and decoder.
Error Control Classification
The error control problem can be classified in several ways.
I Type of errors: how much clustering — random, burst, catastrophic

I Type of modulator output: digital (“hard”) vs. analog (“soft”)

I Type of error control coding: detection vs. correction

I Type of codes: block vs. convolutional

I Type of decoder: algebraic vs. probabilistic

The first two classifications are used to select a coding scheme according to
the last three classifications.
Types of Error Potection
I Error detection
Goal: avoid accepting faulty data.
Lost data may be unfortunate; wrong data may be disastrous.
Solution: checksums are included in messages (packets, frames, sectors).
If any part of the message is altered, then the checksum is not valid
(with high probability).

I (Forward) error correction (FEC or ECC).

Use redundancy in encoded message to estimate from the received data
(senseword) what message was actually sent.
Optimal estimate is message that is most probable given what is
received (MAP, maximum a posteriori). The best estimate is typically
the message that is “closest” to the senseword.
Block Codes
I k-digit blocks of information digits are encoded into n-digit codewords
(n ≥ k) by adding p = n − k redundant check digits.
data data

Encoder Encoder
(general) (systematic)

codeword data checks

I There is no memory betwen blocks. The encoding of each data block is

independent of past and future blocks.
I An encoding in which the information digits appear unchanged in the
codewords is called systematic.
I Adding parity bits is systematic
I 8b/10b is not systematic (each 8 bit sequence is mapped onto one or two
10 bit sequences)
Convolutional Codes
I For convolutional codes, a filter continuously adds additional bits to the
bit stream
I Each n-bit codeword block depends on the current codeword and on the
previous m codewords. m is the memory order
I Here is the simplest convolutional code.
cj
<latexit sha1_base64="O2Gh8MGOAyTtYhyX7hEoGSqERPA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEtMeCF48V7Qe0oWy2k3btZhN2N0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4Zua3n1BpHssHM0nQj+hQ8pAzaqx0z/qP/XLFrbpzkFXi5aQCORr98ldvELM0QmmYoFp3PTcxfkaV4UzgtNRLNSaUjekQu5ZKGqH2s/mpU3JmlQEJY2VLGjJXf09kNNJ6EgW2M6JmpJe9mfif101NWPMzLpPUoGSLRWEqiInJ7G8y4AqZERNLKFPc3krYiCrKjE2nZEPwll9eJa2LqndV9e4uK/VaHkcRTuAUzsGDa6jDLTSgCQyG8Ayv8OYI58V5dz4WrQUnnzmGP3A+fwBBDI3A</latexit>

mi
<latexit sha1_base64="hdimLzbrDbulcQHXiTCFKHo6zyM=">AAAB6nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKaI4BLx4jmgckS5idzCZDZmaXmV4hhHyCFw+KePWLvPk3TpI9aLSgoajqprsrSqWw6PtfXmFtfWNzq7hd2tnd2z8oHx61bJIZxpsskYnpRNRyKTRvokDJO6nhVEWSt6PxzdxvP3JjRaIfcJLyUNGhFrFgFJ10r/qiX674VX8B8pcEOalAjka//NkbJCxTXCOT1Npu4KcYTqlBwSSflXqZ5SllYzrkXUc1VdyG08WpM3LmlAGJE+NKI1moPyemVFk7UZHrVBRHdtWbi/953QzjWjgVOs2Qa7ZcFGeSYELmf5OBMJyhnDhCmRHuVsJG1FCGLp2SCyFYffkvaV1Ug6tqcHdZqdfyOIpwAqdwDgFcQx1uoQFNYDCEJ3iBV096z96b975sLXj5zDH8gvfxDU7Ejck=</latexit> <latexit sha1_base64="Puzy2oRXMCpNM0CvXrZJp5aOVKo=">AAAB7nicbVDLSgNBEOz1GeMr6tHLYBC8GHZFNMeAF48RzAOSJcxOOsmQmdllZlYISz7CiwdFvPo93vwbJ8keNLGgoajqprsrSgQ31ve/vbX1jc2t7cJOcXdv/+CwdHTcNHGqGTZYLGLdjqhBwRU2LLcC24lGKiOBrWh8N/NbT6gNj9WjnSQYSjpUfMAZtU5qyV7GL4Npr1T2K/4cZJUEOSlDjnqv9NXtxyyVqCwT1JhO4Cc2zKi2nAmcFrupwYSyMR1ix1FFJZowm587JedO6ZNBrF0pS+bq74mMSmMmMnKdktqRWfZm4n9eJ7WDaphxlaQWFVssGqSC2JjMfid9rpFZMXGEMs3drYSNqKbMuoSKLoRg+eVV0ryqBDeV4OG6XKvmcRTgFM7gAgK4hRrcQx0awGAMz/AKb17ivXjv3seidc3LZ07gD7zPH+/Ij0c=</latexit>

mi 1
D
<latexit sha1_base64="o/ix+BeaHmAXSaA/TPGT7hNgKd0=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSJ4KomI9ljQg8cW7Ae0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O2vrG5tb24Wd4u7e/sFh6ei4peNUMWyyWMSqE1CNgktsGm4EdhKFNAoEtoPx7cxvP6HSPJYPZpKgH9Gh5CFn1Fipcdcvld2KOwdZJV5OypCj3i999QYxSyOUhgmqdddzE+NnVBnOBE6LvVRjQtmYDrFrqaQRaj+bHzol51YZkDBWtqQhc/X3REYjrSdRYDsjakZ62ZuJ/3nd1IRVP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbIo2BG/55VXSuqx41xWvcVWuVfM4CnAKZ3ABHtxADe6hDk1ggPAMr/DmPDovzrvzsWhdc/KZE/gD5/MHll+MxA==</latexit>

+
<latexit sha1_base64="IVCoeUnizH8bjRcQ8fwrKirEvJw=">AAAB6HicbVBNS8NAEJ34WetX1aOXxSIIQklEtMeCF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dtbWNza3tgs7xd29/YPD0tFxS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfju5nffkKleSwfzCRBP6JDyUPOqLFS47JfKrsVdw6ySryclCFHvV/66g1ilkYoDRNU667nJsbPqDKcCZwWe6nGhLIxHWLXUkkj1H42P3RKzq0yIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmrfsZlkhqUbLEoTAUxMZl9TQZcITNiYgllittbCRtRRZmx2RRtCN7yy6ukdVXxbipe47pcq+ZxFOAUzuACPLiFGtxDHZrAAOEZXuHNeXRenHfnY9G65uQzJ/AHzucPcHuMqw==</latexit>

cj
<latexit sha1_base64="e6dWeErO+v/a25kqjcDUn6s8Su0=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBiyUR0R4LXjxWsB/QhrLZTtq1m03Y3Qgl9Ed48aCIV3+PN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFSm/Wzxwtv2i9X3Ko7B1klXk4qkKPRL3/1BjFLI5SGCap113MT42dUGc4ETku9VGNC2ZgOsWuppBFqP5ufOyVnVhmQMFa2pCFz9fdERiOtJ1FgOyNqRnrZm4n/ed3UhDU/4zJJDUq2WBSmgpiYzH4nA66QGTGxhDLF7a2EjaiizNiESjYEb/nlVdK6rHrXVe/+qlKv5XEU4QRO4Rw8uIE63EEDmsBgDM/wCm9O4rw4787HorXg5DPH8AfO5w/h648+</latexit>

I This generates two output bits for every input bit, the current data bit,
and the XOR with the previous data bit.
I For this rate 1/2 convolutional code, m = 1 and n = 2. This is used in
GSM.
I Convolutional codes are widely used in digital communications. They
don’t need a fixed block size. Examples are Viterbi codes and trellis
codes. These are covered in EE279.
Block Codes: Simple Parity-Check Codes
Append one check bit to data bits so that all codewords have the same
overall parity — either even or odd.
Even-parity codewords are defined by a single parity-check equation:
c1 ⊕ c2 ⊕ · · · ⊕ cn = (c1 + c2 + · · · + cn ) mod 2 = 0 ,
where ⊕ denotes the exclusive-or operation.
If we XOR cn to both sides of the above equation, we obtain an encoding
equation:
cn = c1 ⊕ c2 ⊕ · · · ⊕ cn−1 .
This shows how to compute the check bit cn from the data bits
c1 , . . . , cn−1 .
Any single bit error (or any odd number of errors) can be detected.

Any bit ci can bePconsidered to be the check bit because it can be computed from the other
n − 1 bits: ci = j6=i cj .
Polynomial Division
I We can consider parity as the remainder after polynomial division.
I For data bits c1 , · · · , c7 the parity bit is the remainder after dividing

c1 x6 + c2 x5 + c3 x4 + c4 x3 + c5 x2 + cx6 + c7

by x + 1.
I Example: c1 , · · · , c7 = 1010001 divided by 11
Encoding 110001R1 Decoding 1100010
11 1010001 11 10100011 Parity
Bit
11 11
110001 1100011
11 11
000001 0000011
11 11
1 Remainder 0 Remainder

Addition and subtraction are without carry (XOR)

Cyclic Redundancy Check (CRC)
I If we use a longer divisor polynomial we can detect which bit is in error
for a much longer bit stream
I For an optimal polynomial of order n we can
I Detect any single or double bit error in 2n − 1 bits
I With a factor 1 + x, detect any error in an odd number of bits
I Lots of different polynomials are possible. Two very common ones are
I CRC-16-IBM : x16 + x15 + x2 + 1
I CRC-CCITT : x16 + x12 + x5 + 1

I Other CRC codes from CRC-3 to CRC-64

I Can be applied to any length block less than 2n − 1
I Used in Bluetooth, USB, GSM, X.25, ACARS (ACARS-16-ARINC)
CRC Implementation
I Polynomial division can be implemented using linear feedback shift
registers. This does what we do with long division, clocking in one new
bit each cycle.
CRC-16:

CRC-CCITT:

I Remainder is left in the shift register at the end of the calculation,

appended to the transmitted bits
I On reception, the division of the data with the appended CRC remainder
will produce 0 if there are no errors
I If there is a single error, the CRC remainder identifies which bit is in error
Error Correction: Simple Product Codes
Arrange data bits in a two-dimensional array. Append parity check bits at
the end of each row and column.

k1 info bits row checks

n = (k1 + 1)(k2 + 1)
k2 info bits
k = k1 k2
n − k = k1 + k2 + 1
column checks

Single error causes failure of one row equation and one column equation.
Incorrect bit is located at the intersection of the bad row and bad column.
Double errors can be detected — two rows or two columns (or both) have
the wrong parity — but cannot be corrected.
Some triple errors cause miscorrection. Which?
Error Correction: Hamming Codes
Simple product codes are simple but inefficient:
I a failed parity-check equation locates row or column of error
I however, a satisfied equation gives little information

An “efficient” equation gives one bit of information about the error

location. It “looks” at half the codeword bits and is “independent” of other
equations.
Basic idea:
I We will send a block of size n = 2m − 1
I We’ll use m bits for parity, and the rest for data
I Each parity bit corresponds to half of the block
I A single error and its location can be identified by which parity equations
fail. This pattern is called the error syndrome
Hamming Codes
To determine the parity equations,
I Index each output bit from one to 2n − 1, and represent it in binary
I Any index that has a single bit is used for parity (i.e. 00100)
I All other indexed outputs are sequentially filled with input bits
I The parity bit is computed for all indexes that have the same index bit
set (i.e all the odd samples for the first parity bit)

Example: 2n − 1 = 7
I Parity bits are 001, 010, and 100
I Data bits are the rest, 011, 101, 110, and 111
I We’ll have 3 parity bits, 4 data bits, and seven bits total
I This is called a (7,4) code
Hamming Codes
I The following table defines a (7, 4) Hamming parity-check code.

c1 c2 c3 c4 c5 c6 c7
p1 p2 d1 p4 d2 d3 d4 

p1 1 0 1 0 1 0 1 

p2 0 1 1 0 0 1 1 3 parity-check equations


p4 0 0 0 1 1 1 1


I Each column is the binary representation of that index

I Each row defines one parity bit equation
I The 1’s indicate which codeword bits affect which parity-check
equations.
I We can double the block size and only need one more parity bit, for a
(15,11) code. This gets very efficient as n gets large.
Hamming Codes: Parity-Check Equations and Matrix
The following three equations are satisfied by all (and only) valid codewords:

c1 ⊕ c3 ⊕ c5 ⊕ c7 = 0
c2 ⊕ c3 ⊕ c6 ⊕ c7 = 0
c4 ⊕ c5 ⊕ c6 ⊕ c7 = 0

The check equations can be described by a parity-check matrix:

 
1 0 1 0 1 0 1
H= 0 1 1
 0 0 1 1
0 0 0 1 1 1 1
Valid codewords are characterized the equation:
 
c1
 .. 
H  .  = 0 3×1
c7

In other words, a 7-tuple c is a codeword if and only if HcT = 0 .

Hamming Codes: Encoding Equations
Each of the parity codeword bits c1 , c2 , c4 appears in only one equation.
The parity bits c1 , c2 , c4 are computed from the data bits, c3 , c5 , c6 , c7 .

c1 = c3 ⊕ c5 ⊕ c7
c2 = c3 ⊕ c6 ⊕ c7
c4 = c5 ⊕ c6 ⊕ c7

These linear encoder equations can be written as a vector-matrix product.

 
1 1 0
1 0 1
[ c1 c2 c4 ] = [ c3 c5 c6 c7 ] P = [ c3 c5 c6 c7 ] 
0 1 1 .


1 1 1
Hamming Code Error Detection and Correction
The parity equations are
c1 ⊕ c3 ⊕ c5 ⊕ c7 = 0
c2 ⊕ c3 ⊕ c6 ⊕ c7 = 0
c4 ⊕ c5 ⊕ c6 ⊕ c7 = 0
where c1 = p1 , c2 = p2 , and c4 = p4 are parity bits.
Parity bit error
I Each parity bit occurs in just one equation, so only that equation will
have a parity error
I A single parity equation error is an error in that parity bit
Data bit error
I Each data bit occurs in multiple equations, each of which will have a
parity error
I The data bit error is the bit shared by those multiple equations
Either way, a single bit error can be detected and corrected.
Hamming Code Error Detection and Correction
For larger blocks, we just add more parity bits.
For example, the parity-check matrix for (15,11) is
 
1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 
H=  0 0 0 1 1 1 1 0 0 0

0 1 1 1 1 
0 0 0 0 0 0 0 1 1 1 1 1 1 1 1

I There are now four parity equations, with parity bits p1 , p2 , p4 , and p8
I The data bits appear in 2, 3, or 4 equations.
I A single parity equation error is an error in the parity bit
I Multiple parity equation errors is an error in the bit shared by these
equations

All single bit errors can be detected and corrected.

Error Detection Conclusion
I A single overall parity-check equation detects single errors.
I Hamming codes use m equations to correct one error in 2m − 1 bits.
I Most useful when the error rate is low. ECC RAM uses Hamming codes
I More sophisticated algorithms needed when multiple errors likely in a block
I Extensions
I We can use nonbinary equations if we create symbols from sequences of
bits. E.g., four bits can represent 0, 1, . . . , 15.
I Nonbinary check equations can use more advanced arithmetic, such as
mod 16.
I We can add other types of equations, c1 + 2c2 + 3c3 + · · · and
c1 + αc2 + α2 c3 + · · · to be able to detect and correct multiple bit errors.
Next Classes
Wednesday : Spread Spectrum, Radar, GPS, CDMA
Friday : Questions on projects
Following Friday : Projects due