Turbo Codes

Emilia Käsper
ekasper[at]tcs.hut.fi

Introduction
This paper is an introductory tutorial on turbo codes, a new technique of error-
correction coding developed in the 1990s. The reader is expected to be familiar
with the basic concepts of channel coding, although we briefly and informally
review the most important terms.
The paper starts with a short overview of channel coding and the reader is re-
minded the concept of convolutional encoding. Bottlenecks of the traditional ap-
proach are described and the motivation behind turbo codes is explained. After
examining the turbo codes design more in detail, reasons behind their efficiency
are brought out. Finally, a real-world example of the turbo code used in the third
generation Universal Mobile Telecommunications System (UMTS) is presented.
The paper is mainly based on two excellent tutorials by Valenti and Sun [6], and
Barbulescu and Pietrobon [2]. The scope of this paper does not cover implementation-
specific issues such as decoder architecture, modulation techniques and the like.
For a follow-up on these topics, the interested reader is referred to Part II of the
second tutorial [3].

Channel Coding
The task of channel coding is to encode the information sent over a communica-
tion channel in such a way that in the presence of channel noise, errors can be
detected and/or corrected. We distinguish between two coding methods:

• Backward error correction (BEC) requires only error detection: if an er-

ror is detected, the sender is requested to retransmit the message. While this
method is simple and sets lower requirements on the code’s error-correcting
Turbo Codes 2

Output y1,i

Input
xi−1 xi−2 xi−3
xi

Output y2,i

Figure 1: A convolutional encoder

properties, it on the other hand requires duplex communication and causes

undesirable delays in transmission.

• Forward error correction (FEC) requires that the decoder should also be
capable of correcting a certain number of errors, i.e. it should be capable of
locating the positions where the errors occurred. Since FEC codes require
only simplex communication, they are especially attractive in wireless com-
munication systems, helping to improve the energy efficiency of the system.
In the rest of this paper we deal with binary FEC codes only.

Next, we briefly recall the concept of conventional convolutional codes. Con-

volutional codes differ from block codes in the sense that they do not break the
message stream into fixed-size blocks. Instead, redundancy is added continuously
to the whole stream. The encoder keeps M previous input bits in memory. Each
output bit of the encoder then depends on the current input bit as well as the M
stored bits. Figure 1 depicts a sample convolutional encoder. The encoder pro-
duces two output bits per every input bit, defined by the equations

y1,i = xi + xi−1 + xi−3 ,

y2,i = xi + xi−2 + xi−3 .

For this encoder, M = 3, since the ith bits of output depend on input bit i, as well
as three previous bits i − 1, i − 2, i − 3. The encoder is nonsystematic, since the
input bits do not appear explicitly in its output.
An important parameter of a channel code is the code rate. If the input size (or
message size) of the encoder is k bits and the output size (the code word size) is n
bits, then the ratio nk is called the code rate r. Since our sample convolutional en-
coder produces two output bits for every input bit, its rate is 12 . Code rate expresses
Turbo Codes 3

the amount of redundancy in the code—the lower the rate, the more redundant the
code.
Finally, the Hamming weight or simply the weight of a code word is the number
of non-zero symbols in the code word. In the case of binary codes, dealt with in
this paper, the weight of a code word is the number of ones in the word.

A Need for Better Codes

Designing a channel code is always a tradeoff between energy efficiency and band-
width efficiency. Codes with lower rate (i.e. bigger redundancy) can usually cor-
rect more errors. If more errors can be corrected, the communication system can
operate with a lower transmit power, transmit over longer distances, tolerate more
interference, use smaller antennas and transmit at a higher data rate. These proper-
ties make the code energy efficient. On the other hand, low-rate codes have a large
overhead and are hence more heavy on bandwidth consumption. Also, decoding
complexity grows exponentially with code length, and long (low-rate) codes set
high computational requirements to conventional decoders. According to Viterbi,
this is the central problem of channel coding: encoding is easy but decoding is
hard [7].
For every combination of bandwidth (W ), channel type, signal power (S) and re-
ceived noise power (N ), there is a theoretical upper limit on the data transmission
rate R, for which error-free data transmission is possible. This limit is called chan-
nel capacity or also Shannon capacity (after Claude Shannon, who introduced the
notion in 1948). For additive white Gaussian noise channels, the formula is
 
S
R < W log2 1+ [bits/second].
N

In practical settings, there is of course no such thing as an ideal error-free channel.

Instead, error-free data transmission is interpreted in a way that the bit error prob-
ability can be brought to an arbitrarily small constant. The bit error probability, or
bit error rate (BER) used in benchmarking is often chosen to be 10−5 or 10−6 .
Now, if the transmission rate, the bandwidth and the noise power are fixed, we get
a lower bound on the amount of energy that must be expended to convey one bit
of information. Hence, Shannon capacity sets a limit to the energy efficiency of a
code.
Although Shannon developed his theory already in the 1940s, several decades
later the code designs were unable to come close to the theoretical bound. Even
Turbo Codes 4

in the beginning of the 1990s, the gap between this theoretical bound and practi-
cal implementations was still at best about 3dB. This means that practical codes
required about twice as much energy as the theoretical predicted minimum.1
Hence, new codes were sought that would allow for easier decoding. One way
of making the task of the decoder easier is using a code with mostly high-weight
code words. High-weight code words, i.e. code words containing more ones and
less zeros, can be distinguished more easily.
Another strategy involves combining simple codes in a parallel fashion, so that
each part of the code can be decoded separately with less complex decoders and
each decoder can gain from information exchange with others. This is called the
divide-and-conquer strategy.
Keeping these design methods in mind, we are now ready to introduce the concept
of turbo codes.

Turbo Codes: Encoding with Interleaving

The first turbo code, based on convolutional encoding, was introduced in 1993 by
Berrou et al. [4]. Since then, several schemes have been proposed and the term
“turbo codes” has been generalized to cover block codes as well as convolutional
codes. Simply put,
a turbo code is formed from the parallel concatenation of two codes separated by
an interleaver.
The generic design of a turbo code is depicted in Figure 2. Although the general
concept allows for free choice of the encoders and the interleaver, most designs
follow the ideas presented in [4]:

• The two encoders used are normally identical;

• The code is in a systematic form, i.e. the input bits also occur in the output
(see Figure 2);

• The interleaver reads the bits in a pseudo-random order.

The choice of the interleaver is a crucial part in the turbo code design . The task
of the interleaver is to “scramble” bits in a (pseudo-)random, albeit predetermined
1
A decibel is a relative measure. If E is the actual energy and Eref is the theoretical lower
bound, then the relative energy increase in decibels is 10 log10 EE
ref
. Since log10 2 ≈ 0.3, a
twofold relative energy increase equals 3dB.
Turbo Codes 5

Input xi Systematic output xi

Output I
Encoder I
Interleaver
Output II
Encoder II

Figure 2: The generic turbo encoder

fashion. This serves two purposes. Firstly, if the input to the second encoder is in-
terleaved, its output is usually quite different from the output of the first encoder.
This means that even if one of the output code words has low weight, the other
usually does not, and there is a smaller chance of producing an output with very
low weight. Higher weight, as we saw above, is beneficial for the performance of
the decoder. Secondly, since the code is a parallel concatenation of two codes, the
divide-and-conquer strategy can be employed for decoding. If the input to the sec-
ond decoder is scrambled, also its output will be different, or “uncorrelated” from
the output of the first encoder. This means that the corresponding two decoders
will gain more from information exchange.
We now briefly review some interleaver design ideas, stressing that the list is by no
means complete. The first three designs are illustrated in Figure 3 with a sample
input size of 15 bits.

1. A “row-column” interleaver: data is written row-wise and read column-

wise. While very simple, it also provides little randomness.

2. A “helical” interleaver: data is written row-wise and read diagonally.

3. An “odd-even” interleaver: first, the bits are left uninterleaved and en-
coded, but only the odd-positioned coded bits are stored. Then, the bits are
scrambled and encoded, but now only the even-positioned coded bits are
stored. Odd-even encoders can be used, when the second encoder produces
one output bit per one input bit.

4. A pseudo-random interleaver defined by a pseudo-random number gen-

erator or a look-up table.
Turbo Codes 6

Input
x1 x2 x3 x4 x5
x6 x7 x8 x9 x10
x11 x12 x13 x14 x15

Row-column interleaver output

x1 x6 x11 x2 x7 x12 x3 x8 x13 x4 x9 x14 x5 x10 x15

Helical interleaver output

x11 x7 x3 x14 x10 x1 x12 x8 x4 x15 x6 x2 x13 x9 x5

Odd-even interleaver output

Encoder output without interleaving
x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15
y1 - y3 - y5 - y7 - y9 - y11 - y13 - y15
Encoder output with row-column interleaving
x1 x6 x11 x2 x7 x12 x3 x8 x13 x4 x9 x14 x5 x10 x15
- z6 - z2 - z12 - z8 - z4 - z14 - z10 -
Final output of the encoder
y1 z6 y3 z2 y z12 y7 z8 y9 z4 y11 z14 y13 z10 y15

Figure 3: Interleaver designs

Turbo Codes 7

There is no such thing as a universally best interleaver. For short block sizes, the
odd-even interleaver has been found to outperform the pseudo-random interleaver,
and vice versa. The choice of the interleaver has a key part in the success of the
code and the best choice is dependent on the code design. For further reading,
several articles on interleaver design can be found for example at [1].

Turbo Codes: Some Notes on Decoding

In the traditional decoding approach, the demodulator makes a “hard” decision
of the received symbol, and passes to the error control decoder a discrete value,
either a 0 or a 1. The disadvantage of this approach is that while the value of some
bits is determined with greater certainty than that of others, the decoder cannot
make use of this information.
A soft-in-soft-out (SISO) decoder receives as input a “soft” (i.e. real) value of
the signal. The decoder then outputs for each data bit an estimate expressing the
probability that the transmitted data bit was equal to one. In the case of turbo
codes, there are two decoders for outputs from both encoders. Both decoders
provide estimates of the same set of data bits, albeit in a different order. If all
intermediate values in the decoding process are soft values, the decoders can gain
greatly from exchanging information, after appropriate reordering of values. In-
formation exchange can be iterated a number of times to enhance performance.
At each round, decoders re-evaluate their estimates, using information from the
other decoder, and only in the final stage will hard decisions be made, i.e. each bit
is assigned the value 1 or 0. Such decoders, although more difficult to implement,
are essential in the design of turbo codes.

Turbo Codes: Performance

We have seen that the conventional codes left a 3dB gap between theory and prac-
tice. After bringing out the arguments for the efficiency of turbo codes, one clearly
wants to ask: how efficient are they?
Already the first rate 13 code proposed in 1993 made a huge improvement: the gap
between Shannon’s limit and implementation practice was only 0.7dB, giving a
less than 1.2-fold overhead. (In the authors’ measurements, the allowed bit error
rate BER was 10−5 ). In [2], a thorough comparison between convolutional codes
and turbo codes is given. In practice, the code rate usually varies between 12 and
1
6
. Let the allowed bit error rate be 10−6 . For code rate 12 , the relative increase in
Turbo Codes 8

Systematic Output xi

Output y1,i
Input
xi xi−1 xi−2 xi−3

Interleaver
Output y2,i
x0i
x0i−1 x0i−2 x0i−3

Figure 4: The UMTS turbo encoder

energy consumption is then 4.80dB for convolutional codes, and 0.98dB for turbo
codes. For code rate 16 , the respective numbers are 4.28dB and -0.12dB2 . It can
also be noticed, that turbo codes gain significantly more from lowering the code
rate than conventional convolutional codes.

The UMTS Turbo Code

The UMTS turbo encoder closely follows the design ideas presented in the origi-
nal 1993 paper [4]. The starting building block of the encoder is the simple con-
volutional encoder depicted in Figure 1. This encoder is used twice, once without
interleaving and once with the use of an interleaver, exactly as described above.
In order to obtain a systematic code, desirable for better decoding, the following
modifications are made to the design. Firstly, a systematic output is added to the
encoder. Secondly, the second output from each of the two encoders is fed back
to the corresponding encoder’s input. The resulting turbo encoder, depicted in
Figure 4, is a rate 13 encoder, since for each input bit it produces one systematic
output bit and two parity bits. Details on the interleaver design can be found in
the corresponding specification [5].
2
Although the relative value is negative, it does not actually violate the Shannon’s limit. The
negative value is due to the fact that we allow for a small error, whereas Shannon’s capacity applies
for perfect error-free transmission.
Turbo Codes 9

As a comparison, the GSM system uses conventional convolutional encoding in

combination with block codes. The code rate varies with the type of input; in the
case of speech signal, it is 260
456
< 12 .

Conclusions
Turbo codes are a recent development in the field of forward-error-correction
channel coding. The codes make use of three simple ideas: parallel concatena-
tion of codes to allow simpler decoding; interleaving to provide better weight
distribution; and soft decoding to enhance decoder decisions and maximize the
gain from decoder interaction.
While earlier, conventional codes performed—in terms of energy efficiency or,
equivalently, channel capacity—at least twice as bad as the theoretical bound sug-
gested, turbo codes immediately achieved performance results in the near range of
the theoretically best values, giving a less than 1.2-fold overhead. Since the first
proposed design in 1993, research in the field of turbo codes has produced even
better results. Nowadays, turbo codes are used in many commercial applications,
including both third generation cellular systems UMTS and cdma2000.
Turbo Codes 10

References
[1] University of South Australia, Institute for Telecommunications Research,
Turbo coding research group. http://www.itr.unisa.edu.au/
~steven/turbo/.

[2] S.A. Barbulescu and S.S. Pietrobon. Turbo codes: A tutorial on a new class of
powerful error correction coding schemes. Part I: Code structures and inter-
leaver design. J. Elec. and Electron.Eng., Australia, 19:129–142, September
1999.

[3] S.A. Barbulescu and S.S. Pietrobon. Turbo codes: A tutorial on a new class of
powerful error correction coding schemes. Part II: Decoder design and perfor-
mance. J. Elec. and Electron.Eng., Australia, 19:143–152, September 1999.

[4] C. Berrou, A. Glavieux, and P. Thitimajshima. Near Shannon limit error-

correcting coding and decoding: Turbo codes. In Proceedings of the IEEE In-
ternational Conference on Communications, Geneva, Switzerland, May 2003.

[5] Third Generation Partnership Project(3GPP). Multiplexing and Channel Cod-

ing (FDD), March 2005. TS 25.212 Version 6.4.0.

[6] M. C. Valenti and J. Sun. Turbo codes. In F. Dowla, editor, Handbook of RF

and Wireless Technologies, pages 375–400. Newnes, 2004.

[7] A. M. Viterbi. Shannon theory, concatenated codes and turbo coding. http:
//occs.donvblack.com/viterbi/index.htm, 1998.