The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) The Art of Error Correcting Coding The Art of brror Correcting Coding Robert H. Morelos-Zaragoza SONY Computer Science Laboratories, Inc. JAPAN JOHN WILEY & SONS, LTD Copyright Q 2002 by John Wiley & Sons, Ltd Baffins Lane, Chichester, West Sussex, P019 lUD, England National 01243 779777 International (+44) 1243 779777 e-mail (for orders and customerservice enquiries): cs-books@viley.co.uk Visit our HomePage on http://www.wileyeurope.com or http://www.wiley.com All Rights Reserved. No part of this publicationmay be reproduced, stored in a retrieval system, or transmitted, in anyform or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under t h e t e r mof the CopyrightDesigns and PatentsAct 1988 or under the termsof a licence issued by the Copyright Licensing Agency, 90 Tottenham CourtRoad, London, W1P9HE, UK, without the permission in writing of the Publisher, with the exception of any material supplied specifically for the purposeof being entered and executed on a computer system, for exdusiveuse by the purchaserof the publication. Neither the author(s) nor JohnWiley & Sons, Ltd accept any responsibility or liability for loss or damage occasioned to any person or property through using the material, instructions, methodsor ideas contained herein, or acting o r refraining from acting as a result of such use. The author(s) andPublisher expressly disclaim all implied warranties, including merchantabilityof fitness for any particular purpose. Therewill be no duty on the author(s) or Publisher to correct any errorsor defects in thesoftware. Designations used by companies to distinguish their products areoften claimed as trademarks. In all instances in capital or capital letters. where John Wiley & Sons, Ltd is aware of a daim, the product names appearinitial Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. Other W h y Editorial Ofices John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, USA WILEY-VCH VerlagGmbH Pappelallee 3, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia John Wiley & Sons (Canada) Ltd, 22 Worcester Road Rexdale, Ontario, M9W 1L1, Canada John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 British Library Cataloguing inPublication Data A catalogue record for this bookis available from theBritish Library ISBN 0471 49581 6 Produced fromLaTeX files supplied by the author. Printed and bound inGreat Britain by Antony Rowe Ltd, Chippenham, Wiltshire. This book is printed on acid-free paper responsibly manufactured fromsustainable forestry, in which at least two trees are planted foreach one used for paper production. Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi .................................... ... x111 The ECC website 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Error correcting coding: Basic concepts . . . . . . . . . . . . . . . . . . . . 1.1.1 Block codesandconvolutionalcodes ................. 1.1.2 Hamming distance, Hamming spheres and error correcting capability l .2 Linear block codes ............................... 1.2.1 Generatorandparity-checkmatrices . . . . . . . . . . . . . . . . . . 1.2.2 Theweight is the distance . . . . . . . . . . . . . . . . . . . . . . . 1.3 Encoding and decoding of linear block codes . . . . . . . . . . . . . . . . . 1.3.1 Encoding with G and H . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Standard array decoding . . . . . . . . . . . . . . . . . . . . . . . . 1.3.3 Hamming spheres, decoding regions and the standard array . . . . . . 1.4 Weight distribution and error performance . . . . . . . . . . . . . . . . . . . 1.4.1 Weight distribution and undetected error probability over a BSC . . . 1.4.2 Performance bounds over BSC AWGN and fading channels . . . . . 1.5 General structure of a hard-decision decoderof linear codes . . . . . . . . . . 1 3 3 4 5 6 6 7 7 8 11 12 12 13 19 2 Hamming.GolayandReed-Mullercodes ..................... 2.1 Hammingcodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Encoding and decodingprocedures . . . . . . . . . . . . . . . . . . 2.2 ThebinaryGolaycode ............................. 2.2.1 Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Arithmetic decoding of the extended (24. 12. 8) Golay code . . . . . . 2.3 Binary Reed-Muller codes ........................... 2.3.1 Booleanpolynomialsand RM codes . . . . . . . . . . . . . . . . . . 2.3.2 Finite geometries and majority-logic decoding . . . . . . . . . . . . 23 23 24 . 25 25 26 26 27 27 28 3 Binarycycliccodesand BCH codes . . . . . . . . . . . . . . . . . . . . . . . . 33 3.1 Binary cyclic codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3 . l .1 Generator and parity-check polynomials . . . . . . . . . . . . . . . . 33 vi CONTENTS ....................... 3.1.2Thegeneratorpolynomial 3.1.3 Encoding and decoding of binary cyclic codes . . . . . . . . . . . . . 3.1.4Theparity-checkpolynomial ...................... 3.1.5 Shortened cyclic codesand CRC codes . . . . . . . . . . . . . . . . 3.2 General decoding of cyclic codes . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 G F (2m) arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Binary BCH codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4Polynomialcodes ................................ 3.5 Decoding of binary BCH codes . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 General decoding algorithm for BCH codes . . . . . . . . . . . . . . 3.5.2 TheBerlekamp-Masseyalgorithm(BMA) . . . . . . . . . . . . . . . 3.5.3 PGZ decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.4EuclideanAlgorithm(EA) ....................... 3.5.5Chiensearch and error correction . . . . . . . . . . . . . . . . . . . 3.5.6 Errors-and-erasures decoding ..................... 3.6 Weight distribution and performance bounds . . . . . . . . . . . . . . . . . . 3.6.1 Errorperformanceevaluation ..................... 34 35 36 37 39 41 44 45 46 48 49 52 53 55 55 56 57 4 Non-binary BCH codes: Reed-Solomon codes . . . . . . . . . . . . . . . . . . . 4.1 RS codes as polynomialcodes . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 From binary BCH to RS codes . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Decoding RS codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1Remarks on decodingalgorithms . . . . . . . . . . . . . . . . . . . . 4.3.2 Errors-and-erasures decoding ..................... 4.4 Weight distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 63 63 64 69 69 73 5 Binaryconvolutionalcodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Basic structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1Recursivesystematicconvolutionalcodes ............... 5.1.2Freedistance .............................. 5.2Connections with blockcodes . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Zero-tail construction . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2Direct-truncationconstruction ..................... 5.2.3 Tail-biting construction . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.4 Weight distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Weight enumeration and performance bounds . . . . . . . . . . . . . . . . . 5.4 Decoding: Viterbi algorithm with Hamming metrics . . . . . . . . . . . . . . 5.4.1 Maximumlikelihooddecoding and metrics . . . . . . . . . . . . . . 5.4.2The Viterbi algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3 Implementation issues . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Puncturedconvolutionalcodes . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 Implementation issues related to punctured convolutional codes . . . 5.5.2 RCPC codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 75 80 81 81 81 82 82 83 84 86 87 88 90 96 99 100 6 Modifyingandcombiningcodes . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Modifying codes ................................ 6.1.1 Shortening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 103 103 CONTENTS vii 6.1.2 Extending ................................ 6.1.3 Puncturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1.4 Augmenting and expurgating . . . . . . . . . . . . . . . . . . . . . . 6.2Combiningcodes ................................ 6.2.1 Time-sharing of codes . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.2Direct-sums of codes . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.3Products of codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.4Concatenatedcodes ........................... 6.2.5Generalizedconcatenatedcodes .................... 105 105 106 108 108 109 111 117 119 7 Soft-decisiondecoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Binary transmission over AWGN channels . . . . . . . . . . . . . . . . . . . 7.2 Viterbi algorithm with Euclidean metric . . . . . . . . . . . . . . . . . . . . 7.3 Decoding binary linear block codes with a trellis . . . . . . . . . . . . . . . 7.4TheChasealgorithm .............................. 7.5Orderedstatisticsdecoding ........................... 7.6Generalized minimum distancedecoding . . . . . . . . . . . . . . . . . . . . 7.6.1 Sufficient conditionsforoptimality .................. 7.7 List decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Soft-output algorithms ............................. 7.8.1Soft-output Viterbi algorithm . . . . . . . . . . . . . . . . . . . . . . 7.8.2Maximum-a-posteriori (MAP) algorithm . . . . . . . . . . . . . . . 7.8.3 Log-MAP algorithm .......................... 7.8.4 Max-Log-MAP algorithm ....................... 7.8.5Soft-output OSD algorithm . . . . . . . . . . . . . . . . . . . . . . . 123 124 124 130 131 133 134 135 136 136 136 139 141 142 142 ............................. 145 8.1 Iterativedecoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 8.2Productcodes .................................. 149 8.2.1 Parallel concatenation:turbocodes . . . . . . . . . . . . . . . . . . 149 8.2.2 Serial concatenation .......................... 155 8.2.3Blockproductcodes .......................... 157 8.3 Low-density parity-checkcodes . . . . . . . . . . . . . . . . . . . . . . . . 161 8.3.1 Tanner graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 8.3.2 Iterative hard-decision decoding: The bit-flip algorithm . . . . . . . . 163 8.3.3Iterativeprobabilisticdecoding: belief propagation . . . . . . . . . . 164 8 Iterativelydecodablecodes 9 Combiningcodesanddigitalmodulation ..................... 171 9.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 9.1 .1 Examples of signal sets . . . . . . . . . . . . . . . . . . . . . . . . . 172 ........................... modulation Coded 9 .l .2 174 9.1.3 Distance considerations ........................ 175 9.2Trellis-codedmodulation(TCM) ........................ 176 9.2.1Setpartitioning and trellis mapping . . . . . . . . . . . . . . . . . . 176 9.2.2Maximum-likelihooddecoding ..................... 177 9.2.3Distanceconsiderations and errorperformance . . . . . . . . . . . . 177 9.2.4Pragmatic TCMand two-stagedecoding . . . . . . . . . . . . . . . . 178 ... v111 CONTENTS 9.3Multilevelcodedmodulation(MCM) ...................... 9.3.1Constructions and multi-stagedecoding . . . . . . . . . . . . . . . . 9.3.2 Unequal-error-protectionwith MCM . . . . . . . . . . . . . . . . . 9.4 Bit-interleaved codedmodulation(BICM) . . . . . . . . . . . . . . . . . . . 9.4.1Graymapping .............................. 9.4.2Metric generation: De-mapping . . . . . . . . . . . . . . . . . . . . 9.4.3 Interleaving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5Turbo trellis-coded modulation(TTCM) . . . . . . . . . . . . . . . . . . . . 9.5.1Pragmaticturbo TCM . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.2Turbo TCM with symbol interleaving . . . . . . . . . . . . . . . . . 9.5.3 Turbo TCM withbit interleaving . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 183 185 191 191 192 193 194 194 194 194 197 Appendix A Weight distributions of extended BCH codes . . . . . . . . . . . . . . 207 A.lLength8 ..................................... 207 A.2Length16 .................................... 207 A.3Length 32 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 A.4Length 64 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 A.5 Length128 ................................... 211 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Preface This bookis the result of hundreds of emails fromall over the world with questions on theory and applications of error correcting coding (ECC), from colleagues from both academia and industry. Most of the questions have been from engineersand computer scientists needing to select, implement or simulate aparticular coding scheme. The questions were sparked by an ECC web site that was initially set up at Imai Laboratoryat the Institute of Industrial Science, University of Tokyo, atthe beginning of 1995. The readerwill notice the absence of theorems and proofs in this text. The approach is to teach basic concepts by using simple examples. References to theoretical developments are made when needed. This bookis intended to be a reference guideto error correcting coding techniquesfor graduate students and professionals interested in learning the basic techniques and applications ofECC. Computer programs that implement the basic encoding and decoding algorithmsof practical coding schemes are available on a companionweb site at: http://the-art-of-ecc.com This site is referred to as the “ECC website” throughout the text. This bookis unique in that it introduces the basic concepts of error correcting codes using simpleillustrative examples. Computer programs,written in C language andavailable on the ECC web site, help to further illustrate the implementation of basic encoding and decoding algorithmsof important coding schemes, such as convolutional codes, Hamming codes, BCH codes, Reed-Solomon codes and turbo codes, andtheir application in coded modulation systems. Thematerial focuses on basic algorithms for analyzing and implementing ECC. There is a rich theory of ECC that will be touched upon,by referring to the appropriate material. There are many good books dealing with the theory of ECC, e.g., references [LC], [MS],[PW], [Blah], [Bos], [Wic], just to cite a few. Readers may wish to consult them before, during or after going throughthe material in this book. Each chapter describes, using simple and easy to follow numerical examples,the basic concepts of a particular coding or decoding scheme, rather than going into the detail of the theory behind it. Basic analysis tools are given throughout the book, to help in the assessment of the error performance of a particular ECC scheme, for some basic channel models. With the companion website, this makes the book unique. The bookdeals with the art of error correcting coding, in the sense that it addresses the need for selecting, implementing and simulating algorithms for encoding and decoding of codes for error correction and detection. The bookis organized as follows. In the firstchapter, the basic concepts of error correction, and coding and decoding techniques,are introduced. Chapter 2 deals with important and simple to understand families of codes, such asthe Hamming, Golay and Reed-Muller codes. In Chapter 3, cyclic codes and the important family of BCH codes are described. Finite field arithmetic is introduced and basic decoding algorithms, such as X THE ART OF ERROR CORRECTING CODING Berlekamp-Massey, Euclidean and PGZ, are described and easy to follow examples given to understand their operation. Chapter 4 deals with Reed-Solomon codes and errors-anderasures decoding.A comprehensive treatment of the available algorithms is given, along with examples of their operation. In Chapter 5, binary convolutional codes are introduced. Focus in this chapter is on the understanding of the basic structure of these codes, along with a basic explanation of the Viterbi algorithm with Hamming metrics. Important implementation issues are discussed. In Chapter 6, several techniques for modifying a single code or combining several codesare given and illustrated by simple examples. Chapter 7 deals with softdecision decoding algorithms, some of which havenot yet received attention in the literature, such as a soft-output ordered statistics decoding algorithm. Moreover, Chapter 8 presents a unique treatment of turbo codes, both parallel concatenated and serial concatenated, and block product codes, froma coding theoretical perspective. In the same chapter, low-density parity check codes are examined. For all these classes of codes, basic decoding algorithms are described and simple examples are given. Finally, Chapter 9 deals with powerful techniques that combine error correcting coding with digital modulation, and several clever decoding techniques are described. A comprehensive bibliography is included, for readers who wish to learn more about the beautiful theory that makes it all work. It is my hope that this book will become a valuable and indispensable tool for both students and practitioners of this interesting, exciting and never-ending area of information theory. I would like to thank the following persons for influencing this work. Professor Francisco GarciaUgalde, Universidad Nacional Autonoma de MCxico, for introducing me to the exciting world of error correcting codes. Parts of this book are based onmyBachelor’s thesis under his direction. Professor Edward Bertram, University of Hawaii, for teaching me the basics of abstract algebra. Professor David Mufioz, Instituto Technologico y de Estudios Superiores de Monterrey, MCxico, for his kindness and support. Professors Tadao Kasami, Hiroshima City University, Toru Fujiwara, University of Osaka, and Hideki Imai, University of Tokyo, for supporting my stays as a visiting academic researcher in Japan. Dan Luthi and Advait Mogre, LSI LogicCorporation, for many stimulating discussions and the opportunity to experience the process of putting ideas into silicon. Professor Marc P.C. Fossorier, Universityof Hawaii, forhis help. My colleague Dr. Misa MihaljeviC,SonyComputer Science Laboratories, for pointing out connections between decoding and cryptanalysis. I would also like to thank wholeheartedly Dr. Mario Tokoro, President of Sony Computer Science Laboratories, and Professor Ryuji Kohno, Yokohama National University, for making it possible for me to have a fine environment in which to write this book. In particular, I want to express my eternal gratitude to Professor Shu Lin, now at the University of California at Davis, who supported me when I was a graduate student in Hawaii, and encouraged me to continue my research in this fascinating topic. Last but not least, I want to thank the many students and colleagues who throughout the years listened to my lectures in Mexico, Japan and the U.S.A. I dedicate this book to the memory of Richard W. Hamming, Claude Shannon and Gustave Solomon, three extraordinary gentlemen who greatly impacted the way people live and work today. Robert H. Morelos-Zaragoza Tokyo, Japan, April 2002. Foreword In modem digital communication andstorage systems design, information theoryis becoming increasingly important. Thebest example of this is the appearance and quick adoption of turbo and block product codes in many practical satellite and wireless communication systems. I am pleased to recommend this new book, authoredby Dr. Robert Morelos-Zaragoza,to those who are interested in error correcting codes or have to apply them. Thebook introduces key concepts of error correcting coding (ECC)in a mannerthat is easy to understand. Thematerial is logically well structured and presented using simple illustrative examples. This, together with the computer programs available on the web site, is a novel approach to teaching the basic techniques used in the design and application of error correcting codes. One of the best features of the book is that it provides anatural introduction to the principles and decoding techniques of turbo codes, LDPC codes,and product codes, froman algebraic channel coding perspective. In this context, turbo codes are viewed as punctured product codes. With simple examples, the underlying ideas and structures used in the construction and iterative decoding of product codesare presented in an unparalleled manner. The detailed treatment of various algebraic decoding techniques for the correction of errors and erasures using Reed-Solomon codesis also worth a mention. On the applications of ECC in combined channel coding and digital modulation, or coded modulation, the author does a goodjob in introducing the basic principles that are used in the construction of several important classes of coded modulation systems. I believe that practitioner engineers and computer scientists will find this book to be botha good learningtool and a valuablereference. The companion ECCweb site is a uniquefeature that is not found anywhere else. Incidentally, this web site was born in my laboratory at the University of Tokyo in 1995, whereDr. Morelos-Zaragoza worked until June of 1997 and did a very good job as my associate researcher, writing many high-quality papers. Robert is polite, modest and hard-working,and is always friendly. In summary, I strongly recommend The Art of Error Correcting Coding as an excellent introductory and reference book onthe principles and applications of error correcting codes. Professor Hideki Imai The University of Tokyo Tokyo, Japan, April 2002 The ECC web site The Art of Error Correcting Coding and its companion web site, the ECC web site, offer a new and unique approachto teaching the fundamental conceptsof error correcting coding. The book explains in a clear and easyto understand manner, withsimple illustrative examples, basic error correcting coding (ECC) techniques alongwith their decoding algorithms. Many practical ECC techniquesare covered in the book, such as cyclic codes, BCH codes, RS codes, convolutional codes, turbo codes, product codes and low-density parity-check (LDPC) codes. In parallel with the tutorial treatment of the book, a companion web site provides readerswith computer programs that implement decoding algorithms of the most important families of error correcting codes. This is a novel hands-on method of teaching the art of ECC. Moreover, many of the computer programs on the web site can be used to simulate advanced techniques of error correcting coding, such as belief propagation (BP) decoding of LDPC codes and iterative decoding of product codes based on maximum-a-posteriori (MAP) decoding of the component codes. Also, programs are available on the ECC web site to simulate combinationsof codes and digital modulation formats,and include trellis-coded modulation (TCM),multilevel coded modulation (MCM),bit-interleaved CM (BICM) and turboTCM (T-TCM). Highlights of The Art of Error Correcting Codingare the following: * Comprehensive treatment of decoding procedures BCH for and RS codes * General decoders forRS codes Arbitrary shortening, arbitrary starting zero, errors-and-erasures decoding using the Berlekamp-Massey, Euclidean or Peterson-Gorenstein-Zierler(PGZ) algorithms. * Techniques for modifying and combining linear codes Direct-sum, product, concatenationand generalized concatenation (GC). Reliability-based decodingof linear block codes Generalized minimumdistance (GMD) decoding algorithm for RS codes, Chase algorithms and ordered-statistics decoding (OSD) algorithm for binary linear block codes.Viterbi decoding using atrellis. * Soft-input soft-output(SISO) decoding of binary linear block codes THE ART OF ERROR CORRECTING CODING xiv ThisincludesSO-Chase and SO-OSD algorithms, whichhavenotreceived attention in other textbooks. Optimal MAP decoding and its approximations. Belief propagation decoding. Combined coding and modulation TCM, multilevel coding and unequal error protection (UEP), BICM and turbo TCM. A companion web site for the book The Art of Error Correcting Coding has been set up and is located permanentlyat the following URL address: the-art-of-ecc.com The ECCweb site contains computer programs written in C language to implement algorithms for encoding and decoding of important families of error correcting codes. The web site is maintainedby the author, to ensure that the domain name remains unchanged.An important advantageof having a companionweb site is that it allows the author to post update notes, new computer programsand simulation results relevant to the contents of the book. The computer programsin the ECG web site are organized in two ways:by topic and by function. In the topical organization of the programs, the logical structure of the book is closely followed, going from simple syndrome-based decoding of linear block codes to moreelaborate algebraic decodingover finite fields of BCH and Reed-Solomon codes, passing through Viterbi decoding of convolutional codesand decoding of combinations and constructions of codes, to iterative decoding of turbo and product codes, belief-propagation decoding of low-density parity-check codes and applications in coded modulationtechniques. The indexof programs by topic is summarized below. 1. Linear block codes - Computing the weight distribution - Bounds on error performance overAWGN,flat Rayleighfading andBSC channels 2. Hamming, Golay and Reed-Muller codes - Hamming codes and syndrome-based decoding - Binary Golay(23,12,7) code and look-up table decoding - Extended Golay(24,12,8) code and syndrome-based decoding 3. Binary cyclic codes andBCH codes - Error-trapping decodingof binary cyclic codes - Berlekamp-Massey, Euclidean and PGZ algorithms - Chien searchand error correction - Errors-and-erasures decoding - Error performance evaluation 4. Reed-Solomon codes - Berlekamp-Massey, Euclideanand PGZ algorithms THE ECC WEB SITE xv - Errors-and-erasures decoding - Weight distributions and error performance evaluation 5. Binary convolutional codes - Weight enumeration sequences and error performance evaluation - Viterbi algorithm with Hamming metrics - Decoding of punctured convolutional codes 6. Soft-decision decoding - Viterbi algorithm with Euclidean metrics - Decoding binarylinear block codeswith a trellis - Chase type-I1 algorithm - Ordered statistics decoding (OSD) - GMD decoding 7. Soft-input soft-output (SISO) algorithms - MAP and log-MAP - Max-log-MAP and SOVA - SO-OSD and SO-Chase 8. Iteratively decodable codes - Parallel concatenation (turbo codes) - Serial concatenation (product codes) - Block product codes - LDPC codes:iterative decoding (bit-flip and belief-propagation) 9. Coded modulation - Trellis coded modulation (TCM)and pragmatic TCM - Multilevel coded modulation (MCM) and unequal-error-protectionMCM - Bit-interleaved coded modulation (BICM) - Turbo TCM The functional organization of the programs in the ECC web site is intended for readers who already know exactly whatthey are looking for.In particular, this classification of the programs is followed with respect to the decoding algorithms. The organization of the programs accordingto their functionality is summarizedbelow. 1. Basic performance analysis tools - Bounds and approximations on the block and bit error rate - Data files with weight distributions of extended binaryBCH codes - Program to compute the weight distribution of a codegiven its generator matrix 2. Hard-decision decoding - Syndrome decodingof short linear block codes xvi THE ART OF ERROR CORRECTING CODING - Berlekamp-Massey algorithm-based decoding of RS and BCH codes - Euclidean algorithm-based decoding of RS and BCH codes - Peterson-Gorenstein-Zierler (PGZ) algorithm-based decoding of RS and BCH codes - Viterbi algorithm 3. Soft-decision decoding - Viterbi algorithm for convolutional and linear block codes - Chase type-I1 algorithm - GMD algorithms - OSD algorithm 4. SISO decoding - Optimal MAP algorithm (also known as the BCJR’ algorithm) and log-MAP algorithm - Max-log-MAP andsoft-output Viterbi (SOVA)algorithms - SO Chase and SO-OSD algorithms for decoding linear block codes 5. Iterative decoding - Partial product (“parallel concatenation”) codes - Full product (“serial concatenation”) codes - Sum-product or BP algorithm for LDPC codes 6. Combination of ECC and modulation - MLD decoding of TCM and two-stage decodingof pragmatic TCM - Multistage decodingof multilevel codes - Applications of multilevel codes in unequal error protection - BICM - Turbo TCM BCJR stands for Bahl-Cocke-Jelinek-Ravivafter the authors The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) Index 2"-ary weight of an integer, 46 Algebraic geometry codes, 61 A-posteriori probability (APP), 122, 135 ARQ, 99 Array code, 1 13 Asymptotic coding gain, 173, 187 greater than, with block partitioning, 187 Augmented code, 104 Automatic repeatrequest, 99 AWGN channel, 15, 122 metric equivalence to changing sign, 122 reliability, 129 BCH Bound, 45 BCH code, 44 extended, 57 general decoder, 48 how to compute weight distribution, 57 performance AWGN channel, 57 specified by zeros, 45 BCH decoder Berlekamp-Massey algorithm, 47 Euclidean algorithm, 47 general, 48 Massey algorithm, 63 PGZ algorithm, 47 BCJR algorithm, 137 BEC channel, 55 Belief propagation decoding, 162 Berlekamp-Massey algorithm, 47,49-52 discrepancy, 49 errors-and-erasures errata evaluator, 68 errata locator, 68 modified discrepancy, 68 modified Forney algorithm, 68 modified Forney syndrome, 67 Bit error probability AWGN, 2 Block code concept, 3 Boolean function, 27 Bound BCH, 45 bit error probability, AWGN channel, 57 Chernoff, 19 Hamming, l I nonbinary case, 12 RS decoder, 72 union AWGN channel, 16 convolutional code, 84 multilevel modulation code for UEP, 186 Burst error correcting capability, 113 Chase algorithm, 129 soft output, 156 correction factor, 157 scaling factor, 157 Chien search, 55 Code self-dual, 7,29 Coded modulation bit-interleaved (BICM), 189 main idea, 2 MCM, 2,173, 180 multistage decoding, 182 parallel decoding, 185 unequal error protection, 183 Pragmatic TCM symbol transformation, 177 two-stage decoding, 176 TCM, 2, 173, 174 example modified state diagram, 176 MLD decoding, 175 turbo trellis(TTCM), 192 Coding gain, 2, 16 asymptotic, 173, 187 Complementary error function (erfc), 2 Concatenated code, 115 Concatenated coding, 2 Conjugate elements, 43 Constellation, 170 Construction X3, 109 Convolutional code, 2 block code obtained from, 79 complete weight enumerator sequence, 83 constraint length, 74 218 THE ART OF ERROR CORRECTING CODING defined, 3 direct-truncation, 80 finite-state machine, 73 generator sequences, 75 polynomial generator matrix, 77 recursive systematic (RSC code), 78 RSC code, 144 state diagram, 74 tail-biting, 80 union bound over BSC and AWGN channels, 84 weight distribution block codes from, 8 1-84 weight enumerating sequence, 82 zero-tail code, 79 Correlation discrepancy, 133 Coset, 104 decomposition, 104, 109, 118 leader, 10 representative, 118, 140 CRC code, 37 popular polynomials, 38 Cycle set, 43 Cyclic code defined, 33 encoding by division by B(x),36 extended, 57 general decoder, 39 MLS code, 37 RM code, 29 shortened, 37 syndrome decoding, error-trapping, 39 zeros of, 34 Cyclic shift, 33 Cyclotomic coset, 43 Decoding BCH codes general, 48 BCJR algorithm, 137 Belief propagation, 162 Berlekamp-Massey algorithm, 47,49-52 Chase algorithm, 129 soft output, 156 Chien search, 55 Depth, 88 Euclidean algorithm, 47,53-54 Forney algorithm for RS codes, 62 GMD algorithm, 132 Log-MAP algorithm, 139 look-up table, 10 MAP algorithm, 137 Massey algorithm, 63 Max-Log-MAP algorithm, 140 MLD, 86, 175 Modified Forney algorithm, 68 ordered statistics algorithm, 131 soft-output, 140 Parallel for multilevel codes, 185 PGZ algorithm, 47,52-53 SOVA algorithm, 134 Sudan algorithm, 67, 134 SW-SOVA algorithm, 136 two-stage, 115, 176 Viterbi algorithm, 85-94 with standard array, 8 Decomposable code, 107, 117 Direct-truncation code, 80 Discrepancy Berlelamp-Massey algorithm, 49 Disjunctive normal form, 27 Distance designed of BCH code, 44 free, 79 Hamming, 4 minimum Hamming, 4 minimum squared Euclidean, 173 squared Euclidean, 15 Dual code, 6 example, 7 of cyclic code, 37 of RM code, 28 Encoding non-systematic, 35 recursive with parity-check matrix, 36 systematic, 16, 35 with generator matrix, 8 with parity-check matrix, 8 Erasure, 55 value, 67 Erasure correction for binary linear codes, 55 Erasure locator polynomial, 67 Error positions, 47 values, 47 Error bursts, 2 Error correcting capability, 5 Error correcting code as subset, 4 defined, 3 minimum Hamming distance, 4 Error evaluator polynomial, 62 Error locator polynomial, 47 Error polynomial, 46 Error propagation, 186 Euclidean algorithm, 47, 53-54 polynomials same up toa constant, 66 Euclidean geometry (EG) code, 45 INDEX 219 Extended code. 103 Iterative bit-flip decoding, 161 Iterative decoding convergence, 152 Factoring polynomials in GF(2m) is hard, 55 Field, 40 Galois, 40 arithmetic, 41 element order, 43 representations, 41 Finding factors of z 2 m1,43 Finite geometry, 28 Flat Rayleigh fading channel bound, 18 model, 17 Fomey algorithm, 62 Fourier transform BCH decoding with, 67 Free distance, 79 + Gallager code, 160 Galois field, 40 Generalized concatenated code, 117 array codes, 113 Generator matrix, 6 Generator polynomial, 34 of BCH code,44 of RS code, 62 GMD decoding, 132 Golay code, 25 Greatest common divisor (GCD), 53 Hamming code, 23 shortened (71,64,3) code, 37 Hamming space defined, 4 distance, 4 sphere, 5 Hard-decision decoding, 16 general structure, 20 Incidence vector, 29 Inner code, 1 10 Interleaver, 152 block, 111 convolutional, 116 cyclic, 114 Ramsey, 111, 116, 152 random, 152 S-random, 152 Irreducible factors, 34 Irregular LDPC code record performance, 3 Iterative belief propagation algorithm message passing, 163 Iterative belief propagation decoding, 162 Key equation, 41 LDPC code, 159 error detection capability, 166 Likelihood, 85 Linear code as vector subspace, 6 Linear feedback shift-register (LFSR), 49 List decoding, 134 Log and antilog tables, 42 Log-likelihood metric, 122 Log-likelihood ratio (LLR), 145 extrinsic, 146 Log-MAP algorithm, 139 Low-density panty-check code, 159 MacWilliams identity, 56 MAP algorithm, 137 Massey algorithm, 63 Matrix generator and parity-check, 6 Vandermonde, 45 Max-Log-MAP algorithm, 140 Maximum-a-posteriori probability, 137 Maximum-distance-separable (MDS) code, 62 weight distribution, 71 Maximum-length sequence (MLS) code, 37 Meggit decoder, 39 Message passing, 163 Metric log-likelihood, 122 Metric normalization, 90 Minimal polynomial, 43 Minimum Hamming distance, 4 Minimum squared Euclidean distance, 173 MLD decoding, 175 defined, 15 Viterbi algorithm, 86 Modified Fomey syndrome, 67 Modified syndrome polynomial, 67 Modulation as mapping, 170 Monte Carlo integration, 19 MSED, 173 Multilevel coding, 180 Multilevel modulation code definition, 18 1 Multistage decoding, 182 Natural labeling, 174 Non-primitive BCH codes, 38 Non-systematic cyclic code, 35 220 THE ART OF ERROR CORRECTING CODING Nyquist bandwidth, 169 Q-function, 2 , Order of an element inG F ( 2 m ) 43 Ordered statistics decoding, 131 Orthogonal checks, 30 Outer code. 1IO RCPC codes, 98 Reed-Muller (RM) code, 27,45, 119 decoder for cyclic code, 32 majority-logic decoding, 31 number of minimum weight codewords, 29 Reed-Solomon (RS) code, 2 as polynomial code, 61 binary image of, 62 encoding as a polynomial evaluation, 61 generator polynomial, 62 weight distribution, 7 1 Reliability AWGN channel, 129 Repeat-and-accumulate code, 154 Repetition code example, 4, 5, 11 probability decoding error, 14 RS decoder bound bit error probability, 72 bound word error probability, 72 error evaluator polynomial, 62 errors-and-erasures, 67 direct solution, 71 errata evaluator, 68 errata locator, 68 modified discrepancy, 68 modified Forney algorithm, 68 modified Forney syndrome, 67 Forney algorithm, 62 Massey algorithm, 63 Parallel concatenated code, 147 Parallel decoding of multilevel codes, 185 Parity-check matrix, 6 of BCH code, 45 of cyclic code, 36 Parity node, 159 Parity sub-matrix, 7 Partition level, 117-1 18 Path memory, 87,88 Pearl’s algorithm, 162 Perfect code definition, 1 1 Permutation, 111 Permutation matrix, 147 PGZ algorithm, 47, 52-53 Polynomial associated with vector, 33 erasure locator, 67 errata locator, 68 error, 46 error evaluator, 62 error locator, 47 generator, 34 minimal, 43 modified syndrome, 67 parity-check, 36 primitive, 41 syndrome, 39 Polynomial code, 27,45 Primitive element, 41 polynomial, 41 Probability a-posteriori, 135 AWGN, Q-function, 2 bit error, 16 bit error, BPSK over AWGN, 2 correct decoding, 14 incorrect decoding, 14 undetected error, 13 Product code, 109 decoding, 1 15 Projective geometry (PG) code, 45 Punctured code, 103 Punctured convolutional codes, 94 Puncturing as shortening the dual code, 103 Self-dual code, 103 Set partitioning, 174 block for unequal error protection, 185 hybrid, 183 Shannon limit, 73 Shortened code, IO 1 additional correctable error patterns, 102 Shortening depth, 38 Signal point, 170 Sliding window SOVA algorithm, 136 Soft decision decoding, 15 Soft-output Chase algorithm, 156 Soft-output ordered statistics algorithm, 140 SOVA algorithm, 134 Spectral efficiency, 169 Squaring construction, 108 Standard array as look-up table, 10 construction, 9 decoding, 8 State diagram convolutional code, 74 INDEX for computing weight distribution, 81 for computing weight enumerating sequence, 82, 176 Subcode property, 119 Sudan algorithm, 134 Sum-product algorithm, 162 Supercode, 104 Syndrome as evaluation of zeros of code, 47 as vector, 9 circuit for computing, 48 Syndrome polynomial, 39 Syndrome trellis, 128 Systematic cyclic code, 35 Systematic encoding, 3 Tail-biting code, 80 Tanner graph, 159 Time-sharing code, 106 Trellis diagram, 76 Trellis structure example 3-level coded 8-PSK modulation, 181 of array codes, 113 of block and convolutional codes, 3 Ungerboeck mapping, 174 Turbocode, 106, 143, 147 as a punctured product code, 148 component RSC code, 144 Two-dimensional code, 110 Two-stage decoding, 115, 176 Two-step majority-logic decoding, 31 Unequal error protection multilevel modulation code, 183 Unequal error protection code, 99, 109, 118 example, I O Vandermonde matrix, 45 Variable node, 159 Viterbi algorithm, 86-94 ACS, 94 branch synchronization, 89 traceback, 92 traceback memory, 76 Viterbi decoder off-the-shelf, 177 Weight distribution convolutional codes, 81-84 defined, 12 extended BCH codes, 57 Weight enumerating sequence, 82 Zero-tail code, 79 The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) 1 Introduction The history of ECC started with the introduction of the Hamming codes [Ham], at about the sametime as the seminal work of Shannon [Sha]. Shortly after, Golaycodeswere invented [Gol]. These first classes of codes are optimal, in a senseto be definedin a subsequent section. Figure 1 shows the block diagram of a canonical digital communications/storage system. This is the famous Figure 1 in most books onthe theory of ECC. The information source and destination will include any source coding scheme matchedto the nature of the information. The ECC encodertakes as input the information symbols from the source and adds redundant symbols to it, so that most of the errors - introduced in the process of modulating a signal, transmitting it over anoisy medium and demodulatingit - can be corrected. ................................................ .._.._.._.._.... Figure 1 A canonical digital communications system. Usually, the channel is assumed to besuch that samples of an additive noise process are added to the modulated symbols (in their complex baseband representation). The noise samples are assumed to be independent from the source symbols. This model is relatively easy to track and includes additive white Gaussian noise (AWGN) channels, flat Rayleigh fading channels,and binary symmetric channels(BSc). At the receiver end, the ECC decoder utilizes the redundant symbols to correct channel errors. In the case of error detection, the ECC decoder canbe thought of as a re-encoderof the received messageand a checkthat the redundant symbols generatedare the same as those received. In classical ECC theory, the combination of modulation, noisy medium and demodulation was modeled as a discrete memoryless channel with input V and output F . An example of THE ART OF ERROR CORRECTING CODING 2 this is binary transmission over an AWGN channel, which is modeled as a binary symmetric channel (BSC) with a probability of channel error p - or transition probability - equal to the probability of a bit error forbinary signalling over an AWGN, P = Q ( E ) : where is the Gaussian Q-function‘, and Eb/No is the signal-to-noise ratio (SNR) per bit. This case is treated later in this chapter. In 1974, Massey [Mas31 suggested considering ECC and modulation as a single entity, known in modern literature as coded modulation. This approach provides a higher efficiency and coding gain2 than the serial concatenation of ECC and modulation, by joint design of codes and signal constellations. Several methods of combining coding and modulation are covered in this book, including: Trellis-coded modulation (TCM) [Ungl] and multilevel coded modulation (MCM) [IH]. In a coded modulation system, the (soft-decision) channel outputs are processed directly by the decoder. In contrast, in a classical ECC system, the hard decision bits from the demodulator are fed to a binary decoder. Codes can be combined in several ways. An example of serial concatenation (that is, concatenation in theclassical sense) is the following. For years, the most popular concatenated ECC scheme hasbeen the combination of an outer Reed-Solomon code, through intermediate interleaving, and an inner binary convolutional code. This scheme hasbeen used in numerous applications, ranging from space communications to digital broadcasting of high definition television. The basic idea is that the soft-decision decoder of the convolutional code produces bursts of errors that can be broken into smaller pieces by the deinterleaving process and handled effectively by the Reed-Solomon decoder. Reed-Solomon codes are non-binary codes that work with symbols composedof several bits, and can deal with multiple bursts of errors. Serial concatenation hasthe advantage that it requires two separate decoders, one for the inner code and one forthe outer code,instead of a single but very complex decoder for the overall code. Thisbookexamines these types of ECC systems. First, basic code constructions and their decoding algorithms, in the Hamming space (that is, dealing with bits), are presented. Subsequently, the second part of the book introduces soft-decision decoding algorithms for binary transmission, that work over the Euclidean space and achieve a reduction inthe required transmitted power per bit of at least 2 dB, compared with Hamming-space (harddecision) decoders. Several kinds of soft-decision decoders are considered, with attention given to their algorithmic aspects (how they work) instead of their theoretical aspects (why they work). Finally, combinations of codes and interleaving for iterative decoding and combined coding andmodulation are the topic of the last part of the book. Note that, in terms of the complementary error function, Q(z) = i e r f c ( z / f i ) . Coding gain is defined as the difference in signal-to-noise ratio between the coded system and an uncoded system with the same rate. INTRODUCTION 3 1.1 Error correcting coding: Basic concepts All error correcting codes are based on the same basic principle: Redundancy is added to information in order to correct any errors that may occur in the process of storage or transmission. In a basic (and practical) form, redundant symbolsare appended to information symbols to obtain a coded sequence or codeword. For the purpose of illustration, a codeword obtained by encoding with a block code is show in Figure 2. Such an encodingis said to be systematic. This meansthat the information symbols always appear in the first IC positions of a codeword. The remainingn - IC symbols in a codewordare some functionof the information symbols, and provide redundancy that can be used for error correctioddetection purposes. The set of all code sequences iscalled an error correcting code,and will be denoted by C. n symbols f information c- ksymbols I -K- > - redundancy1 n-ksymbols Figure 2 Systematic block encoding for error correction. l .I . I Block codes and convolutional codes According to the manner in which redundancy is addedto messages, ECC can be divided into two classes: block and convolutional. Both types of coding schemes have found practical applications. Historically, convolutional codes have been preferred, apparently because of the availability of the soft-decision Viterbi decoding algorithm and the belief for many years that block codes couldnot be efficiently decoded with soft-decisions. However, recent developments in the theory and designof soft-decision decoding algorithmsfor linear block codes have helped to dispel this belief. Moreover, the best ECC known to date (beginning of the twenty-first century) are block codes(irregular low-density parity-checkcodes). Block codes process the information on a block-by-block basis, treating each block of information bits independently from others. In other words, block coding is a memoryless operation, in the sense that codewords are independent from each other. In contrast, the output of a convolutional encoder depends not only on the current input information, but also on previous inputs or outputs, either on a block-by-blockor a bit-by-bit basis. For simplicity of exposition, we begin with a study of the structural properties of block codes. Many of these properties are common to both types of codes. It should be noted that block codes have in fact memory, when encoding is thought of as a bit-by-bit process and within a codeword.Most recently the difference between block and convolutional codes has becomeless and less well defined, specially after recent advancesin the understanding of the trellis structure of block codes and the tail-biting structure of some convolutional codes. Indeed, colleagues workingon convolutional codes sometimesrefer to block codesas “codes with time-varying trellis structure.” Similarly, researchers working with block codesmay consider convolutional codesas “codes with a regular trellis structure.” THE ART OF ERROR CORRECTING CODING 4 1.1.2 Hamming distance, Hamming spheres and error correcting capability Consider an error correcting code C with binary elements.As mentioned above,block codes are considered for simplicity of exposition. In order to achieve error correcting capabilities, not all the 2" possible binary vectorsof length n are allowed to be transmitted. Instead, C is a subset of the n-dimensional binary vector spaceV2 = (0, l}", such that its elements are as far apart as possible. In the binary space V,, distance is defined as the number of entries in which two vectors differ. Let el = ( x l , ~x l, , l , . . . ,Z I , ~ - ~and ) ZZ = ( 2 2 , 0 , 2 2 , 1 , . . . , x,,"-l) be two vectors in V2. Then the Hamming distance between 3 1 and 2 2 , denoted d H ( Z 1 , Z Z ) ,is defined as dH(517Z2)= I {i : Z l , i # m,i, 0 Li n} 1, (1.3) where IAl denotes the number of elements in (or the cardinality of) a set A . Given acode C , its minimumHammingdistance, d,i,, is defined as theminimum Hamming distance amongall possible distinct pairs of codewords in C, dmin = -Fin v1,vzEC {dH(al,fiz)lfil # V,}. (1.4) Throughout the book, (n,IC, d,in) is used to denote the parameters of a block code of length n, that encodes messagesof length IC bits and has a minimum Hamming distance dmin. The assumption is made that the size of the code is (Cl = 2 k . Example 1 The simplest error correcting code is a binary repetition code of length 3. It repeats each bit three times, so that a '0' is encoded onto the vector (000) and a '1' onto the vector (1 1 1). Since the two codewords differ in all three positions, the Hamming distance between them is equal to three. Figure 3 is a pictorial representation of this code. The 3dimensional binary space correspondsto the set of 23 = 8 vertices of the three-dimensional unit-volume cube. The Hamming distance between codewords (000) and (1 11) equals the number of edges in a path between them. This is equivalent to the number of coordinates that one needs to change to convert (000) into (1 1l), or vice versa. Thus d H ((000), (111))= 3. Since there are only two codewords in this case, dmin = 3. Figure 3 A (3,1,3) repetition code in a 3-dimensional binary vector space. The binary vector space V2 is also known as a Hamming space. Let V denote a codeword of an error correcting code C. A Hamming sphere &(V), of radius t and centered aroundV, is the set of vectors in V, at a distanceless than or equalto t from the center V , St(.) = { Z E V2ld,y(Z,U) 5 t }. (1.5) 5 INTRODUCTION Note that the size of (or the number of codewords in) &(V) is given by the following expression Example 2 Figure 4 shows the Hamming spheres of radius t = 1 around the codewords of the (3,1,3) binary repetition code. Figure 4 Hamming spheres of radius t = 1 around the codewords of the (3,1,3) binary repetition code. Note that the Hamming spheres forthis code are disjoint, that is, there is no vector in V2(or vertex in the unit-volume three-dimensional cube) that belongs to both S 1(000) and S1(111). As a result, if there is a changein any one position of a codewordV , then the resulting vector will still lie inside a Hamming sphere centered at V. This conceptis the basis of understanding and defining the error correcting capability of a codeC. The error correcting capability, t , of a code C is the largest radius of Hamming spheres around all the codewords V E C, such that for all different pairs Gi,Vj E C, the corresponding Hamming spheresare disjoint, i.e., &(G) t = _max (elSt(vi) n St(vj) = 0, vi # vj}. (1.7) Vi,Uj€C In terms of the minimum distanceof C, dmin, an equivalent and more common definition is t = l(dmin - 1)/21j (1.8) where 1x1 denotes the largest integer less than or equalto x. Note that in order to compute the minimum distancedmin of a block codeC, in accordance with Equation (1.4), a total of (at most) 2 k ( 2 k - 1) distances between distinct pairs of codewords are needed. Thisis practically impossible even for codesof relatively modest size, say, IC = 50. One of the advantages of linear block codes is that the computation of dmin requires one only to know the Hamming weight of all 2k - 1nonzero codewords. 1.2 Linear block codes As mentioned above, finding a good code meansfinding a subset of V2 with elements as far apart as possible. This isvery difficult. In addition, even when such aset is found, there is still the problem of how to assign codewords to information messages. THE ART OF ERROR CORRECTING CODING 6 Linear codes are vector subspaces of Vz. This means that encoding can be accomplished by matrix multiplications. In terms of digital circuitry, simple encoders can be built using exclusive OR’s, AND gates and D flip-flops. In this chapter, the binary vector spaceoperations of sum and multiplication are meant to be theoutput of exclusive-OR (modulo addition) 2 and AND gates, respectively. The tables of addition and multiplication for binary elements are: a 0 0 1 1 b 0 1 0 1 a+b 0 1 1 0 a.b 0 0 0 1 which correspond, as mentioned previously, to the outputs of a binary“X-OR’ logic gate and a binary “AND” logic gate, respectively. I .2.1 Generator and parity-check matrices Let C denote a binarylinear ( n ,k , dmin) code. Since C is a k-dimensional vector subspace, it has a basis, say {VO,V I , .. . , iik-l}, such that any codeword V E C can be represented as a linear combination of the elements in the basis: V = ZLG, (1.10) where Since C is a k-dimensional vector space in V2, there is an ( n - k)-dimensional dual space CT,generated by the rows of a matrix H , called theparity-check matrix, such thatGHT = 0, where HT denotes the transpose of H . In particular, note that for any codeword tj E C, UHT = 0. (1.12) Equation ( 1.12) is of fundamental importancein decoding of linear codes, as will be shown in section 1.3.2. A linear code CL that is generated by H is a binary linear (n,n - k , d k i n ) code, called the dual codeof C. 1.2.2 The weight is the distance As mentioned in section 1.1.2, a nice feature of linear codes is that computing the minimum distance of the code amounts to computing the minimum Hamming weight of its nonzero 7 INTRODUCTION codewords. In this section, this fact is shown. First, define the Hamming weight, wtH(2), of a vector5 E V, as the number of nonzero elementsin 2 . From the definition of the Hamming distance, it is easy to see that & H ( % ) = d H ( % : 0). For a binary linear code note that the distance d H ( V 1 ; Vij2) dH(e1 ‘u2; 8) = W t H ( v 1 e2). (1.13) c, + + + Finally, by linearity, V l Cij2E C. As a consequence, the minimum distance of C can be computed by finding the minimum Hamming weight amongthe 2k - 1 nonzero codewords. This is simpler than the brute force search among all pairs of codewords, although still a considerable task even for codes of modest size (or dimension k ) . 1.3 Encoding and decoding of linear block codes 1.3.1 Encoding with G and H Equation (1.10) gives an encoding rule for linear block codes that can be implemented in a straightforward way. If encoding is to be systematic, then the generator matrixG of a linear block (n:k : d m i n ) code C can be brought to a systematic form, Gsys, by elementary row operations and/or column permutations.Gsys is composed of two sub-matrices: The k-by-k identity matrix, denoted I k , and a k-by-(n - IC) parity sub-matrix P , such that where / Po,o P0,l ... &,n-k-1 \ (1.15) Since G H T = 0 , it follows that the systematic form,Hsys,of the parity-check matrixis Hsys x (P’-Iln-k 1. (1.16) Example 3 Consider a binary linear (4: 2: 2 ) code with generator matrix To bring G into systematic form, permute (exchange) the second and fourth columns and obtain Thus the parity-check sub-matrixis given by It is interesting to note that in this case, the relation P = PT holds3. From (1.16) it follows In this case, the code in question is referred to as a se[f-dualcode. See also Section 2.2.3. THE ART OF ERROR CORRECTING CODING 8 that the systematic formof the parity-check matrix is In the following let 2L = (uo; u1 . . . u k - 1 ) denote an information message to be encoded and V = (VO; v1 . . . ;~ " - 1 ) the corresponding codewordin C. If the parameters of C are such that k < ( n - k), or equivalently the code rate k / n < l / 2 , then encoding with the generator matrixis the most economical. Thecost considered here is in terms of binary operations. In such case V = iiGSys= ( U 7up); (1.17) where U p = 'L1P = ( w k w k + l ; . . . vn-l) represents the parity-check part of the codeword. However, if k > ( n - k ) , or k / n > 1/2, then alternative encoding withthe paritycheck matrix H requires less number of computations. In this case, wehave encoding based on Equation (1.12), (21; 'Up)HT = 0, such that the ( n - IC) parity-check positions 'uk w k + l . . . w,-1 are obtained as follows: W; = uop0,j + u1p1,; + ' . + uk-lPk-1,;; ' k I:j < n. (1.18) Stated in other terms, the systematic form of a parity-check matrix of a linear code has as entries of its rows the coefficients of the parity-check equations, from which the values of the redundant positions are obtained. This fact will be used when low-density parity-check codes are presented, in Section 8.3. Example 4 Consider the binary linear (4; 2; 2) code from Example 3. Let messages and codewords be denoted by U = (210;u1)and V = ( W O ; 211 w2 w 3 ) , respectively. From Equation (1.18), we have that The correspondence betweenthe 2' = 4 two-bit messagesand codewords is as follows: (00) l+ (0000) (01) e (0110) (10) H (1011) (11) H (1101) (1.19) 1.3.2 Standardarraydecoding In this section, a decoding procedure is presented that finds the closest codeword V to a received noisy word F = fj e. The error vector E E (0; l}" is produced by a BSC, as depicted in Figure 5. It is assumed that the crossover probability (or BSC parameter) p is such thatp < 1/2. A standard array [Sle] for a binary linear (n;I C ; &in) code C is a table of all possible received vectors F arranged in such a way that the closest codeword V to F can be read out. + INTRODUCTION 9 received sent Figure 5 A binary symmetric channel model. Table 1 The standard array of a binary linear block code. - S Go 0 V0 - el S1 e2 e2 - =0 =0 G2 ... el ... e1 + V 1 + 32 ... ... v1 Gk-l V2.b-1 V2k-1 el + e2 + '62k-1 + The standard array containsZ n P k rows and 2k 1 columns. The entries of the rightmost 2k columns of the array contain allthe vectors in V2 = ( 0 , l } n . In order to describe the decoding procedure, the concept of syndrome is needed. The syndrome of a word in V2 is defined from Equation (1.12) as S = THT, (1.20) where H is the parity-check matrix of C. That S is indeed a set of symptoms that indicate errors is shown as follows. Suppose that a codeword E C is transmitted over a BSC and received as F = V E . The syndromeof T is + g FHT = (V + e)fp = (gp, (1.21) where to obtain the last equality Equation (1.12) hasbeen used. Therefore, the computation of the syndrome canbe thought of as alinear transfonnation of an error vector. Standard array construction procedure 1. As the first row, in the positions corresponding the to 2k rightmost columns, enterall the codewords of C, beginning with the all-zero codeword in the leftmost position. In the position corresponding to the first column, enter the all-zero syndrome. Let j = 0. 2. Let j = j 1. Find the smallest Hamming weight word E j in V2, not in C , and not included in previous rows. The correspondingsyndrome Sj = ejHT is the first (rightmost) entry of the row. The 2k remaining entries in that row are obtained by adding E j to all the entries in the first row (the codewords of C). 3. Repeat the previous step until all vectors in V2 areincluded in the array. Equivalently, let j = j 1. If j < then repeat previous step, otherwise stop. + + THE ART OF ERROR CORRECTING CODING 10 Example 5 The standardarray of the binary linear (4,2,2) code is the following: 00 0000 1000 0100 0001 S 00 11 10 01 01 0110 1110 0010 0111 10 101 1 0011 1111 1010 11 1101 0101 1001 1100 + Decoding withthe standard array proceeds as follows. Let F = V E be the received word. Find the word inthe array and outputas decoded messageii the header of thecolumn in which F lies. Conceptually, this process requires storing the entire array and matching the received word to an entry in the array. However, a simplified decoding procedure canbe found by noticing that every row in the array has the same syndrome. Each row of the array, denoted Rowi, for 0 5 i < 2 n - k , is a coset of C , such that Row2 = { E2 v12) E C} . The vectorei is known as the coset leader. The syndromeof the elements in the i-th row is given by + Si = (e2 + V ) H T = EiHT, (1.22) which is independent of the particular choice of V E C. The simplified decoding procedureis: Compute the syndrome of the received word F = V, + Si, = (eg + v) HT = E i , H T , and find Sir in the leftmost columnof the standard array. Then read out the value of e i , , from the second column,an add it to the received word to obtain the closest codeword V’ E C to F. Therefore instead of n x 2” bits, standard array decoding can be implemented with an array of n x 2n”.-kbits. Example 6 Consider againthe binary linear (4,2,2) code from Example3. Suppose that the codeword V = (0110) is transmitted andthat F = (0010) is received. Then the syndrome is From the standard array of the code, the corresponding cosetleader E’ = (0100) is found, and therefore the estimated codeword is V‘ = F E’ = (0010) (0100) = (0110). One error has been corrected! This may sound strange, since the minimum distanceof the code is only two and thus according to (1 .S) single error correction is impossible. However, this can be explained by looking again at the standard array of this code (Example 5 above). Note that the third row of the array contains two distinct binary vectors of weight one. This means that only three out of a total of four single-error patterns can be corrected. The error above is one of those correctable single-error patterns. It turns out that this (4,2,2) code is the simplest instance of a linear unequal error protection (LUEP) code [WV, Van]. This LUEP code has a separation vector S = (3, a), which means that the minimum distance between any two codewords for which the first message bit differs is at least three and that for the second message bit is at least two. + + If encoding is systematic, then the above procedure givesthe estimated message U‘ in the first k positions of V’. This is a plausible reason for having a systematic encoding. INTRODUCTION 11 1.3.3 Hamming spheres, decoding regions and the standard array The standardarray is also a convenientway of understanding the concept of Hamming sphere and error correcting capability of a linear code C, introduced in Section 1.1.2. By construction, note that the 2k rightmost columnsof the standard array, denoted Colj, for 1 5 j 5 2 k , contain a codewordV j E C and aset of Z n P k - 1words at the smallest Hamming distance fromVj, that is, C01j = { V j + ei(Ei E ROW^: 0 5 i < 2n-k}. (1.23) The sets Colj are the decoding regions,in the Hamming space, around each codeword B j E C, for 0 5 j 5 2k - 1. This is to say that if codeword VU^ E C is transmitted over a BSC and the received word F lies in the set Colj, then it will be successfully decoded into V j . Hamming bound The set Colj and the error correcting capability t of code C are related by the Hamming sphere St("): A binary linear (n;k:&,in) code C has decoding regions Colj that properly containHammingspheres St(Bj),i.e., St(iij) Colj. By noticing that the size of Colj is 2n-k, and using Equation (1.6),we obtain the celebrated Hamming bound (;) 5 2n-k. (1.24) i=O The Hamming bound has several combinatorial interpretations. One of them is: The number of syndromes, 2n-k, must be greater than or equal to the number of correctable error patterns, (y ) . c:=:, Example 7 The binary repetition (3; 1;3) code has generator matrix G = ( 1 1 1) and parity-check matrix 1 1 0 " = ( l 0 1) Accordingly, its standard array is the following: S 0 00 100 11 10 01 000 010 001 l 111 011 101 110 The four vectorsin the second columnof the array (i.e., the coset leaders) are the elements of the Hamming sphereSI(000) in Figure 4, which consists of all binary vectors of length three with Hamming weightless that or equal to one. Similarly, the entries of the third (rightmost) column of the array are the elements of S 1(1 11). For this code, the Hamming bound (l .24) holds with equality. Block codes satisfying the bound (1.24) are said to be p e ~ e c codes. t The only perfect nontrivial codes are the binary Hamming (2- - 1;2m - m - 1;3) codes, the nonbinary 12 THE ART OF ERROR CORRECTING CODING (m;B Hamming - m - 1; 3) codes, q > 2, the repetition (n;1; n ) codes, the paritycheck (n;n - 1: 2)codes, the binary Golay ( 2 3 ; 1 2 , 7code ) and the ternary Golay (11:6: 5 ) code. The extended codes (obtained by appending an overall parity-check bit) of the Hamming and Golay codesare also perfect. For nonbinary linear codes, defined over a field of q elements, with q = p m and p > 2 a prime number,the Hamming bound becomes (1.25) 1.4 Weight distribution and error performance When selecting a particular coding scheme, it is important to assess its error performance. There are several measures of the performance of an ECC scheme.In this section, expressions for linear codes are introduced, for three basic channel models: TheBSC model, the additive white Gaussian noise(AWGN) channel modeland the flat Rayleigh fading channel model. I .4.I Weight distribution and undetected error probability overa BSC + The weight distribution W ( C )of an error correcting code C, is defined as the set of n 1 integers W ( C )= (Ai: 0 5 i 5 n}, such that there are Ai codewords of Hamming weight i in C, for i = 0; 1 : . . . n. An expression for the probability of undetected error of a linear code over a BSC is derived next. First, note that the Hamming weightof a word V , wtH(V) equalsthe Hamming distance to the all-zero word, wtH(V) = d ~ ( u0). , Also, as noted before, the Hamming distance between any two codewords 81 V2 in a linear code C equals the Hamming weight of their difference, dH(V1; V 2 ) = dH(U'1 U 2 : 0) = W t H ( V 1 8 2 ) = wtff(83); + + where, by linearity of C , u3 E C. The probability of an undetected error, denoted P,(C), is the probability that the received word differs from the transmitted codewordbut the syndrome equalszero. That is, s = ( v + E ) H ~ = E H ~ =W o EEC. Therefore, the probability that the syndrome of the received word is zero equalsthe probability that an error vector is a nonzero codewordin C. With transmission over a BSc, the probability that the error vector E has weight i equals the probability that i bits are in error and that the remaining n - i bits are correct. Let P(C;i) denote this probability. Then P(E;i ) = p i ( 1 - p ) - . For an undetected error to occur, the error vector E must be a nonzero codeword. There are Ai vectors of weight i in C. It follows that n n (1.26) INTRODUCTION 13 Equation (1.26) gives the exact value of Pu(C).Unfortunately, for most codes of practical interest the weight distribution W ( C )is unknown. In these cases, using the fact that the number of codewords of weight i is less than or equalto the totalnumber of words of weight i in the binary space V,, the following upperbound is obtained: (1.27) Expressions ( l .26) and (1.27) are useful when an ECC scheme is applied for error detection only, such as in communication systemswith feedback and ARQ. When a code is employed for error correction purposes,the expressions derivedin the next sections are useful. Figure 6 shows plot a of Pu(C)compared with the upper bound in the right-hand side (RHS) of (1.27). 0.1 0.01 0.0001 0.001 1e-05 P Figure 6 Exact value and upper bound on the probability of undetected error for a binary linear (4,2,2) code over an BSc. 1.4.2 Peflormance bounds over BSc, AWGN and fading channels The purposeof this section is to introduce basic channel modelsthat will be considered in the book, as well as the corresponding expressionson the error correction performance of linear codes. Errorcorrection for the BSC is consideredfirst. THE ART OF ERROR CORRECTING CODING 14 The BSC model For a binarylinear code C, as explained in the previous section, a decoding procedureusing the standard array will decode a received vectorinto the closest codeword. A decoding error will be made wheneverthe received vectors falls outside the correct decoding region. Let Li denote the number of coset leaders of weight i in the standard array of a linear code C . The probability of a correct decodingequals the probability that an error vector is a coset leader and givenby c e PC(C)= Li $(l - p ) n - i ; (1.28) i=O where C is the largest Hamming weight of a coset leader E in the standard array. For perfect codes, C = t , and such that, from the Hamming bound(1.24), c c(;) e t Li = i=O = 2n-k. i=O For binary codesin general, Equation (1.28) becomes a lowerbound on PC(C), since there exist coset leaders of weight greater than t. The probability of incorrect decoding, denoted by P, (C), also known as the probability ofa decoding error, is equal to the probability of the complement set of the event of correct decoding, i.e., P, (C) = l - PC(C). From Equation(l .28), it follows that e P,(C) = 1 - c Li pi(l - P Y . (1.29) i=O Finally, based onthe above discussionfor PC(C), the following upper boundis obtained, (1.30) which can also be expressed as (1.31) i=t+l with equality if, and only if, code C is perfect (satisfies the Hamming bound with equality). Example 9 Figure 7 shows the values of P, (C) from (1.3 l), as a function of the crossover probability p of a BSc, for the binary repetition (3, l ,3) code. INTRODUCTION 15 1 0.1 0.01 0.001 0.0001 Figure 7 Probability of a decoding error for the binary repetition(3,1,3)code. The AWGN channel model Perhaps the most important channel model in digital communications is the additive white Gaussian noise(AWGN) channel. In this section, expressions are given for the probability of a decoding error and of a bit error for linear block codes over AWGN channels. Although similar expressions hold for convolutional codes, for clarity of exposition, they are introduced together with the discussion on soft-decision decoding with the Viterbi algorithm and coded modulation in subsequent chapters. The following results constitute valuable analysis tools for the error performance evaluationof binary coded schemes over AWGN channels. Consider a binary transmission system, with coded bits in the set (0, l} mapped onto real values {+l,-l}, respectively, as illustrated in Figure 8. In the following, vectors are n-dimensional andthe following notation isused to denote avector: 3 = (Q, 2 1 , . . . ,~ ~ - 1 ) The conditionalprobability density function (pdf)of the channel output sequence y, given the input sequence5 is given by (1.32) where p R ( f i )is the pdf of n statistically independent and identically distributed (i.i.d.) noise samples, each of which is Gaussian distributed with mean ,un = 0 and variance g n = No/2, and NOis the one-sided powerspectral density of the noise. It is easy to show that maximumlikelihood decoding (MLD) of a linear code C over this channel selects a sequence 2 that minimizes the squared Euclidean distance between the received sequencey and 2 , n-l (1.33) i=O See, e.g., [WJ], [Will and [BM]. It should be noted that a decoder using Equation (1.33) as a metric is referred to as a soft-decision decoder, independently of whether or not MLD is performed. In Chapter 7, soft-decision decoding methodsare considered. The probability of a decodingerror with MLD, denoted Pe(C),is equal to the probability that a coded sequence Z is transmitted andthe noise vectorfi is such that the received sequence . THE ART OF ERROR CORRECTING CODING 16 l ' l Figure 8 Binary coded transmission system over an AWGN channel. + y = 2 f i is closer to a different coded sequencei E C, x # 3 . For alinear code C, it can be assumed that the all-zero codeword is transmitted. Then P, ( C )can be upper bounded,based on the union bound [Cla] and the weight distributionW ( C ) ,as follows: ( l .34) where R = k / n is the code rate, &/No is the energy per bit-to-noise ratio (or SNR per bit) and &(x) is given by (1.2). Figure 9 shows the evaluation of expressions for hard-decision decoding (1.30)and softdecision decoding (1.34) for the binary (3,1,3) code.Hard-decision decoding means using a decoder for the BSC which is fed by the outputs from a binary demodulator. The equivalent BSC has a crossoverprobability equal to [Pro, WJ] Note that in this particular case, since the code is perfect and contains only two codewords, both expressions are exact, not upper bounds. Figure 9 also serves to illustrate the fact that soft-decision decoding performsbetter than hard-decision decoding,in the sense of requiring less transmittedpower to achieve the same P,((?). The difference (in dB) betweenthe corresponding SNR perbit is commonly referred to as coding gain. In [FLR],the authors show that for systematic binary linear codes with binary transmission over an AWGNchannel, the probability of a bit error, denoted Pb(C),has the following upper bound: (1.35) Interestingly, besides the fact that the above bound holds only for systematic encoding, the results in [ E R ] show that systematic encoding minimizesthe probability of a bit error. This means that systematic encoding isnot only desirable, but actually optimal in the above sense. Example 10 Consider alinear binary (6,3,3) code with generator and parity-check matrices G= (:1 : : : 0 1 0 0 1 1 :)7 H= l 0 1 1 0 0 1 1 0 0 1 0 0 1 1 0 0 1 ( ) , respectively. The weight distribution of this code is W ( C ) = { 1,0,0,4,3,0,0},which can be verified by direct computation of all the codewords C = ( U , U p ) : INTRODUCTION 17 0.01 0.0001 1e-06 1e-l 2 l le-l4 1 2 3 4 \l 1 #1 ' , 5 6 7 8 91011121314151617 Eb/No (dB) I 0 , Figure 9 Probability of a decoding error for hard-decision decoding (Pe(3,1,3)-HDD) and soft-decision (Pe(3,1,3)-SDD) decoding of a binary (3,1,3) code. Binary transmission over an AWGN channel. l3 000 001 010 011 100 101 110 111 ijp 000 101 011 110 110 011 101 000 In this particular case, MLD can be performed by simply computing the squared Euclidean distance, Equation (1.33), between the received sequence and all of the eight candidate codewords. The decoded codeword is selected as that with the smallest distance. In Chapter 7, efficient methods for MLD and near-MLD of linear block codes are presented. Figure 10 shows simulations and union bounds with hard-decision and soft-decision MLD decoding with binary transmissionover an AWGNchannel. The flat Rayleigh fading channel model Another important channel model is that of flat Rayleigh fading. Fading occurs in wireless communication systemsin the form of a time-varyingdistortion of the transmitted signal. In this book, we consider the case of flat Rayleigh fading. The term "flat" refers to the fact that the channel is not frequency selective, so that its transfer function in the frequency domainis constant [BM, WJ,Pro]. As a result, a (component-wise)multiplicative distortion is present in the channel, as shown in the model depicted in Figure 11, where ct: is a vector with n component i.i.d. random variables ai, 0 5 i < n, each having a Rayleighpdf, P,; (ai) = Qie - 4 / 2 , Q2 2 0. (1.36) THE ART OF ERROR CORRECTING CODING 18 0.1 0.01 U a r 0.001 : 0.0001 i 1e-05 0 1 2 3 4 5 6 Eb/No (dB) 7 6 9 1 0 Figure 10 Simulations and union bounds for the binary (6,3,3) code. Binary transmission over an AWGN channel. With this pdf, the average SNR per bit equals E,,/No (i.e., that of the AWGN), since the second momentof the fading amplitudesis E{ a:} = 1. In evaluating the performance of a binarylinear code over aflat Rayleigh fading channel, a conditional probability of a decodingerror, Pe(CI&), or of a bit error, Pb(C16),is computed. The unconditional error probabilities are then obtained by integration over a product of ai, with pdf given by Equation (1.36). encoder Information U source Binary Mapping 0 -+l - 1' +-l L Information destination x l Soft-decision decoder Figure 11 Binary coded transmission system over a flat Rayleigh fading channel. The conditional probabilities of error are identical to thoseobtained for binary transmission over an AWGN channel. The main difference is that the arguments in the Q(.) function, which correspond to the pairwise probability of a decoding error, are now weighted by the fading amplitudesai.Considering a coded binary transmission system without channel state information (CSI),we have that (1.37) with c W A, = ai. i=l (1.38) INTRODUCTION 19 le-05 5 6 7 8 9 10 1 1 12 13 14 15 16 Eb/No (dB) 17 18 19 20 Figure 12 Simulation results (SIM), compared with the Monte Carlo union bound (Pe(3,1,3)_MC)and the Chemoff bound (Pe(3,1,3)EXP),for the binary (3,1,3)code. Binary transmission over a flat Rayleigh fading channel. Finally, the probability of a decoding error with binary transmission overflata Rayleigh fading channel is obtained by taking the expectation with respect to A,, There are several methods to evaluate or further upper bound expression (1.39). One is to evaluate numerically(1.39) by Monte Carlo integration,with the approximation where A,(C) denotes the sum of the squares of W i.i.d. random variables with Rayleigh distribution, given by (1.38), generated in the C-th outcome of a computer program, and N is a sufficiently large number that depends on the range of values of P, (C). A good rule of thumb is that N should be at least 100 timeslarger than the inverse of P,(C). (See [Jer], pp. 500-503.) Another methodis to bound the Q-function by an exponential function(see, e.g., [WJ], pp. 82-84) and to perform the integration, or to find a ChernofSbound. This approach results in a loose boundthat, however, yields a closed expression(see also [Will, p. 526, and [BM], p. 7 18): (1.41) The bound (1.41) is useful in cases where afirst-order estimation of the code performanceis desired. THE ART OF ERROR CORRECTING CODING 20 - .EXP" * 1 L h 0 m a 0.001 : 0.0001 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Eb/No (dB) Figure 13 Simulation results (SIM-(6,3,3)), compared with the Monte Carlo union bound (Pe(6,3,3)1MC)and the Chernoff bound (Pe(6,3,3)_EXP),for the binary (6,3,3)code. Binary transmission over a flat Rayleigh fading channel. Example 11 Figure 12 shows computer simulationresults of the binary (3,1,3) code over a flat Rayleigh fading channel.Note that the Monte Carlointegration of the union bound gives exactly the actual error performance of the code, since there is only one term in the bound. Also note that the Chernoff bound is about 2 dB away from the simulations, at a signal-tonoise ratio per bit E,/No > 18 dB. Example 12 Figure 13 shows the results of computer simulations of decoding the binary (6,3,3) code of Example 10, over aflat Rayleigh fading channel.In this case, the union bound is relatively loose at low values of Eb/No due to the presence of more terms in the bound. Again, it can be seen that the Chernoff bound is loose by about 2dB, at a signal-to-noise ratio per bit E,/No > 18 dB. 1.5 General structure of a hard-decision decoderof linear codes In this section, the general structure of a hard-decision decoder for linear codes is summarized. Figure 14 shows a simple block diagram of the decoding process. Notethat since hard decisions are assumed, bits at the output of a demodulator are fed into a decoder designed for a BSc. Let V E C denote a transmitted codeword. The decoder has at its input a noisy received vector F = V E. A two-step decoding procedure of a linear code is: + 0 Compute the syndrome S = FHT. Based on the code properties, the syndrome is a linear transformation of the error vector introducedin the channel, S = EHT: Based on the syndrome S , estimate the most likely error vector 8 and subtract it (modulo 2 additionin the binary case) from the received vector. ( 1.42) 21 INTRODUCTION - r - Compute syndrome Find error vector Buffer received vector Figure 14 General structure of a hard-decision decoder of a linear block code for the BSC model. Althoughmost practical decoders will notperform the aboveprocedure as stated, it is nonetheless instructive to think of a hard-decision decoder as implementing a method of solving equation(1.42).Note that any method of solving this equation constitutes a decoding method. For example, one could attempt to solve the key equation by finding a pseudo-inverse of HT,denoted ( H T ) + ,such that HT(HT)+= In,and the solution e = .(HT)+: (1.43) has the smallest Hamming weight possible. As easy as this may sound, it is a formidable task. This issue will be visited again when discussing decoding methods for BCH and ReedSolomon codes. The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) 2 Hamming, Golay and Reed-Muller codes In this chapter, important examples linear binary codes are introduced. They serve to introduce more ECC concepts as well as clever decoding algorithms. Hamming codes are perhaps the most widely knownclass of block codes, with thepossible exception of Reed-Solomon codes. As mentioned in Chapter 1, Hamming codes are optimal in the sense that they require the smallest amount of redundancy, for agiven block length, tocorrect any single error. The binary Golay code is the only other nontrivial example of an optimal triple-error correcting code. (The only other binary optimal codes are repetition and single parity-check (SPC) codes.) Reed-Muller codes can be defined as codes with an elegant combinatorial definition that are easy to decode. 2.1 Hamming codes Recall from Chapter 1, Equation (1.12), that any codeword code C satisfies UHT = 0. tj in a linear (n,IC, dmin) block (2.1) A useful interpretation of this equation is that the maximum number of linearly independent columns of the parity-check matrix H of C is equal to dmin - 1. In the binary case, for dmin = 3, the above equation translates into the sum of any two columns of H not equal to the all-zero vector. Suppose that the columns of H are binary vectors of length m. There are up to 2* - 1possible nonzero distinct columns. Therefore, the length of a binary single-error correcting code is given by n<2m-1. This inequality is precisely the Hamming bound (1.24) for an error correcting code of length n, with n - IC = m and t = 1. Consequently, a code achieving this bound with equality is known as a Hamming code. Example 13 With m = 3, we obtain the Hamming ( 7 , 4 , 3 )code, with parity-check matrix 1 1 1 0 1 0 0 1 1 0 1 0 0 1 THE ART CORRECTING OF ERROR 24 CODING In the rest of this section, an encoding and decoding algorithm is given that is suitable for software implementation and for simulation purposes. Later in the book, effective softdecision decoding algorithms are given, which can be used over real-valued channels such as the AWGN channel and the Rayleigh fading channel. As noted before, Hamming codes have the property that their parity-check matrix H has all of its columns different. If a single error occurs, in position j , 1 5 j 5 n, then the syndrome of the received vector equals the column of H in the position in which the error occurred. Let e denote the error vector added in the transmission of a codeword over a BSC channel, and assume that all of its components are equal to zero except for the j-th component, ej = 1. Then, the syndrome of the received wordequals where h j denotes the j-th column of H , and 1 5 j 5 n. 2. I . 1 Encoding and decoding procedures From Equation (2.2) above, it foliows that if it is possible to express the columns of H as the binary representation of integers, then the value of the syndrome directly gives the position of the error. This is the idea in the algorithms for encoding and decoding presented below. The columns of the parity-check matrix H are expressed as binary representations of integer numbers i in the range [l,n] and in increasing order. Let the resulting matrix be denoted by H'. Clearly, the code associated with H' is equivalent to the original Hamming code with parity-check matrix H , up to a permutation (or exchange) of positions. Recall from Chapter 1 that the parity-check matrix in systematic form contains the ( n - k) x ( n - k) identity matrix, I n - k , as in Equation (1.16). Clearly, when expressing H' with columns equal to the binary representation of the (integer) column number, the identity matrix is contained in those columns of H' that correspond to even powers of 2, i.e., of the form 2e, 0 5 l < m. This is illustrated in the example below. Example 14 Let m = 3. Then a systematic parity-check matrix is 1 1 0 1 1 0 0 0 1 1 1 0 0 1 Matrix H' is given by the binary representation of integers 1 to 7 (in the following, the topmost part corresponds to the least-significant bit in the binary representation of an integer): 1 0 1 0 1 0 1 0 0 0 1 1 1 1 and matrix In-k is contained in columns 1 , 2 and 4. In general, for a (2m - l, 2m - 1 - m ) Hamming code, the identity matrix is contained in column numbers 1,2,4, . . . ,2"-l of H'. HAMMING, GOLAY AND REED-MULLER CODES 25 Encoding Encoding proceeds as dictated by Equation (1.18) of Chapter 1. When computing the parity check bit p j , for 1 5 j 5 m, the column position numbers are examined and those which are not powers of two correspond to message positions and the corresponding message bits are included in the computation. This encoding procedure is somewhat more complicated than that of a systematic Hamming code. In return, however, decoding is extremely simple. Depending on the application, this approach may provethe most appropriate, since it is usually decoding that is required to be veryfast. Decoding Once the codewords have been computed in accordance to matrix H ' , decoding is easy. The syndrome (2.2) equals the position number in which the error occurred! In a decoding procedure, after the computation of syndrome S , when regarded as an integer, the erroneous position is then corrected as v, = v, B 1: where @ denotes exclusive-or (i.e., 0 @ 0 = 0, 0 @ 1 = 1, 1@ 0 = 1,and 1 @ 1 = 0). Program hamming.c in the ECC home page implements the above encoding and decoding procedures for binary Hamming codes. 2.2 ThebinaryGolaycode Golay [Gol] noticed that (?) = 211 i=O This equality shows the possible existence of a perfect binary (23; 12,7) code, with t = 3, that is, capable of correcting all possible patterns of at mostthree errors in 23 bit positions. In his paper, Golay gave a generator matrix of such a triple-error correcting binary code. Because of its relatively small length (23), dimension (12) and number of redundant bits (1 l), the binary (23; 12,7) Golay code can be encoded and decoded simply by using look-up tables. The program golay2 3 . c in the ECC home page uses a 16K x 23 bits encoding table and an 8K x 23 bits decoding table. 2.2.1 Encoding Encoding is based on a look-up table (LUT) that contains a list of all the 212 = 4096 codewords, which are indexed directly by the data. Let U denote a 12-bit binary vector representing the data to be encoded, and let V denote the corresponding 23-bit codeword. The encoder LUT is constructed by generating all 4096 12-bit vectors and computing the syndrome of a pattern for which the 12 MSB equal to the information bits and the 11 LSB equal to zero. The 1 l-bit syndrome then becomes the LSB part of the codeword. The LUT is aone-to-one mapping from U onto V, which can be expressed as V = LUT(U) = ( U ; get-syndrome(U: 0)). (2.3) THE ART CORRECTING OF ERROR 26 CODING In the construction of the encoder LUT, advantage is taken from the cyclic nature of the Golay code. Its generator polynomial’ is g(x) = El1 + d o + + x5 + x4 + x2 + 1; which inhexadecimal notation equals C75.This polynomial is used togenerate the syndrome in the procedure “get-syndrome” indicated in Equation (2.3) above. 2.2.2 Decoding Recall from Chapter l , Figure 14, that the decoder’s task is to estimate the most-likely (i.e., least Hamming weight) error vector e from the received vector F. The decoder for the Golay code is based on an LUT that accepts as input the syndrome S of the received vector F and outputs the error vector E . The procedure to construct the decoder LUT is as follows: 1. Generate all possible error patterns e of Hamming weight less than or equal to three; 2. For each error pattern, compute the corresponding syndrome S = get-syndrome(e); 3. Store at location S of the LUT, the error vector E, LUT(S) = e. With the decoder LUT, upon reception of a corrupted received word F, correction of up to three bit errors is accomplished by the following: 6 = F CE LUT (get-syndrome(F)) where 6 denotes the corrected word. 2.2.3 Arithmetic decoding of the extended (24, 12, 8) Golay code. In this section, a decoding procedure for the extended (24: 12; 8) Golay code, C 2 4 , is presented based on an arithmetic decoding algorithm [Wic, VO]. The algorithm utilizes the rows and columns of the parity submatrix B in the parity-check matrix H = ( B 1 1 1 2 ~ 1 2) . Note that an extended (24; 12,8) Golay code, Ci4,which is equivalent to C 2 4 up to a permutation in the bit positions, can be obtained by adding an overall parity-check bit at the end of each codeword in the (23: 12,7) Golay code. In hexadecimal notation, the twelve rows of B, denoted rowi, l 5 i 5 12, are: Ox7ff, Oxee2, Oxdc5, Oxb8b, Oxf16, Oxe2d, Oxc5b, Ox8b7, Ox96e, Oxadc, Oxdb8, Oxb71. It is interesting to note that sub-matrix B of the parity-check matrix of C 2 4 satisfies B = B T . This means that code C 2 4 is a self-dual code. Details of self-dual codes are not covered in this book. Interested readers are referred to [MS, Wic]. In program golay24. encoding c, is performed by recurrence with H , as indicated in Equation 1.18. As before, let WtH(x) denote the Hamming weight of a vector 3 . Decoding steps are as follows [Wic, VO]: The definition of a generator polynomial is given in Chapter 3. HAMMING, GOLAY AND REED-MULLER CODES 27 1. Compute the syndrome S = T H T . 2 . If wtH(S) 5 3, then set E = ( S , 0) and go to step 8. 3. If wtH(S rowi) 5 2, then set e = ( S + rowi, %i), where %i is a 12-bit vector with only the i-th coordinate nonzero. 4. Compute sB. 5 . If wtH(SB) 5 3, then set e = (0,SB) and go to step 8. 6. If wtH(d3 rowi) 5 2, then set e = (Si,S.B rowi), with Zi defined as above, and go to step 8. 7. F is corrupted by an uncorrectable error pattern, set error failure flag.End of decoding. 8. Set = F e. End of decoding. + + + + 2.3 Binary Reed-Mullercodes Binary Reed-Muller (M) codes constitute a family of error correcting codes that are easy to decode using majority-logic circuits. In addition, codes in this family are known to have relatively simple and highly structured trellises [LKFF]. More on trellises of linear block codes is discussed in Chapter 7. An elegant definition of binary RM code is obtained with the use of binary polynomials (or Boolean functions). With this definition, RM codes become close relatives of BCH codes and RS codes, all members of the class of polynomial codes2. 2.3.1 Boolean polynomials and RM codes This section closely follows the development of [MS]. Let f(xl,x2,... ,xm) denote a Boolean function on m binary-valued variables x1,x2,. . . ,x,. It is well known that such a function can be specified by a truth table. The truth table lists the value of f for all 2m combinations of values of its arguments. All the usual Boolean operations (such as “AND”,“OR’) can be defined in a Boolean function. Example 15 Consider the function f(x1,x2) with the following truth table: x2 x1 f(X1,Xz) 0 0 0 0 1 1 1 0 1 1 1 0 Then, f(z1,~ 2 = ) (XI AND NOT(22)) OR (NOT(21) AND ~ 2 ) . Associated with each Boolean function f,let .f denote the binary vector of length ,2 which is obtained from evaluating f at all possible 2, values of the m variables X I ,xz,. . . ,x,. In the example above, .f = (OllO), where the convention taken for ordering the bit positions of f is in accordance to a binary representation of integers, with x1 being the least-significant bit (LSB) and x, the most-significant bit (MSB). Note also that a Boolean function can be written directly from its truth table to get the disjunctive noma1 form(DNF). Using the DNF, any Boolean function can be expressed as Polynomial codes are presented in Section 3.4. THE ART OF ERROR CORRECTING CODING 28 the sum3 of 2, elementary functions: 1 , q ,x2, . . . ,x,, - f - =1 + + a 2 Z 2 + . . . + urn%, + a123132 21x2, . . . ,x1z2 . . . x,, such that + . .. + ~12...,3132 . . .%m, (2.4) + where i is added to account for independent terms (degree 0). For Example 15, f = 31 32. A binary RM (2,, k , code, denoted RM,,,, is defined as the set of vectors associated with all Boolean functions of degree up to T in m variables. R M , , is also known as the r-th order RM code of length 2,. The dimension of RM,,, can easily by shown to be equal to k = i :i=O (T), which corresponds to the number of ways polynomials of degree up to r can be constructed with m variables. In view of Equation (2.4), a generator matrix for RM,,, is formed by taking as rows the vectors associated with the k Boolean functions which can be expressed as polynomials of degree up to r in m variables. Example 16 The first-order RM code of length 8, RM1,3, is an (8,4,4) binary code, and can be constructed from Boolean functions of degree up to 1 in 3 variables: { 1,x1,z2, Q}, so that - 1 = 1 1 z 1 = 0 0 z 2 = 0 0 3 3 = 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 0 1 1 1 1 0 1 1 1 1 A generator matrix for R M 1 , 3 is thus (i) ( - G= = 1 1 1 1 1 1 1 1 ) 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 1 0 1 0 1 0 1 0 1 Note that code RM1,s can be also obtained from a Hamming (7,4,3)code by appending at the end of each codeword an overall parity-check bit. The only difference between the extended Hamming code and RM1,3 will bea possible permutation of bit (column) positions. Dual codes of RM codes arealso RM codes It can be shown that RMrn-,-1,, is the dual code of RM,,. In other words, the generator matrix of RMm-,-l,, can be used as a parity-check matrix of RM,,. 2.3.2 Finite geometries and majority-logic decoding RM codes can be formulated in terms of ajinite geometry. An Euclidean geometry EG(m,2) of dimension m over G F ( 2 )consists of 2 , points, which are all the binary vectors of length “sum” means logical “XOR” and “multiplication” means logical “AND’ in thk context HAMMING, GOLAY AND REED-MULLER CODES 29 m. Note that the columns of the matrix formed by the last three rows of the generator matrix of RM1,3, see Example 16 above, are the 8 points of EG(3,2).If the zero point is deleted, then a projective geometry PG(m - 1 , 2 ) is obtained. The reader is referred to [LC] for an excellent treatment of finite geometries and RM codes. Finite geometry codes are in fact a generalization of RM codes4. The connection between codes and finite geometries can be introduced as follows: Consider EG(m,2 ) . The columns of the matrix (ZTZ; .. are taken as the coordinates of points of the geometry EG(m,2 ) . Then, there is a one-to-one correspondence between the components of binary vectors of length 2 , and the points of EG(m,2). A given binary vector of length 2 , is associated with a subset of points of EG(m,2). In particular, a subset of EG(m,2 ) can be associated with each binary vector I3 = ( W Iw2, , . . . ,w p ) of length 2 m , by interpreting it as selecting points whenever tui = 1. Stated otherwise, I3 is an incidence vector. Binary Reed-Muller codes can then be defined as follows: The codewords of RM,,, are the incidence vectors of all subspaces (i.e., linear combinations of points) of dimension m - r in EG(m,2) (Theorem 8 of [MS]). From this it follows that the number of minimum weight codewords of RM,,, is The code obtained by deleting (or puncturing) the coordinate corresponding to x, = 0 from all the codewords of M,,, is the binary cyclic RM:,, code, which as x1=x2="'= minimum-weight codewords. In terms of decoding, it turns out that RM codes can be decoded with so-called majoritylogic (ML) decoding. The idea is the following: The parity-check matrix induces ZnPk paritycheck equations. Designing an ML decoder is selecting a subset of the parity-check equations in such a way that a majority vote is taken on the value of a specific code position. As an illustration, consider again the RM1,3 code of Example 16 above. Example 17 Let tj = uG = ( V I , 712, v3 ,714, v s , ' u 6 , w7,W g ) be a codeword in w 1 , 3 . As mentioned before, in this case equation (2.5) is also a parity-check matrix, since r = 1 and m - r - 1 = 1,and the code is self-dual. All possible 15 nonzero combinations of parity-check equations (rows of H ) are the following: 30 THE ART OF ERROR CORRECTING CODING + v 3 + v5 + v7 + v4 + 215 + v2 + v 4 + V 5 + v 7 v2 + + + 217 v1 +v2 +v7 + v 8 + 'U3 f f ?k3 v1 + v 4 + v5 + v8 v2 + v3 f v5 + v8 211 f v4 + + v7 v1 'U3 'U6 213 211 ?& 216 716 0 0 0 0 = 0 = 0 0 0 = 0 = = = = (2.8) The reader is invited to verify that the sum vi + v ~ jof every pair of code bits vi,vj,with i # j , appears in exactly four equations. Whenever a set of equations includes a term vi + vj, but no other sum of pairs appears more than once, the parity-checks involved are said to be orthogonal on positions vi and vj. It is now shown howa single error can be corrected. Let T=Vijfe=(rl,r2,rQ,r4,r5,r6,r7,r8) denote a received vector after codeword V is transmitted over a BSc. Suppose that a single error is to be corrected in the fifth position, v5. A procedure to design an ML decoder for this case is as follows: TWOequations involving the term vi v5 are selected, with i # 5, and another set of two equations with the term vj v5, j # 5, with i # j. Select (arbitrarily, as long as i # j and both different than 5), say, i = 3, j = 4. There are four parity checks orthogonal to the term w3 215. Select any two of them. Do the same for the term v4 + v5. The syndromes associated with these equations are denoted S1 and S, for v3 + v5 and S3 and S4 for v4 715, + + + + A S1 = A TI +r3 + r 5 S2 = r3 +r4 S3 = S4 = A 7-2 A TI +r7 + r 5 +r6 +r4 + r 5 +r7 + 1-4 + r5 + T8 Because V is acodeword in RM1,3, the set of equations (2.9) is equivalent to A S1 = el +e3 + e 5 + e 7 A S2 e3 + e 4 +-e5+ e 6 A S3 e2 + e 4 + e s + e 7 A S4 = el + e 4 + e 5 + e 8 Since SI,Sa, S3 and S4 are orthogonal on orthogonal on e5 can be formed as: A Si = A Si = e3 + e5 and e; e4 (2.10) + e5, a new pair of equations + e; e: + e ; (2.1 1) HAMMING, GOLAY AND REED-MULLER CODES 31 where el, j = 3,4; 5, represents the ML estimate from (2.9). Equations (2.1 1) are orthogonal on e; and consequently the value of e; can be obtained by a majority vote, for example, setting it to the output of an “AND’ gate with inputs S i and Si. Suppose that the codeword V = (11110000) is transmitted and F = (11111000) is received. Then (2.10) gives: S 1 = TI + r3 + r5 + 7-7 = 1 S4 = TI + r4 + r5 + SZ = r3 + ~4 + r5 + 7-6 = 1 S3 = 7-2 + r4 + r5 + 7-7 = 1 + T8 =1 (2.12) + Thus both e3 e5e4 e5 are estimated as having value equal to ‘l’. From (2.1 l), it is concluded that e5 = 1, and the estimated error vector is e = (OOOOlOOO), from which the estimated codeword is V = F e = (11110000). This shows how an error in the fifth position can be corrected with a two-step ML decoder. + In the previous example, it was shown howan error in a specific position of an RM1,3code can be corrected. A similar procedure can be applied to every position in the code. Therefore, a total of eight ML estimates will be obtained. In general, an R M , , code can be decoded with an (T 1)-step ML decoder capable of correcting any combination of up to L(2m-2 - 1)/2J random errors [MS, LC]. + code is simpler to decode. In a cyclic code5 C, if In addition, a cyclic RM:,, (v1 ; 212: . . . ;v,) is a codeword of C, then the cyclic shij? (W,; wl; . . . vn-l) also is a codeword of C. Therefore, once a position can be corrected via ML decoding, the remaining positions can also be corrected with the same algorithm (or hardware circuit) by cyclically shifting received codewords, until all n positions have been tried. Example 18 In this example, a decoder for the cyclic code, a binary Hamming (7; 4;3) code, is derived. To obtain the parity-check equations from those of the RM1,3 code, remove the coordinate v1 for which x1 = x2 = . . . = 2, from all equations. Let the codewords of RMT,, be indexed by re-labeling the codeword elements: (v2 ; v3 v4 v5 Vug: v7 v g ) + (v1; v2 ~3 v4 v5 ; v6 : v7). As before, anML decoder for correcting an error in an arbitrary position (say, thejifth position again) can be derived. This can be shown to result in the following seven nonzero (linearly independent) parity-check equations: (2.13) See Chapter 3 THE ART OF ERROR CORRECTINGCODING 32 open at n=7 V Figure 15 A majority-logic decoder for a cyclic RM*(1,3) code. In a manner similar to the previous example, the syndromes S1 and S2 below are orthogonal on v4 and 215, and S2 and S3 are orthogonal on 215 and 216: (2.14) + Based on the estimates eh e; and e; formed to give the finalestimate, + e; two additional orthogonal equations on e5 can be (2.15) where e;, j = 4: 5; 6, represents the ML estimate from the previous step. This results in the circuit shown in the Figure 15. The circuit operates as follows. Initially the contents of the seven-bit register are set to zero. Suppose that a single error is contained in the received word in position i , for 1 5 i 5 7. At each clock cycle, the contents of the register are cyclically shifted to the right by one position. Time, in clock cycles, is denoted by the subindex n in the following. Consider first the case i = 1. That is, there is an error in the first codeword position. After three cycles, the error is contained in register 5 (vs). The output of the majority-logic circuit is set to e , = 1. Four cycles later (a total of seven cycles), the first received bit isoutput and the error corrected. Consider now the case i = 7. After nine cycles, the error is detected and e , = 1. Again, four cycles later (total 13 cycles), the bit in the last position is output and the error corrected. This decoder has a latency of 13 cycles. Every 13 cycles, the contents of the shift register are cleared and a new codeword can be processed. In the next chapter, binary cyclic codes and the powerful family ofBCH introduced. codes are The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) 3 Binary cyclic codes and BCH codes The aim of this chapter is to introduce a minimum set of concepts necessary for the understanding of binary cyclic codes and for the efficient implementation of their encoding and decoding procedures. Also in this chapter, the important family of binary BCH codes is introduced. BCH codes are a family of cyclic codes, which gives them an algebraic structure that is useful in simplifying their encoding and decoding procedures. Binary BCH codes with minimum distance 3, better known as Hamming codes, have been a very popular choice in computer networks and in memories, due to their simple and fast encoding and decoding. Also, a shortened (48; 36,5) BCH code is used in the U.S. cellular TDMA system specification, standard IS-54. 3.1 Binarycycliccodes Cyclic codes are a class of error correcting codes that can be efficientlyencoded and decoded using simple shift-registers and combinatorial logic elements, based on their representation using polynomials. In this section, the fundamental concepts of cyclic codes are discussed. 3.1.1 c Generator and parity-check polynomials Let denote a linear (n:k) block code. Let ii and 6 denote a message and the corresponding codeword in respectively. Cyclic codes are linear codes with properties that make them suitable for hardware implementation. To every codeword 6 a polynomial v(z) is associated, c, 6 = ( W ~ ; w 1 ; . ' . ; W n - 1 ) ~ ~ ( s ) = ~ ~ + W l z + . . . W nn-l - 1.z The indeterminant z serves to indicate the relative position of an element vi of .li as a term wixi of a(z),0 5 i < 71. A linear block code C is cyclic if and only if every cyclic shift of a codeword is another codeword, v = ( W O ;211;. . . ; wn-l) E c* = ( W n - l ; W O ; ' . . ;W n - 2 ) E c. In the language of polynomials, a cyclic shift by one position, denoted d l ) ( x ) , is accomplished by a multiplication by z modulo (x" - l), U(z)E C e v(')(%) = xU(x) mod (xn - l) E C. Shift-registers can be used for this purpose, as illustrated in Figure 16. THE ART OF ERROR CORRECTINGCODING 34 Figure 16 A cyclic shift register Example 19 Consider the case of n = 7. Then, a cyclic shift by one position of the vector V = (O1O1011)equals d l ) = (1010101). In terms ofpolynomials, a ( x ) = x + 5 3 + 2 5 + 2 6 and + d l ) ( x ) = x~(z) = ' x x4 + x6 + x7 mod (x7 + l) = x2+x4+x6+x7+(x7+1)+1 = l+22+x4+x6 3.I .2 The generator polynomial An important property of cyclic codes is that all code polynomials U(z) are multiples of a unique polynomial, g ( x ) , called the generator polynomial of the code. This polynomial is specified by its roots, which are called the zeros of the code. It can be shown that the generator polynomial g(z) divides (x" - 1).(As with integers, " a ( z )divides b(x)" whenever b(z) = q ( x ) a ( z ) . )Therefore, to find a generator polynomial, the polynomial (x" - 1) must be factored into its irreduciblefuctors, 4j (x),j = 1;2:. . . : l , (x" - 1) = 41 ( x ) 4 2(x). . . de(.). (3.1) + Also, note that over the field of binary numbers, a - b and a b (modulo 2) give the same result. In the remainder of the text, as all the codes are defined over the binary field, or its extensions, no distinction is made between these operations. As a consequence of the above, the polynomial g ( x ) is given by ax)= &(x). j€Jc(l,a,...,c} Example 20 With coefficients over Z2 = (0: l},the polynomial x7 x7 + 1 factors as + I = (x + 1)(..3 + x + 1)(.3 + x 2 + 1). Some examples of cyclic codes of length 7 are: 0 A binary cyclic Hamming (7: 4; 3) code is generated by the polynomial g(x) = x3 + II: + 1. 0 A binary cyclic parity-check (7: 6; 2) code is generated by = (.3 0 + x + 1 1 ( ~+322 + 1). A binary maximum-length-sequence (7: 3: 4)code is generated by g(x) = + 1)(x3+ x + 1). BINARY CYCLIC CODES AND BCH CODES 35 3.1.3 Encoding and decoding of binary cyclic codes The dimension of a binary cyclic ( n ,k ) code is given by k = n - deg [g(x)], where deg [.] denotes the degree of the argument. Since a cyclic code C is also linear, any set of k linearly independent (LI) vectors can be selected as a generator matrix. In particular, the binary vectors associated with g(z),xg(z),. . ., xk-lij(x) are LI. These vectors can be used as rows of a generator matrix of C. In this case a non-systematic encoding rule is achieved. That is, the message bits do not appear explicitly in any positions of the codewords. Example 21 Consider the cyclic Hamming (7,4,3) code, with generator polynomial g(z) = x3 x 1 e g = (1101). A generator matrix for this code is + + G= ( 1 0 0 0 1 1 0 0 0 1 1 0 1 0 1 1 0 1 0 1 0 0 1 0 0 0 0 ' 1 ) Alternatively, the parity sub-matrix of a generator matrix of a cyclic code can be constructed with the vectors associated with the following polynomials: mod g(x), xn-l xn-k+l mod 9(x), xn-' mod g ( x ) , and a systematic encoding is obtained, as illustrated in the example below, Example 22 Let c be the cyclic Hamming ( 7 , 4 , 3 )code. Then g(z) = z3+ x + 1, and x + 1, z6 mod (x3 + x + 1) = ' z5mod (x3+ x + 1) = 'x + x + 1, x4 mod (x3 + x + 1) = 'x + x, z3mod (x3 + x + 1) = z + 1. It follows that a systematic generator matrix of C is G= f 1 0 0 0 1 0 1 0 1 0 0 1 1 1 \ 0 0 0 1 0 1 1 / Let U(x) be associated with a message to be encoded. Encoding of codewords of a binary cyclic code can be either non-systematic or systematic, depending on the way that the message is processed: Non-systematic encoding v(2) = U(z)g(x) (3.3) Systematic encoding U(x) = zn-"u(z) + [z"-'fi(x) mod g(.)]. (3.4) THE ART OF ERROR CORRECTING CODING 36 3.1.4 The parity-check polynomial Another polynomial, h ( z ) ,called the parity-check polynomial, can be associated with the parity-check matrix. The generator polynomial and the parity-check polynomial are related by g ( z ) h ( z )= 2" 1. (3.5) + The parity-check polynomial can be computed from the generator polynomial as &(x) = ( z n l)/g(z) = h0 hlz . . . h k x k .Then, a parity-check matrix for C is given by using as rows the binary vectors associated with the first n - k - 1 nonzero cyclic shifts h(j)(z)= zjh(z) mod (xn - l),j = 0,1,. . . ,n - k - 1. + + + + H= Example 23 The parity-check polynomial for the cyclic Hamming (7,4,3) code, with generator polynomial g(z) = z3+ z + 1, is h ( z )= (x7 + z 1) = z4 x' + z 1. A parity-check matrix for this code is + H= + 1 1 1 1 0 1 0 0 0 1 1 1 0 1 0 0 0 1 1 1 0 1 ( + + . In the same manner as with linear codes, systematic encoding can be also performed by a recursion with h ( z )using u(z)h(z)= 0 mod (5" 1). + Systematic encoding can be performed as follows [PW, LC]. Suppose first that the code rate k / n 5 0.5. Let G(%) = uo u1z . . . U k - 1 denote a polynomial of degree k - 1, whose coefficients ut,e = 0, l, . . . , k - 1, are the message bits to be encoded. Let 'u(z) denote the code polynomial polynomial in C corresponding to the information polynomial 21(z). In the first step, v! = ut, for k = 0, l , .. . , k - 1. From the cyclic structure of the code, it follows that the redundant bits v!, e = k , k 1,. . . , n - 1,are obtained recursively via the parity-check relations + + + + e-1 ve = c U j h ( ! - k ) , j , l? = k , k + l,...,n- 1 (3.7) j=O where h ( t - k ) , j denotes the j-th entry in the ( k - k)-th row of matrix (3.6). In the case of high-rate cyclic (n,k ) codes, say k / n > 0.5, encoding by division of z"-'G(z) by g(z)is more efficient. Either way, byrecursion with h(z)or by division by g(z), the coefficients of the code polynomials are in systematic form, so that the first k coefficients are the message bits, and the remaining n - k coefficients constitute the redundant bits. Figure 17 shows the block diagram of an encoder for a binary cyclic code with generator polynomial g(z).Initially, the switch (lower right portion of the figure) is in position 1, and the message bits are both transmitted through the channel and simultaneously fed to a sequential BINARY CYCLIC CODES AND BCH CODES 37 circuit that multiplies by xn-k and divides by ij(z).After k cycles, the shift register contains the remainder of the division, the switch moves to position 2 and the contents of the register are sent through the channel. Figure 17 Circuit for systematic encoding: Division by ij(z). Duals of cyclic codes and maximum-length-sequence codes By analogy with linear codes, the dual code of a cyclic code C with generator polynomial is the cyclic code CL generated by the polynomial h(z). The important class of maximum-length-sequence (MLS) cyclic codes [PW] has as members the duals of the cyclic Hamming codes. An MLS cyclic (2m - 1,m, 2m-1) code is generated by the polynomial g(z) = (xn l)/p(x), wherep(z) is aprimitivepolynomial'. g(%) + 3.1.5 Shortened cyclic codes and CRC codes There are many practical applications in which anerror correcting code with simple encoding and decoding procedures is desired, but existing constructions do not give the desired length, dimension and minimum distance. The following is an example from an email recently sent to the author: We plan to use a simple FEC/ECC scheme to detectkorrect single-bit errors in a 64-bit data block. The objective is to find or choose an ECC scheme to correct single-bit errors withup to 8 bits of overhead, giving amaximum of 72 bits ( 64 data bits plus 8 redundant bits) in total. Naturally, since 72 is not of the form 2m - 1, none of the cyclic codes studied so far can be applied directly. One possible solution is to use a cyclic Hamming (127,120,3) code and to shorten it until a dimension k = 64 is reached. This yields a shortened Hamming (71,64,3) code2. Shortening is accomplished by not usingall the information bits of a code. Let S For a definition of primitive polynomial, see page 41. This is an example to introduce the concept of shortening. A (72,64,4) single-error-correctingldouble-errordetecting (SECIDED) code, based on a shortened Hamming code, and adding an overall parity-check bit, was proposed for the IBM model 360 mainframe ([Hsia] and Chapter 16 of [LC]). THE ART OF ERROR CORRECTING CODING 38 denote the number of information bits not used, referred to as the shortening depth. Let C denote a cyclic (n,k , d ) code. A shortened message is obtained by fixing S (arbitrary) message positions to zero. This leaves k - S positions available for the message bits. Without loss of generality, let the highest positions in a message be set to zero. Then U ( % ) = uo u1z . . . ~k-1-~z'-'-~. The output of a systematic encoder, when the input is the message polynomial U(z),produces the code polynomial V(z) = z"-'ii(z) [zn-'U(z) mod g(.)], of degree up to n - l - S . This shows that the resulting shortened code C, is a linear ( n - S,k - S , d,) code with d, 2 d. In general, the shortened code C, is no longer cyclic. + + + + Example 24 Let C denote the cyclic Hamming ( 7 , 4 , 3 ) code with generator polynomial g(z) = 1 z z3.A new code is derived from C by sending 2 leading zeros followed by two information bits and the same three redundant bits computed by an encoder in C. This process gives a set of codewords that forms a shortened linear ( 5 , 2 , 3 ) code. + + The fundamental property of a shortened code C, obtained from a cyclic code is that, although the code is generally no longer cyclic, the same encoder and decoder can be used, after the leading zeros are properly taken into account. In computer simulations, it is easy to simply pad the codewords with zeros followed by the codeword in C, and use the same encoding and decoding algorithms discussed in the book. This method is widely used in hardware implementations of Reed-Solomon decoders. Alternatively, the leading zeros in a message do not need to be included in the codeword. Instead, the decoder circuit is modified to multiply the incoming received polynomial f ( z ) by xn-'+, modulo g(z),instead of zn-' modulo g(z) in the conventional decoder. More details on the modified encoder and decoder structures for shortened cyclic code can be found in [PW, LC, Wic], among other references. Another possible solution is to try toconstruct other classes of cyclic codes with thedesired parameters. Interesting classes of cyclic codes not covered in the text are the non-primitive BCH codes [PW] and the Euclidean geometry (EG) and projective geometry (PG) codes [LC]. Yet another possibility is to use a non-binary cyclic code, such as a Reed-Solomon code discussed in the next chapter, and to express it in terms of bits. This binary image of a RS code will have the additional feature of being able to correct many bursts of errors. See Chapter 4 for more information. CRC codes One of the most popular forms of ECC are the cyclic redundancy check codes, or CRC codes. These cyclic codes are used to detect errors in blocks of data. CRC codes are cyclic codes of length n 5 2m - 1. Typically, CRC codes have generator polynomials of the form (1 z)g(z), where g(z) is the generator polynomial of a cyclic Hamming code. Common values of m are 12, 16 and 32. The choice of the generator polynomials is dictated by the undetected error probability, which depends on the weight distribution of the code. The computation of the undetected error probability of a cyclic code is tantamount to determining its weight distribution. This has remained an elusive task, even after 50 years of coding theory, with some progress reported in [FKKL, Kaz] andreferences therein. Below is a list of the most popular generator polynomials of CRC codes, or CRCpolynomials: + BINARY CYCLIC CODES AND BCH CODES m Code CRC-12 CRC-16 CRC-CCITT CRC-32 12 16 16 32 + 39 ax) + zJ + x2 + z + 1 + + x2 + 1 %l2 d 1 Zl6 d 5 + 212 + 2 5 + 1 z32+ zZ6+ + z2’ + 2 1 6 + 2 1 2 + 211 + d o + 2* + x7 + +z5 + 2 4 f 2 2 + z + 1 x16 3.2 General decoding of cyclic codes + Let T(z) = V(z) $x), where e(x) is the error polynomial associated with an error vector produced after transmission over a BSC channel. Then the syndrome polynomial is defined as Figure 18 shows the general architecture of a decoder for cyclic codes. The syndrome polynomial .$(x)is used to determine the error polynomial e(z). Since a cyclic code is first of all a linear code, this architecture can be thought of as a “standard array approach” to the decoding of cyclic codes. ?(x) = $x) + S(x) ........ v Division by g(x) . . . . . .......... . . . . .......... ., W 7 -1 - D S(x) = ?(x) mod g(x) Figure 18 General architecture of a decoder for cyclic codes. The decoding problem amounts to finding the (unknown) error polynomial E(z) from the (known) syndrome polynomial S(z).These two polynomials are related by equation (3.Q which is the basis of a syndrome decoder (also referred to as a Meggit decoder [Meg]) for cyclic codes. A related decoder is the error-trapping decoder [Kasm], which checks if the error polynomial e(x) is contained (“trapped”) in the syndrome polynomial S(z). Only a limited number of classes of codes have relatively simple decoders, e.g., cyclic Hamming and Golay codes. As the error correcting capability t = L(dmin - 1)/2] increases, however, the complexity of an architecture based only on (combinatorial) detection of errors becomes too large. Suppose that an error in the position corresponding to zn-’ (the firstreceived bit) occurs. In other words, e(z) = xn-l. The corresponding syndrome polynomial is S(z) = x”-lrnodij(z). The code is cyclic, and thus ifan error pattern affecting a given position is detected, any other error can bedetected as well, by cyclically shifting the contents of the syndrome polynomial and the error polynomial. The syndrome decoder checks the syndrome for each received position and, if the pattern zn-lmodg(z) is detected, that position is corrected. Example 25 In this example, the decoding of a cyclic (7: 4; 3) Hamming code is illustrated. For this code, ij(x) = z3 z + 1. The syndrome decoding circuit is shown in Figure 19. The + 40 THE ART CORRECTING OF CODING ERROR received bits are stored in ashift register and at the same time fed to adivide-by-g(z) circuit. After all the seven bits have been received, theshift register contents are shiftedone at a time, and a combinatorial gate checks if the syndrome polynomial z6mod(l z z3)= 1 x2, or (101) in binary vector notation,is present in theshift register when the output of the gate is equal to one, the error is at the position z6 and is corrected. At the same time, the error isfed back to the divide-by-g(z) circuitto bring thecontents of the register equal to all zeros, upon successful completion of decoding. This also allows detection of any anomaliesat the end of the decoding process, by checking that the contents of the shift register are not all equal to zero. + + + x3 Figure 19 Syndrome decoder for a binary cyclic Hamming (7,4) code. Attention is nowfocused on cyclic codes with large error correcting capabilities, for which the decoding problem can be treated as that of solving sets of equations. Because of this, the notion of a jield, a set in which one can multiply, add and find inverses, is required. Cyclic codes have a rich algebraic structure.It will be shownlater that powerful decoding algorithms can be implemented efficiently when the roots of the generator polynomial are invoked and arithmetic over ajinitejield used. Recall that thegenerator polynomial is the product of binary irreducible polynomials: g(z) = n &(X). j€JC{l,Z,...,l} The algebraic structure of a cyclic codes enables one to find the factors (roots) of each4 j (x) in a splitting jield (also known as extension field). In the case of interest, that is, when the underlying symbols are bits, the splitting field becomes a Guloisjield3. Some authors refer to Galois fields as jnite j e l d s . The standard notation that will be used in the text is G F ( g ) , where q = 2m. (Although, in general, q can be the power of any prime number.) Example 26 In this example, the reader is reminded that the concept of splitting field is very familiar. Consider the field of real numbers. Over this field, it is well known that the ~~ After the famous French mathematician Evariste Galois (18 1 1 - 1832). BINARY CYCLIC CODES AND BCH CODES polynomial z2 where i = 41 + 1is irreducible. However, over the compZexfield,it splits into (z + i) (z - i), a. Thus the complex field is the splitting field of the real field! 3.2.1 GF(2") arithmetic It can be shown, with basic abstract algebra [PW, LC] concepts, that, in the field of binary numbers, any polynomial of degree m can be split over GF(2m).For the purposes of this book, it is sufficient to learn basic computational aspects of finite fields. Serious readers are urged to study a good textbook on abstract algebra4. Decoding with GF(2") arithmetic allows replacement of complex combinatorial circuits with practical processor architectures that can solve Equation (3.8) as a set of linear equations. In the following, the necessary tools to solve equations involved in decoding of cyclic codes are introduced. Important propertiesof GF(2") The field GF(2") is isomorphic (with respect to "+") to the linear space (0, l}". In other words, for every element /3 E GF(2"), there exists a unique m-dimensional binary vector up E {O,l}m. There is aprimitive element a E GF(2m),such that every nonzero element ,B in GF(2") can be expressed as p = d , 0 5 j 5 2" - 2. This element a is the root of an irreducible polynomial, called a primitive polynomial, p ( z ) over (0, l}, i.e., p ( a ) = 0. A primitive element a of the field GF(2m) satisfies the equation a2m-1 = 1, and n = 2" - 1 is the smallest positive integer such that an = 1. Example 27 Let p ( z ) = z3 + z + 1, a primitive polynomial of GF(23).Let (Y be a primitive element such thatp(a) = a3 + Q + 1 = 0 and a7 = 1.The table below showsthree different ways to express, or representations of, elements in GF(23). + i Power a3 a4 Polynomial Vector 1 00 l 010 100 01 1 110 111 101 CY Q2 l+ff a +a2 ff5 1+ff+ff2 a6 1 +a2 When adding elements in GF(2m),the vector representation is the most useful, because a simple exclusive-or operation is needed. However, when elements are to be multiplied, the power representation is the most efficient. Using the power representation, a multiplication becomes simply an addition modulo 2m - 1. The polynomial representation maybe appropriate when making operations modulo a polynomial. An example of the need of this polynomial representation was seen in the discussion of shortened cyclic codes, where the value of zn-' mod g(z) was required. The author likes [Her]. THE ART OF ERROR CORRECTING CODING 42 In the power representation, because a2m-1 - 1 holds, note that a2" = aa2m-1- a , = a2Cy2m-1 - a2,and so on. This is to say that the powers of a are to be computed a2m+l modulo 2m - 1. Applying the same argument shows that a-' = = CY^"-^. For Example 27 above, a-' = = a6. In general, theinverse p-' = ak ofan element p = ae is found by determining the integer IC, 0 5 IC < 2m - l such that ~ y =~1,+which ~ can be expressed as t? IC = 0 mod (2m - 1). Therefore, C = 2m - 1 - IC. Also, in the polynomial representation, the equation p ( a ) = 0 is used in order to reduce the expressions. In Example 27, a3 = a3 0 = a3 (a3 a 1) = a 1. + + + + + + Log and anti-log tables A convenient way to perform both multiplications and additions in G F ( 2 m ) is to use two look-up tables, with different interpretations of the address. This allows one to change between polynomial (vector) representation and powerrepresentations of an element of GF(2m). The anti-log table A ( i ) is useful when performing additions. The table gives the value of a binary vector, represented as an integer in natural representation, A ( i ) ,that corresponds to the element a i . The log table L ( i )is used when performing multiplications. This table gives the value of a power of alpha, that corresponds to the binary vector represented by the integer i. The following equality holds: aL(i) = A ( '). The best way to understand how to use the tables in the computation of arithmetic operations in G F ( 2 m ) is through an example. Example 28 Consider tables are the following: Address i 0 1 2 3 4 5 6 7 with p ( @ ) = a3 + a + 1, and a7 = 1. The logand GF(2m)-to-vector Anti-log table, A ( i ) 1 2 4 3 6 7 5 0 anti-log Ve~tor-to-GF(2~) Log table, L ( i ) -1 0 1 3 2 6 4 5 + + Consider the computation of an element y = a(a3 a5)3in vector form. Using the properties of GF(23), y can be computed as follows: a3 a5 = 110 111 = 001 = a2. Thus, y = ( ~ ( a =~ ) ~ = cy7(= 1). On the other hand, using the log and anti-log tables, the computation of y proceeds as follows: y = A ( L ( A ( 3 )CB A ( 5 ) )* 3 1) = A ( L ( 3 CB 7 ) * 3 1) = A ( L ( 4 )* 3 l) = A(2 * 3 1) = A(7) = ( A ( 0 )= 1).In the last step, use was made of the fact that a7 = 1. + + + + Anti-log and log tables are used in performing addition and multiplication over GF(2"). Computer programs are available on the ECC web site for simulating encoding and decoding algorithms of BCH and Reed-Solomon codes, with arithmetic in GF(2m).These algorithms are described in subsequent sections. BINARY CYCLIC CODES AND BCH CODES 43 More propertiesof GF(2m) The minimal polynomial &(x) of an element aiis the smallest degree polynomial that has aias a root. The following properties regarding minimal polynomials can be shown. The minimal polynomial 4i(x) has binary coefficients and is irreducible over GF(2) = (0; l}. Moreover, q5i(x) has roots ai, . . ., a 2 " - l i , where K divides m. These elements are known as the conjugate elements of aiin GF(2m).The powers of the conjugate elements form a cyclotomic coset (see [MS], p. 104, and [GG], p. 391): ci =A { a;. 2i: 4 ; . . . 2"-li}. Cyclotomic cosets (also called cycle sets in [PW], p. 209) have the property that they partition the set Z2m -1 of integers modulo 2m - 1.That is, cyclotomic cosets are disjoint, that is, their intersection is the empty set Ci n Cj = 8, i # j , and and the union of all cyclotomic cosets U iCi = & m - 1 . Example 29 The cyclotomic sets modulo 7 are: CO = (0) = {1;2;4} = {3,6,5} c1 C, The primitive element Q of GF(2m)satisfies the equation = 1, and all elements can be expressed as powers of Q. From this it follows that the polynomial 1) factors over the binary field as + M + +ej (x); (x2m-1 1) = j=O and splits completely over G F ( 2 m )as + n 2m-2 (x2m-1 1) = (x + a j ) . (3.9) j=O The order ni of an element ,!? = aiof GF(2m)is the smallest positive integer such that pni = 1. The order ni of every element in GF(2m)divides 2m - 1. Importantly, the degree of a minimal polynomial q!~i(x) is equal to the cardinality (number of elements) of the cyclotomic coset Ci, deg [4i(Z)l = ICil. + This suggests the following method of finding all factors of (z2m-1 1): 1. Generate the cyclotomic cosets modulo 2m - 1. 2. For each cyclotomic coset C,, compute the minimal polynomial +,(x) as the product of linear factors (x - a i s ) where , is E C,, $,(x) = n + (x ai.). (3.10) i, CC, This method can be used in computing the generator polynomial of any cyclic code of length n = 2m - 1. It is used in the computer simulation programs for BCH codes available on the ECC web site, to compute the generator polynomial given the zeros of the code. 44 THE ART CORRECTING OF ERROR CODING + Example 30 Consider with p ( z ) = z3 + z 1. The roots of each of the factors of the polynomial x7 1are shown in the following table. The reader is invited to verify thatin fact the products of the linear factors in (3.10) give the resulting binary polynomials. + C S CO = (0) C1 {1:2:4) C3 = (3; 6: 5 ) Conjugate elements l a,a2,a4 Minimal polynomial, 4s(x) #o(x) = z 1 + 41(z) = 2 3 + 2 + 1 43(x) = 2 3 a3,a6,a5 + x2 + 1 3.3 BinaryBCHcodes BCH codes are cyclic codes that are constructed by specifying their zeros, i.e., the roots of their generator polynomials: + A BCH code of dmin 2 2td 1 is acycliccode whose generator polynomialg(x) has 2td consecutive roots ab,ab,ab+',a b + 2 t d - 1 . Therefore, a binary BCH (n;k:&in) g(%) code has a generator polynomial = LCM{4b(x):dbfl (x):". :4 b+2td-l(Z)}: length n = LCM{nb; nb+l;. . . nb+2td-1}, and dimension k = n - deg [g(.)]. A binary BCH code has a designed minimum distance equal to 2 t d 1. However, it should be noted that its true minimum distance may be larger. + Example 31 With G F p 3 ) ,p(x) = x3 + x + 1,t d = 1 and b = 1,the polynomial g(z) = LCM{41(z):42(2)) = x3 + x + 1: generates a binary BCH (7,4,3) code. (This is actually a binary cyclic Hamming code!) Note that the Hamming weight of g(.) is equal to 3, so that - in this case, but not always - the designed distance is equal to the true minimum distance of the code. Example 32 Consider GF(24),p ( z ) = x4 + z + l, with t d = 2 and b = 1.Then generates a double-error-correcting binary BCH (15,7,5) code. Example 33 With GF(24),p ( z ) = x4 + x + 1, t d = 3 and b = 1, the polynomial s(z) l.( = -=M{41 = (x4 (x2 z - d ; 4 3 b l 4 5 (x)) + + q z 4 + + 2 2 + + 1) + + 1) o + z8 + z5 + x4 + x2 + z + 1 x3 generates a triple-error-correcting binary BCH ( l 5,5,7) code. BINARY CYCLIC CODES BCH AND CODES 45 The lower bound on the minimum distance of a BCH code, known as the BCH bound, is derived next. This is useful not only to estimate the error correcting capabilities of cyclic codes in general, but also serves to point out particular features of BCH codes. Note that the elements a b : . . . a b + 2 t d - 1 are roots of the generator polynomial tj(x), andthatevery codeword ij in the BCH code is associated with a polynomial B(x) which isa multiple of tj(x). It follows that ij(z) E C U ij(ai)= 0; b 5 i b 2 t d . (3.1 1) + Codeword V then satisfies the following set of 2 t d equations, expressed in matrix form, based on (3.1 1): v , - ~) = 0. (3.12) Consequently, a parity-check matrix of a binary cyclic BCH code is given by 1 1 ... ab Qb+l (3.13) 1 Q b + 2 t (dQ- 1b + 2 t d - 1 ) 2 .. . (&2td-1 ) - 1 This matrix H has the characteristic that every 2 t d x 2 t d submatrix (formed by an arbitrary set of 2 t d columns of H ) is a Vandermonde matrix (see, e.g, [GG], p. 95). Therefore (see Section 2.1), any 2 t d columns of H are linearly independent, from which it follows that the minimum distance of the code is d 2 2 t d 1. ([PW] p. 270, [MS] p. 201, and [LC] p. 149.) Another interpretation of the results above is the following: + (BCH bound.) If the generator polynomial g ( x ) of a cyclic (n;k: d ) code has t consecutive roots, say ab ,..., . then d 2 2 t 1. + 3.4 Polynomialcodes The important class of cyclic polynomial codes includes cyclic RM codes, BCH and ReedSolomon codes, and finite-geometry codes [KLP, PW, LC]. Polynomial codes are also specified by setting conditions on their zeros: Let Q be a primitive element of GF(2mS).Let S be a positive integer, and b a divisor of 2s - 1. Then ah is a root of the generator polynomial g ( x ) of a p-th order polynomial code if and only if b divides h and min Wz. ( h 2 ' ) = j b : with 0 < j <]:[ - p: where for an integer i, i = Cki' ie2se, W,=( i ) is dejned as the 2'-ary weight of integer i, m-l i=O 46 THE ART OF ERROR CORRECTING CODING According to this definition, both BCH and Reed-Solomon codes are polynomial codes with b = m = 1. Reed-Muller codes are subcodes of polynomial codes with S = 1. Finitegeometry codes ([LC], Chapter 8) occur as dual codes of polynomial codes [PW]. Below, following [LC], the specifications of the zeros of finite-geometry codes are presented. Euclidean geometry (EG) codes Let Q be a primitiveelement of GF(2mS).Let h be a nonnegative integer less that 2"' - 1. Then ah is aroot of the generatorpolynomialg(x)of a ( p ;s)-orderEG code of length 2mS- 1 if and only if 0 < max Wp(hce)) 5 ( m - p - 1)(2' - 1); ose<s where h(e)= 2eh modulo 2ms - 1. For S = 1, EG codes become cyclic M*,,@ codes, and therefore EG codes are regarded as generalized RM codes. Projective geometry (PG) codes Let a be a primitive element of GF(2(m+1)S).Let h be a nonnegative integer less that 2(m+1'" - 1. Then ah is a root of the generatorpolynomial g(x) of a ( p :s)-order PG code of length n = (2ms - l)/(2' - 1) ifand only $h is divisible by 2' - 1and where h(') = 2eh modulo 2mS - l, and 0 5 j 5 m - U. 3.5 Decoding of binary BCH codes The main idea in decoding binary BCH codes is to use the elements p E G F ( 2 m ) to number the positions of a codeword (or, equivalently, the order of the coefficients of the associated polynomial). This numbering illustrated in Figure 20 for a vector F = ( ro r1 . . . rnP1) with corresponding F(x). positions 1 a a n-l Figure 20 Codeword position numbering using elements of GF(2"). Using G F ( 2 m ) arithmetic, the positions of the errors can be found, by solving a set of equations. These equations can be obtained from the error polynomial E(x) and the zeros of the code, a j , for b 5 j 5 b 2td - 1, as shown in the following. + + Let F(z) = G(z) e ( x ) represent the polynomial associated with a received codeword, where the error polynomial is defined as ~ ( z= ) ej, xj1 + ej, xjz + . . . + ej, xju (3.14) 47 BINARY CYCLIC CODES AND BCH CODES Let the error locatorpolynomial be defined as n(1+ Y A .(x) = aj+) =1 + .1x + 02x2 + . . . + c7,zY, (3.15) e= 1 with roots equal to the inverses of the error locutions. Then the following relation between the coefficients of .(x) and the syndromes holds (see, e.g., [Pet], [LC] p. 154, [PW] p. 284): Solving the key equation given by (3.16) constitutes the most computationally intensive operation in decoding BCH codes. Common methods to solve the key equation are: Berlekamp-Mussey algorithm(BMA) The BMAwas invented by Berlekamp [Berl] and Massey [Mas2]. This is a computationally efficient method to solve the key equation, in terms of the number of operations in GF(2"). The BMA is a popular choice to simulate or implement BCH and RS decoders in software. Euclidean algorithm(EA) This method to solve the key equation, in polynomial form, was introduced in [SKHN] and further studied in [Man]. Due to its regular structure, the EA is widely used inhardware implementations of BCH and RSdecoders. Direct solution Proposed first by Peterson [Pet], this method directly finds the coefficients of ~ ( z ) , by solving (3.16) as a set of linear equations. The term PGZ (Peterson-GorensteinZierler) decoder is often used in the literature. This is because in [GZ] Peterson's method is applied in decoding nonbinary BCH codes, or RS codes. Naturally, as the complexity of inverting a matrix grows with the cube of the error-correcting capability, the direct solution method worksonly for small valuesof t d . Solutions to (3.16) up to t d 5 6 are given in [ML], sec. 5.3. ~~ It will be seen later that for Reed-Solomon codes e j E G F ( 2 m ) . 48 3.5.I CODING THE ART CORRECTING OF ERROR General decoding algorithmfor BCH codes Figure 21 shows the block diagram of a decoder for BCH codes (both binary and non-binary). The decoder consists of digital circuits and processing elements to accomplish the following tasks: 0 Find the coefficients of the error locatorpolynomial .(x) Find the inverses ofthe roots ofo(z), i.e., the locations of the errors, a31,. . ., aJ' Find the values of the errors ejl, . . e j , . (Not needed for binary codes.) Correct the received word with the error locations and values found. e, 0 POWER SUMS EA, 0 ) Compute BMA SEARCH CHIEN ALGORITHM Find positions - FORNEY Find error values I c(x) Delay RAM Figure 21 Architecture of a BCH decoder with GF(2") arithmetic. One of the advantages gained in introducing GF(2m) arithmetic is thatthe decoding operations can be implemented with relatively simple circuitry and processing elements. As an example, Figure 22 shows how the computation of a syndrome Sj can be implemented in hardware. The multiplier is over G F ( 2 m ) ,and can be constructed with relatively simple combinatorial logic. Some details on the hardware design of processing elements over G F ( 2 m )can be found, for example, in [PW], and Chapters 5 and 10 of [WB]. b+j- 1 I Figure 22 Example of a circuit for computing a syndrome. BINARY CYCLIC CODES AND BCH CODES 49 3.5.2 The Berlekamp-Mussey algorithm (BMA) The BMA is best understood as an iterative procedure to construct a minimum linear feedback shift-register (LFSR) structure, like the one shown in Figure 23, that reproduces the known syndrome sequence S1 ,S2 , . . . , S2td. j = v+l, v+2, ..., 2v Figure 23 LFSR with taps c n , 02, ... U” and output SI,s2, ..., S2,. The goal of the BMA is to find a (connection) polynomial dif1)(x) of minimal degree that satisfies the following equations, derived from (3.16): c Sk-jffjifl) = 0, e2 < IC < i + 1. (3.18) j=O + This is equivalent to requiring that d Z + l ) (x)= 1 o $ ~ + ’+ ) z. . . + o ~ ~ ~ l l ) xbe f i +the l LFSR connection polynomial that produces the partial sequence of syndromes. The discrepancy at iteration i defined as measures how wellthe LFSR reproduces the syndrome sequence, and constitutes a correction term usedin computing the value ofin the next iteration. There are two cases to consider in the algorithm [Pet]’: 0 If di = 0, then the equations (3.18) are satisfied for a(Z+l)(x)= oi(x), e,+l = ei. If di # 0: Let d m ) (x)be the solution at iteration m, such that -1 5 m and ( mis maximal. Then e,) (3.19) i, dm # 0, There is a variation of the BMA, inspired by [Mas2], that is used in some references. This variation will be described in the next chapter. Of course, both versions of the BMA will give the same result! THE ART CORRECTING OF ERROR 50 Withan CODING initial value of i = 0, the computation of dZ+')(x) continues until either - 1 or i = 2td - 1, or both conditions, are satisfied. i 2 ti+, + t d The initial conditions of the algorithm are: @)(x) e-, = 1, (J(O)(z)= 1, = 0, eo = 0, = 1, do = S,. d-1 (3.21) Also note that the BMA has control flow instructions (if-else). For this reason, it is not favored in hardware implementations. However, in terms of number of GF(2") operations, it is very efficient. This version of the algorithm is implemented in most of the C programs to simulate BCH codes that can be found on the ECC web site. Example 34 Let C be the triple-error correcting BCH (15,5,7) code of Example 33. As a reference, to check the numerical computations, the power and vector representations of GF(24),with primitive polynomial p ( z ) = 1 x x4,are listed below: + + , ( z ) = x4 Table of elements of G F p 4 ) p Power 0 1 a a2 a3 a4 a5 cy6 cy7 cy8 a9 a10 cy11 Q12 a13 QI4 +z +l Vector 0000 000 1 0010 0100 1000 0011 0110 1100 1011 0101 1010 0111 1110 1111 1101 1001 + + + + + + + + + + + + + + + + + + + + + + + + A generator polynomial for C is g ( x ) = d o + x8 x5 x4 x2 z 1. Suppose that the information polynomial is U(x) = x x2 x4. Then the corresponding code polynomial is given by V ( x )= x x2 x3 x4 x8 x" d 4 . Let T(z)= 1 x + x 2 x3 x4 x6 x8 2'' be the polynomial associated with a vector F = V E received after transmission of codeword V over a BSC channel. (Vector E corresponds to the error polynomial e(z) = 1 x6 d 2 . Obviously, the decoder does not know this. However, in computing the syndromes below, this knowledge is used to simplify expressions.) The syndromes are: S, S2 = = r ( a ) = 1 + Q6 + a12 = Q S; = a2 BINARY CYCLIC CODES AND BCH CODES 51 S3 = r(a3) = 1 + a3+ a6 = a8 S, = S; = CY4 S5 S6 m Iteration 4: = F(a5)=1+1+1=1 = S: = a i = 3, d3 = 0, a(4)(x)= a(3)(x) = 1 e4 = e3; + ax + a12x2; C4f3-153 ?NO: d4 = S5 S4a14) S3aa4) = 1 + + 0 + a 4 ( a )+ a8a1' = 1. Iteration 5: i = 4, d4 = 1 # 0, m = 2 maximizes (2 - 1) = 1for d2 # 0. + + + + + ~ ( ~ ) (=x ) d4dz1x(4-2)a(2)(~) = (I (1)(a13)-1z2(1 = 1 ax a7x2 a3x3: e5 = max(C4; e, + 4 - 2} = 3; e 5 + 3 - 1 g ?NO: d5 = S6 i- S5a15) S4ap) S3aF) = a 1 . a a4(a7) a8(a3) = 0. + + 0 + + + + + Iteration 6: i = 5, ds = 0, @)(x) = a(5)(x) = 1 + ax + a7x2+ a3z3; e, = e 5 = 3; + 3 - 15 3 ? Yes : End. + THE ART OF ERROR CORRECTING CODING 52 Therefore a ( z ) = 1 + a z + a7z2+ a3z3. That the odd numbered iterations always gave di = 0 in the previous example is no accident. For binary BCH codes this is always the case. The iterations can proceed with only even values of i, with the same results. The only difference is that the stopping criterion needs to be changed to i 2 t?i+a t d - 2. + As a result, decoding complexity is reduced. The reader is invited to previous example, using only three iterations. solve a(.) for the 3.5.3 PGZ decoder This decoding algorithm was first considered by [Pet]. A solution to the key equation (3.16) is to befound using standard techniques for solving a set of linear equations. This solution gives the coefficients of .(x). The decoding problem is that the number of actual errors is unknown. Therefore, a guess has to be made as to the actual number of errors, v, in the received word. Assume that not all the syndromes, Si, 1 5 i 5 2t, are equal to zero. (If all syndromes are zero, then the received word is a codeword and decoding finishes!) The decoder assumes that the maximum number of errors has occurred, v, checks if the determinant Ai, f o r i = v, =id, = t d , and (3.22) is equal to zero. If it is, then a smaller number of errors must have occurred. The value of i is decreased by one, and Ai tested again, repeating the procedure if necessary, until i = 1. Otherwise, if Ai # 0, the inverse of the syndrome matrix is computed and the values of a1;a’; . . . are found, where U = i. In the event that Ai = 0: i = 1;2; . . . t d ,decoding is unsuccessful and an uncorrectable error pattern has been detected. ~ Example 35 In this example, the error locator polynomial for the BCH (15; 5 ; 7) code in Example 34 is found by the PGZ decoding algorithm. First, assume that i = t d = 3 errors occurred. Then the determinant A3 is computed (using cofactors) as follows: A3 = det - (S z: :) = a(a8+ a 8 )+ a 2 ( a 2+a1’)+ a 8 ( a 6 + a ) a8 a 4 a2+7 + a8+11 - -a 14 . Therefore A3 # 0 and three errors are assumed to have occurred. Substituting the syndromes found in Example 34 into the key equation (3.16), (3.23) BINARY CYCLIC CODESAND BCH CODES 53 Note that A;' = a-14= a. The solution to (3.23) is found to be: a4 a2 a8 a a a4 a4 l = a(a7 + a 1 2 ) = a3; a8 = a(a11 +a>= a 7; =a(al5 + It follows that .(x) = 1 ax the BMA in example 34. + a7x2 + @'x3, + a 1 5 + a 1 5 ) = a. which is the same result as that obtained by 3.5.4 Euclidean Algorithm (EA) This algorithm is a well-known recursive procedure to find the greatest common divisor (GCD) betweentwo polynomials (or integers). Its application to BCH decoding is described next. Let an errorevaluatorpolynomial bedefined as A(.) = a ( z ) S ( x ) ;with a syndrome polynomial S ( x )= 1 S l Z . .. S2td52? (3.24) + + + From Equation(3.16), it follows that A(x) = a ( x ) S ( x )mod x 2 t d + 1 . (3.25) The decoding problemcan be translated into finding the polynomial A(x) satisfying (3.25). This can be achieved by applying the extended EA to the polynomials ro(z) = x 2 t d f 1 and r1 (x)= S ( x ) ,such that if at the j-th step r j ( x )= a j ( x ) x 2 t d + l + bj(Z)S(Z); with deg [ r j ( x ) ]5 t d , then A(x) = r j ( x ) and oi(x)= bj(x).(Note that there is no interest, from the decoding viewpoint, inpolynomial ai (x).) The extendedEuclidean algorithm for computing the GCD of two polynomials is: 54 THE ART CORRECTING OFCODING ERROR Stop at iteration j l a s t when deg [rj,,,, (x)]= 0. Then GCD(r0(x);r1 (x))= rk (x),with IC the largest nonzero integer such thatT k (x)# 0 and IC < j l a s t . Example 36 In this example, the error locator polynomial .(x) for the BCH (15; 5;7) code in Example 34 is computed using theEA. Initial conditions TO(Z) = T1(Z) = 27: + S ( x ) = 1 + ax + a2x2 a8x3+ a4x4+ x5 +ax6; = 0; b1(x) = 1. bO(5) 0 j=a: x7 = ( l + a ~ + a ~ x ~ + a ~ x ~ + a ~ x ~ + x ~ + a x ~ ) ( a ~ ~ x + a ~ ~ ) + a ~ x ~ + + + + + = a8x5 a12x4 a11x3 0 1 3 : q2(x) = a14x + a 1 3 ;and b2(x) = bo(.) q2(x)b1(x) = a145 a13. ?-2(x) 0 + j=3: S(Z) = ( a 8 x 5 + a 1 2 x 4 + a 1 1 x 3 + a 1 3 ) ( a ~ x + a+a14x4+a323+a2x2+a112. 2) + + = d 4 x 4+ a3x3 a2x2 a112; q 3 ( x ) = a 8 x + a 2 ;and b3(x) = b1(x) q3(x)b2(2) = a7x2 allz. T3(Z) + + 0 j=4: a8x5+a12x4+a11x3+a13 = (a14x4+a3x3+a2x2+a115)(a9x)+a5x+a13. a5x + a 1 3 ; T4(Z) = q4(x) = a9x,and = b z ( x ) q4(x)b3(x) = ax3 b4(Z) + + a5x2+ a14x + Q l 3 . Because deg[r4(x)] = 1 5 3, the algorithm stops. It follows that .(x) = b 4 ( 5 ) = d 3 ( l ax a7x2 a 3 x 3 ) which , has the same roots as the polynomial obtained by the BMA and PGZ decoder (see example 37 below), and differs only by a constant factor8. As the example above illustrates, generally the error-locator polynomial obtained with the EA will differ from that obtained by BMA or PGZ decoding by a constant factor. In decoding, however, interest is in the roots of these polynomials and not their coefficients, and thus any of the three methods discussedso far can be used to determine .(x). + + + The normalized polynomial unorm(z) = a ; l a ( x ) is identical to the one obtained in the BMA and PGZ procedures. 55 BINARY CYCLIC CODES AND BCH CODES 3.5.5 Chien search and error correction To find the roots of .(x), a simple trial-and-error procedure - called Chien search - is performed. All nonzero elements p of G F ( 2 m )are generated in sequence 1, a, a 2 , ... and the condition .(p-’) = 0 tested. This process is easy to implement in hardware. Moreover, finding roots (factoring) of polynomials over GF(2m)is achallenging mathematical problem that remains to be solved. For binary BCH codes, once the error locations j , , ..., j , are known, the corresponding bits in the received word are complemented and the estimated codeword ).(G generated. Example 37 Continuing with Example 34, the roots of .(x) a-12. Stated in a different way, c(.) factors as are 1, a’ = a-6 and a3 = .(x) = (1 + 2 ) ( 1 + a62)(1 +a122). Consequently, the estimated error polynomial is C(.) =1 + x6 + .l2, and Three errors have been corrected. 3.5.6 Errors-and-erasuresdecoding There are many situations inwhich a decision on a received symbol is not considered reliable. An example is binary transmission over an AWGN channel, with bits mapped to real amplitudes, e.g., BPSK with 0 e + l and 1 e -1. If the received values are too close to zero, then it may be more advantageous, from the viewpoint of minimizing the probability of a decoding error, to declare a “no-decision”. In such a case, the received symbol is “erased” and it is called an erasure9. Declaring erasures is the simplest form of soft-decision, which will be thefocus of attention in Chapter 7. Introduction of erasures has the advantage, with respect to errors-only decoding, that the positions are knownto the decoder. Let d be the minimum distance of a code, v be the number of errors and p be the number of erasures contained in a received word. Then, the minimum Hamming distance between codewords is reduced to at least d - p in the non-erased positions. It follows that the error-correcting capability is L(d - p - 1)/2J and the following relation holds d > 2v p . (3.26) + The above inequality is intuitively satisfying: For a fixed minimum distance, it is twice as difficult to correct an error than to correct an erasure, because the erased positions are already known. For binary linear codes, including binary BCH codes, erasures can be corrected with the following method: In informationtheoreticalterms,the channel (BEC) [COT]. BSC channel becomes atwo-inputthree-output binary erasure 56 THE ART CORRECTING OF ERROR CODING 1. Place zeros in all erased positions and decode to a codeword 60(x). 2. Place ones in all erased positions and decode to a codeword 61 (x). 3. Chose as decoded word the closest 6j(x)to the received wordF(x) in thenon-erased positions, j = 0 , l . (Alternatively, the codeword that required the smallest number of error corrections [ML].) As a result, erasures can be corrected with two rounds of errors-only decoding. + + Example 38 Consider a cyclic Hamming ( 7 , 4 , 3 ) code with g(x) = 1 x x3, and GF(23) with a primitive element a such that p ( a ) = 1 a a3 = 0. Suppose that V ( x )= x + x2 + x4 is transmitted and that p = 2 erasures are introduced at the receiver. Since d = 3 > p, these erasures can be corrected. Let F(x) = f x x2 f x 3 x4 be polynomial associated with the received vector, wheref denotes an erasure. + + + + + + First decoding (f = 0): PO(.) S, = Sf= 0. Therefore, CO(.) + + S1 = Fo(a)= 0, and + + 1+ x + ' x + x3 + x4, S1 = a and S2 = a'. = x x2 x4. The syndromes are = Fo(x) = x x2 x4. Second decoding (f = 1): F1 (x) = Equation (3.16) inthis case is: a01 = a2. Therefore .(x) = 1 +ax, E ( x ) = x and Cl(x)= Fl (x) E(x) = 1 'x x3 x4,which differs from F(x)in one of the non-erased positions. As a result, GO(.) is selected as the decoded word. Two erasures havebeen corrected. + + + + 3.6 Weight distribution and performance bounds In general, the weight distribution of a binary linear ( n ,k) code C can be obtained by enumerating all 2k code vectors V and computing their Hamming weight. Obviously, for large values of IC, this is a formidable task. Let A, denote the number of codewords of Hamming weight W. The MacWilliams identity relates the weight distribution sequence (WDS) of a A linear code, A(%)= A0 Alx 22x2 + . . . A,z", with the WDS of its dual l o ( n ,n - k) A code C l , B ( x ) = Bo Blx xzx' . ' . &x", by + + + + + + + A ( x ) = 2-"+'(1 + x)"B [:I:]' - (3.27) which can also be expressed as (3.28) Therefore, for high-rate codes, it is simpler to compute the WDS B ( x ) of the dual code and then use(3.27) to compute A ( z ) .Alternatively,the WDS of a low-rate code is easy tocompute and the WDS of the dual code can be obtained using (3.28). For some classes of codes, the trellis structure can be used". In [DFK2], a trellis-based method for computing the weight distribution of extended BCH codes of length 128 is given. that the dual code CL has generator matrix H , where H is the parity-check matrix of C More on trellises can be found in Chapter 7. lo Recall 57 BINARY CYCLIC CODES AND BCH CODES For shorter lengths, the weight distributions are not difficult to compute using MacWilliams identity. The definition of extended cyclic code is introduced next. A binary extended cyclic code is obtained from a binary cyclic code by appending, at the start of each code vector, an overallparity-check bit.The extended code of a cyclic code is no longer cyclic. Let H denotes the parity-check of the cyclic code, then the parity-check matrix of the extended code, denoted Hextis given by 1 1 ... Hext = (n H l). (3.29) Appendix A lists the weight distributions of all extended binary BCH codes of length up to 128. These data files are available on the ECC web site. The appendix lists only those terms of Hamming weight up to (n 1)/2, for n = 2m - 1. In the case of extended BCH codes, it holds that = A?). The data is useful in finding the weight distribution of the binary cyclic BCH codes of length up to 127. This can be done by application of the following result (which is obtained as a special case of Theorems 8.14 and 8.15 in [PW]): + Let C be a binary cyclic BCH (n:IC) code withWDS A(z), obtained by eliminatingthe overall parity-check bit in a binary extended BCH ( n 1;IC) code Cext,with WDS A(ext)(z).Then, for W even, + (3.30) Example 39 Consider the binary extended Hamming (8: 4; 4) code. This code has paritycheck matrix (see also Example 23), H= ( 1 1 1 1 1 1 1 1 0 1 1 1 0 1 0 0 0 0 0 1 1 1 0 1 + It can be easily verified that A(ext)(x)= 1 14z4 cyclic Hamming (7; 4; 3), use (3.30) to obtain 8A3 = 4 A y ) 4A4 = (8-4)A3 8A7 = 8 A F ) + z8.To compute the WDS of the binary + + + A3 7; A4 1 7 ; A7 = 1. 3.6.1 Errorperformanceevaluation With knowledge of the WDS of a code C, the error performance can be estimated, as discussed in Chapter l. The WDS and the union bound (1.34) give good estimates of the error performance of code C with binary transmission over an AWGN channel. As an example, the union bound was evaluated using Appendix A for extended BCH codes of length 8 to 64.The results are shown inFigures 24 to 27. Using the WDS from the appendix 58 THE ART CORRECTING OFCODING ERROR and the union bound for flat Rayleigh fading channels (1.40), with Monte Carlo integration and the term A, replaced by wA,/n to account for bit errors, Figure 28 shows union bounds on the bit error rate for extended BCH codes of length 8. Bound for other codes can be computed in the same way. The main point of this section is that, before choosing a particular soft-decision decoding algorithm (see Chapter 7), the weight distribution and the union bound can give an estimate of the performance that is achievable over a certain channel. For theAWGN channel and the flat Rayleigh fading channel, the union boundsare tight at biterror rates below 10W4. Union bounds on bit error rate for extended BCH codesof length 8 1e+OO 1e-02 X -5-4 -3-2-1 0 1 2 3 4 5 6 7 8 9 101112 Es/No (dB) Figure 24 Union bounds on the BER for extended BCH codeof length 8. Binary transmission over an AWGN channel. BINARY CYCLIC CODES AND BCH CODES Union bounds on bit error rate forextended BCH codes of length 16 Figure 25 Union bounds on the BER for extended BCH codeof length 16. Binary transmission over anAWGN channel. Union bounds on bit error rate for extended BCH codes of length 32 1e+OO 1e-02 1e-04 U W m 1e-06 1e-08 1e-l 0 le-12 EdNo (dB) Figure 26 Union bounds on the BER for extended BCH code of length 32. Binary transmission over anAWGN channel. 59 THE ART OF ERROR CORRECTING CODING 60 Union bounds on bit errorrate for extended BCH codes of length 64 Figure 27 Union bounds on the BER for extended BCH code of length 64. Binary transmission over anAWGN channel. Union bounds for extended BCH codes of length 8 over a flat Rayleigh channel 0.1 "ber.ray.8.7.2" -i- "ber.ray.8.4.4 - - - x - - - ~ "ber.ray.8.18 ' ' 0.01 a: W m 0.001 - 0.0001 1e-05 0 1 2 3 4 5 6 7 8 9 W N o (dB) 1 0 1 1 1 2 1 3 1 4 1 5 Figure 28 Union bounds on the BER for extended BCH codeof length 8. Binary transmission over a flat Rayleigh fading channel. The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) Non-binary BCH codes: Reed-Solomon codes In this chapter the most celebrated class of ECC schemes is introduced and their encoding and decoding algorithms explained. Reed-Solomon (RS) codes have found numerous applications in digital storage and communication systems. Examples include the famous RS (255,223,33) code for NASA space communications, shortened RS codes over G F ( 2 8 )for CD-ROM, DVD and Terrestrial Digital HDTV transmission applications, an extended RS (128,122,7) code over G F ( 2 7 )for cable modems, among many others. 4.1 RS codes as polynomial codes Similar to Reed-Muller codes, RS codes can be defined as codewords with components equal to the evaluation of a certain polynomial. As a matter of fact, this was the way that RS codes were originally defined by Reed and Solomon in [RS]. RM codes, finite-geometry codes [LC] and RS codes are all members of a large class of codes: PoZynomial codes [PW], which are closely related to algebraic-geometry (AG) codes [Pre]. Let u(x) = 210 + U12 + . . . + Uk-IZk-1 (4.1) be an information polynomial, with ui E GF(2"), 1 5 i < IC. Clearly, there are 2mk such polynomials. By evaluating (4.1) over the nonzero elements of G F ( 2 " ) , a codeword in an RS (2" - 1,IC, d ) code of length 2" is obtained, v = (u(1) u(a) U(Q2) . . . u(Q2m-2) ) . (4.2) 4.2 From binary BCH to RS codes RS codes can also be interpreted as nonbinary BCH codes. That is, RS codes are BCH codes in which the values of the code coefficient are taken from GF(2"). In particular, for a td-error correcting RS code, the zeros of the code are 2td consecutive powers of Q. Moreover, because over G F ( 2 " ) minimal polynomials are of the form &(x) = (x - a'), 0 5 i < 2" - 1, see Equation (3.9), the factors of the generator polynomial are now linear, and THE ART CORRECTING OF ERROR 62 CODING where b is an integer, usually b = 0 or b = 1. It follows from (4.3), and the BCH bound on page 45, that the minimum distance of a ReedSolomon (n,IC, d ) code C over GF(2") is d n - k 1. On the other hand, the Singleton bound [Sin] d 5 n - IC 1 implies that d = n - k 1. A code that satisfies this equality is known as a maximum distance separable (MDS) code [Sin]. Therefore RS codes are MDS codes. This gives RS codes useful properties. Among them, shortened RS codes are also MDS codes. > + + + Using the isomorphism between GF(2") and (0, l}", for every m-bit vector 30 there is a corresponding element /3 E GF(2"), m-bits 05j U /3j E GF(2"), < 2" - 1. In other words, m information bits can be grouped to form symbols in GF(2"). Conversely, if the elements of GF(2") are expressed as vectors of m bits, then a binary linear code of length and dimension n = m(2" - l ) and k = m(2" - 1 - 2td), respectively, is obtained. The minimum distance of this code is at least 2td 1. This binary image code can correct, in addition to up to t d random errors, many random bursts of errors. For example, any single burst of up to m ( t d - 1) 1 bits can be corrected. This follows from the fact that a burst of errors of length up to m(q - 1) 1 bits is contained in at most Q symbols of GF(2"). Therefore, there are many combinations of random errors and bursts of errors that an RS code can correct. To a great extent, this is the reason why RS codes are so popular in practical systems. + + + Example 40 Let m = 3, and GF(23) be generated by a primitive element Q with p ( a ) = a3 + cy 1 = 0. Let b = 0 and t d = 2. Then there is an RS(7,3,5) code C with + generator polynomial g(.) = .( = z4 + l)(. +CY)(. + cy2)(. + cy3) + Q2Z3 + + CY5. + Q5.2 Q6. By mapping the symbols of GF(23) into binary vectors of length 3, code C becomes a binary (21,9,5) code that is capable of correcting up to 2 random errors as well as any single burst of up to 4 bits. 4.3 Decoding RS codes The core of the decoding algorithms of RS codes is similar to that of binary BCH codes. The only difference is that the error values, e j , , 1 5 t 5 v, for v 5 t d , have to be computed. In general, this is done using the Forney algorithm [For2]. The expression below holds for RS codes with an arbitrary set of 2td consecutive zeros { a b & , l, . ..a b f Z t d - ' 1 7 where g'(.) represents the formal derivative of a(%)with respect to z. (A similar expression can be found in [RC], p. 276.) The polynomial A(.) in (4.4) is anerror evaluatorpolynomial, which is defined as A(z) = o(z)S(x)mod z2td+1. (4.5) NON-BINARY BCH CODES: REED-SOLOMON CODES 63 Before introducing the first example of RS decoding, an alternative version of the BMA is presented, referred to as the Massey algorithm (or MA). The algorithm was invented by Massey in [Mas2], and is also described in [ML, Wic]. Massey algorithmto synthesize an LFSR 1. Initialize the algorithm with U ( % ) = 1 (the LFSR connection polynomial), p ( z ) = z (the correction term), i = l (syndrome sequence counter), C = 0 (register length). 2. Get a new syndrome and compute discrepancy: e 3. Test discrepancy: d = O? Yes: Go to 8. 4. Modify connection polynomial: 5. Test register length: 21 2 i? Yes: Go to 7. 6. Change register length and update correction term: Let C = i - C and p(z) = g ( z ) / d 7. Update connection polynomial: U ( % ) = gneW(x). 8. Update correction term: p ( z ) = zp(z). 9. Update syndrome sequence counter: i = i 10. Stopping condition: If i + 1. < d go to 2. Else, stop. Example 41 Let C be the same RS(7,3; 5 ) code as in Example 40. Suppose that F(z) = ax2 + a5z4 is the received polynomial. Then S1 = F ( 1 ) = Q s 3 - --~((1: 2 -) 5 a + a6 = a and S, = F(a3)= 1 + a5 = a 6 ,S, = F(a) = a3 + a2 = + a3 = a. Equation (3.16) gives: cy5, (:: un") (2) (:) = Three methods of finding U(.) are shown below. Direct solution(PGZ algorithm) Assume two errors. Then A2 = a7 have occurred and +d o = 1 + a3 = a # 0. Therefore two errors must 64 THE ART OF ERROR CORRECTING CODING from which it follows that a(.) =1 + a2 + a6z2= (1 + a2.)(1 + a 4 z ) . Massey algorithm S 1 = a6: p(.) 0 i = 5 > d. Stop. Euclidean algorithm 0 Initial conditions: S2 = a 5 S3 = CY; ; = .p(.) = ax2. S4 = CY. NON-BINARY BCH CODES: REED-SOLOMON CODES z5 = ( 1 65 + 211:+ a5z2+ a x 3 + a24)(a62+ a6) + a 5 2 3 + 2 2 + a2 + + + + + + = a5z3 2 a2 a6; q2(z) = a 6 2 + Q 6 : b2(2) = 0 (a62 a6)(1) = 2 ?-2(2) a6z2+ a z + a 3 : 3 2 q3(z) = a s + a : b 3 ( 2 ) = l + (a32 + a”(a62 ?-3(2) 2 +2 Q6 . = + 2 ) = a3 + a42+ a2x2. Algorithm stops, as deg[73(2)]= 2 = t d . It follows that a(.) = a3 + a42+ a2x2= Q3(1+ ax + a 6 x 2 ) . In all the above algorithms, the following error locator polynomial is found, up to a constant term: a ( 2 ) = 1 Q2 a 6 2 = ( 1 &%)(l+ a42). + + + Therefore, the error positions are jl = 2 and j 2 = 4.In a computer program or a hardware implementation, Chien search yields these two valuesas the (inverse) roots of .(x). Also note that a’(z) = a. (Because, in GF(2m),2a = a a = 0.) To compute the error values, using either the Berlekamp-Massey algorithm (BMA or MA versions) or the PGZ algorithm, the error evaluator polynomial (4.5) is needed, + A = (1 + as + a 6 z 2 ) ( 1+ a6z + a5s2+ a x 3 + as4)mod x5 = ( 1 + a52+ a 3 2 )mod x 5 ; It is important to note that the Euclidean algorithm computes simultaneously .(x) R(%),as a(.) = bj,ast (x)and A(z) = rjlast(x).To verify this note that a3 ~ 3 ( z= ) + a z + a6z2= a 3 ( l+ a52+ a 3 x 2 )= a3h(s). With the error locations determined, the error values from Equation (4.4) are e2 e4 Therefore, E(z)= ax2 = = (a2)’(1+ a5ap2+ ( ~ ~ a - ~ ) = a Q: -’ ( a 4 ) 2+( a5ap4 i + ( ~ ~ a - * )=a a- 5~, + a5z4and the decoded word is e ( 2 ) = F ( % ) + e(2) = 0. The two errors have been corrected. and THE ART OF ERROR CORRECTING CODING 66 Note that the constant p is the same for both polynomials found by application of the extended Euclidean algorithm. The EA findsp . a ( z )and p .A(z),for some nonzero constant p E GF(2m).Nevertheless, both error locator and error evaluator polynomials have the same roots as those obtained by the PGZ or BMA algorithms, and thus the error values obtained are the same. In most of the computer programs to simulate encoding and decoding procedures of RS codes on the ECC web site, the following equivalent method of finding the error values is used [LC]. Let Then the error value is computed as [Berl] i=l if! where 1 5 e 5 v. Yet another alternative to Forney algorithm, for small values of td, is to determine the error values directly as follows. For l 5 e 5 v, the error values ej, are related to the syndromes Si by the set of linear equations: v si= , q Q b + i - l 1 = C e3r. Q(b+i--l).ir (4.8) e=1 where 1 5 i 5 2td. Each v X v submatrix formed by the (known) terms a(b+i-l)jfforms a Vandermonde matrix. After all the v error locations j e are known, any set of v equations of the form (4.8) can be used to find theerror values. In particular, choosing the first v syndromes: is a system of linear equations that can be solved using GF(2m)arithmetic. Example 42 Consider the same RS code and received polynomial in Examples 40 and 41. Then (4.9) gives: The determinant of the 2 x 2 matrix is A = cy4 e2 = a-’det (E: (t14) + a2 = a. From this it follows that + c y 5 ) = cy6 . a2 = cy, = a6((r3 NON-BINARY BCH CODES: REED-SOLOMON CODES 67 and which are the same error values as those obtained withForney algorithm. Again, it is emphasized that this can only be done efficiently (and practically) for relatively small values of the error correcting capability, t d , of the RS code. 4.3.1 Remarks on decoding algorithms Unlike the BMA, inthe EA all the syndromes are used in the first computation step. However, in terms of the number of G F ( 2 m ) operations, the BMA is generally more efficient than the EA. On the other hand, all the steps in the EA are identical, which translates into a more efficient hardware implementation. Also, the three decoding methods discussed here for (binary and nonbinary) BCH codes are examples of incomplete -or bounded distance decoding. That is, they are able to detect situations in which the number of errors exceeds the capability of the code. There are other approaches to decoding BCH codes, the most notable being the use of a discrete Fourier transform over GF(2m).This is covered extensively in [Blah], where the reader is referred to for details. Recently, Sudan[Sud] has introduced an algorithm that allows correction of errors beyond the minimum distance of the code. It applies to RS codes and more generally to AG codes. This algorithm produces a list of codewords (it is a list-decoding algorithm) and is based on interpolation and factorization of polynomials over GF(2m)and its extensions. Sudan algorithm was improved in [Gur]. 4.3.2 Errors-and-erasuresdecoding For the correction of erasures, the main change to the RS decoding procedures described above is that an erasure locator polynomial .(x) needs to be introduced, defined as e= 1 where yie = a Z L for , 1 5 Q 5 p , denotes the position of an erasure. By definition, the positions of the erasures are known. Therefore, only the erasure values need to be found. This can be done, as before, in the Forney algorithm step. In computing the syndromes of the received polynomial, it can be shown that any values of the erasures can be replaced, without any difference in the decoded word. The decoding procedure is similar to the errors-only RS decoder, with the following exceptions. A modified syndrome polynomial, or modified Fomey syndrome,is formed, + T ( x )= S ( x ) ~ ( x )1 mod x Z t d f 1 . The BMA algorithm can be applied to find .(x) (4.10) with the following modifications: 1. The discrepancy is now defined as (4.11) 68 THE ART CORRECTING OF ERROR CODING with do = Tp+l. 2. The algorithm finishes when the following stopping condition is met: i 2 li+l +td - 1 - p/2. After a(.) is obtained, a modiJed errors-and-erasureevaluator, or errata evaluator, is computed as W(.), W(.) = [l T ( z ) ].(x) mod z2td+1. (4.12) + In addition, the following errata locatorpolynomial is computed, 4(x) = . ( . ) d . ) . (4.13) The resulting errata evaluation, or modiJed Fomey algorithm,is givenby (4.14) 1 1. l 5 v, for the error values, and (4.15) 1 5 l! 5 p, for the erasure values. For errors-and-erasures decoding, the Euclidean algorithm can also be applied tothe modified syndrome polynomial T ( z ) ,using 1 T ( z ) instead of S(%) as in errors-only decoding. That is, the inital conditions are TO(.) = z2td+1and q ( z ) = 1 T ( z ) .The algorithm stops when deg[rj(z)]5 L(d - 1 p ) / 2 J ,with W(.) = r j ( z )and .(x) = b j ( x ) . + + + Example 43 Let C be an RS (15,9,7) code over GF(24) with zeros { a , a 2. ., .a 6 ) ,where a is a primitive element satisfying p ( a ) = a4 + a3 + 1 = 0. As a reference for this example, a table of elements of GF(2*)as powers of a primitive element a, with a4 + a3 + 1 = 0, is shown below. ) Table of elements of G F p 4 ) ,p(.) = x4 Power 0 1 a a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 Ql2 a13 a14 Vector 0000 000 1 0010 0100 1000 1001 1011 1111 0111 1110 0101 1010 1101 0011 0110 1100 + x3 + 1. NON-BINARY BCH CODES: REED-SOLOMON CODES 69 The generator polynomial of C is n(2+ 6 g(2) = ai)= IC6 + al2z5+ x4 + a2x3+ a7x2+ all2 + a 6 . i=l Suppose that the polynomial associated with a codeword V is Let the received polynomial be F(2) = a7 + a32 + a13z2a5211 + a14x3a11212 + a7x4+a13513 ax5 + ax6 + a4z7+ a6x8 +a3z10 a10214. + + + + Assume that, aided by side information fromthe receiver, it is determined that the values in positions ao and a5 are unreliable, and thus declared as erasures. Note that E(z) = + a8z3+ a5z5+ a d 2 is the polynomial associated with the errata.' After reception, besides F(z),the decoder knows that p = 2 erasures have occurred in positions a' and a5.Therefore, it computesthe erasure locator polynomial .(x) = (1+ z)(l+ a 5 4 =1 + a102+ a5z2. (4.16) The syndromesof F(x) are computed as: S1 = F(&) = all, S2 = F(a2)= a5,S3 = F(a3)= a2, S4 ---~ (4 ) a- a2 , S 5 - ~ ( a =~ 1 and ) S6 = F(a6)= Accordingly, + S ( 2 ) = 1+ all2 + a5x2+ a 2 z 3+ a2x4 d5 + a14z6. The Forney syndrome polynomial(4.10) becomes T ( z ) = S ( z ) r ( z )+ 1mod 22td+1 = (1+ a'lz + a5x2+ a 2 x 3+ a2z4)(l+ al0z + a 5 z 2 )+ 1 mod x7 = a72 + a6z2+ a7x3+ a1lz4+ a9z5+ 2 6 , The decoder obviously does not know this, except for the positionsof the erasures. This polynomialis used as a reference against which the correctnessof the decoding results can be verified. THE ART CORRECTING OF CODING ERROR 70 Iteration 2: i = 2, m = 0 maximizes (0 - 0) = 0 for do t3 0 = = max{l,O + d 2 d i 1 c d 0 ) ( 2 ) = 1+ a42+ a10z2. + 2 - O} = 2, Iteration 3: i = 3, m = 2 maximizes (2 - 1) = 1 for d2 4 = max{2,1+ 3 - 2) i 2 2 Yes : Algorithm + # 0. (x)= 1 + a 5 x + x 2 . (x)+ d & W a ( 4 ) (2)= a(3) + # 0. = 2, ends. + + Therefore, .(x) = 1 a52 x2 = (1 a32)(l For the last equality, recall that the inverses of the error positions are found, via Chien search, by evaluating a(x)at all the nonzero elements of GF(24). This gives a(a12)= 0 and a ( a 3 )= 0. As a result, the error positions are a-12= a3 and a P 3= a12. From Equation (4.12), the modified errors-and-erasures evaluator is W(.) [l + T ( x ) ]a(.) mod x 2 t d + 1 = (1+ a72 + a6z2 a7x3+ al1x4+ a9x5+ x6)(1+ a52 + x 2 ) mod x7 = 1 + a142+ al1z2+ a11x3 24, (4.18) = + + and the errata locator polynomial from which it follows that $'(x) = 1. The values of the errors and erasures are found from (4.14) and (4.15), respectively, e3 e12 fo f5 + all + a5 + a2 + a 3 )= a3a5 = a 8 , a 12 (I + a2 + a2 + a5 + a12)= a12a4= a , + a l l + all + 1) = a14, = (1+ a+ + a l o ) = a5, = a5(1 = a3(1 = + + from which it follows that the errata polynomial is The decodedpolynomialC(%) = F ( % ) have been corrected. + t?(x)is identical to u(x).Two errors and twoerasures NON-BINARY BCH CODES: REED-SOLOMON CODES 71 Direct solution of errata values For small values of the minimum distance of an RS code, the erasure values may beobtained by solving a set of linear equations. Let C(z) be the error polynomial associated with an error pattern resulting from the presence of v errors and p erasures, v (4.19) e=1 &'=O Then, the following set of linear equations, similar to (4.8), hold between the syndromes and the values of the errors and positions: V U e= 1 ['=l (4.20) where 1 5 i 5 2td. As before, any set of v of the errors and erasures. + p 5 t d equations can be used tosolve the values Example 44 Direct solution of the errata values for the code in the previous example: After the Berlekamp-Massey algorithm and Chien search, the decoder knows that the error polynomial is of the form The errata values can be found by solving the set of linear equations given by(4.20), which can be put in the following matrix form, (4.21) It can be verified that fo = a14,e3 = a8, f5 = a5 and e12 = a are the solutions to (4.21). These are the same errata values as those computed before with the modifiedForney algorithm. 4.4 Weight distribution As mentioned before, RS codes are also MDS codes. The weight distribution of MDS codes can be computed exactly with a closed form expression [MS, LC]: The number of codewords of weight i in an (n;IC, d ) MDS code over G F ( q ) is (4.22) For error performance evaluation of an RS (n;IC, d ) code over GF(2"), the following upper bound on the probability of a bit error Pb(C) for an RS decoder2 is simple to compute and It should be noted that the boundis tight onlyfor bounded distance decoders, suchas those based onBMA, EA or PGZ. 72 THE ARTOF ERROR CORRECTING CODING relatively good, (4.23) where P, denotes the probability of a symbol error at the input of the RS decoder, PS = 1 - (1 -p)", and p denotes the probability of a bit error at the input of the RS decoder. The probability of a word error can be upper bounded by (1.3 l), (4.24) The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) 5 Binary convolutional codes First introduced by Elias [Eli2], binary convolutional codes are perhaps the most popular form of binary error correcting codes andhave found numerous applications: Wireless communications (IMT-2000, GSM, IS-95), digital terrestrial and satellite communication and broadcasting systems, and space communication systems, just to cite a few. They were referred to in [Eli21as convolutionalparity-checksymbols codes. Their most popular decoding method to date, the Viterbi algorithm [Vitl], also finds applications in combinatorially equivalent problems such as maximum likelihood sequence detection (e.g., equalization) and partial-response signaling (e.g., magnetic recording). Most recently, it has been shown that convolutional codes, when combined with interleaving in a concatenated scheme, can perform very close to the Shannon limit [BGT]. In this chapter, the basic properties and decoding procedures for binary convolutional codes are described. 5.1 Basicstructure A convolutional code is an error correcting code that processes information serially, or continuously, in short block lengths. A convolutional encoder has memory, in the sense that the output symbols depend not only on the input symbols, but also on previous inputs andlor outputs. In other words, the encoder is a sequential circuit or a finite state machine. The state of the encoder is defined as the contents of the memory. In the computer programs that implement the Viterbi algorithm and other decoding procedures that involve a trellis, found on the ECC web site, a state transition table, indicating the relation between the input, the previous and current state, and the current output, is employed. A convolutional code consists of the set of all binary sequences produced by a convolutional encoder. In theory, these sequences haveinfinite duration. In practice, the state of the convolutional code is periodically forced to a known state and code sequences produced in a block-wise manner. Example 45 Consider the convolutional encoder depicted in Figure 29. For analysis and decoding purposes, this encoder can be described by Table 2. Note that the coding rate of the convolutional encoder is equal to 1/2, because two output coded bits are produced for every input information bit. In general, a rate-k/n convolutional encoder has k shift registers, one per input information bit, and n output coded bits which are given bylinear combinations (over the binary field, i.e., with exclusive-or gates) of the contents of the registers and the input information bits. 74 THE ART OF ERROR CORRECTING CODING Figure 29 An encoder of a rate-1/2 memory-2 convolutional encoder. Table 2 Input bits, state transitions and output bits. Initial state Information Final state outputs SO[i]Sl [il [il S& l]Sl[i l] d o ) [i]w(l) [i] 00 0 00 00 10 11 00 1 01 0 00 11 01 1 10 00 01 10 10 0 10 1 11 01 11 0 01 01 11 1 11 10 + + For simplicity of exposition, and for practical purposes, only rate-l/n binary convolutional codes are considered in the remainder of the chapter. One reason for doing this is that these are the binary codes most widely used. A technique known as puncturing can be applied that results in convolutional encoders of higher rate. Puncturing is discussed in Section 5.5. The total length of the shift registers in the encoder is referred to as the memory. For the discussion in this chapter, states labelsare integers I associated with the binaryrepresentation of the memory contents, as I = C,";' s j [i]2m-1-j. The constraint length, which for rate-1/2 encoders equals K = m l, is defined as the numberof inputs ( u [ i ] ,u[i-l],..., u[i-m])that affect the outputs (W(') [i], ...,~ ( " - ~ ) at [ i ]time ) i . For the encoder in Figure 29, K = 3. An encoder with m memory elements will be referred to as a memory-m encoder. A memory-m rate-l/nconvolutional encoder can be represented by a state diagram.There are 2m states in the diagram. Because there is only one information bit, two branches enter and leave each state and are labeled by u[i]/do)[i]. . . ~ ( ~ - l ) [ i ] . + Example 46 A state diagram of the memory-2 rate-1/2 convolutional code of example 45 is shown in Figure 30. The encoder of a memory-m rate-l/n binary convolutional code can also be considered a discrete linear time-invariant' system. Therefore, the impulse response, i.e., the output of the encoder when the input is an "impulse" ii = (l000 ...00 ...), completely specifies thecode. Convention: In writing a sequence, the leftmost symbol is meant to be the first symbol that goes into the channel. Note, however, that time-varying convolutional codes can be defined and are studied in [Joh] 75 BINARY CONVOLUTIONAL CODES Figure 30 State diagram of a memory-2 rate-l/2 convolutional encoder. There are n impulse responses of a convolutional encoder of rate l / n , one for each output for j = 0; 1: . . . n - 1. As the impulse goes across the memory elements of the encoder, it “picks up” the connections between the memory elements and the outputs. Evidently, this is a finite-impulse-response (FIR) system. Let go, .... g,-l denote the impulse responses of a rate-l/n convolutional encoder. These are also called the generator sequences - or generators - of the code, based on the observation above, and indicate the actual physical connections of the encoder. Example 47 Continuing with the memory-2 rate-1/2 convolutional code of Example 45, the encoder in Figure 29 produces g o = (1 1;1)and 31 = (1 0: l ) , as shown in Figure 3 1. U 1000000... 10100000... Figure 31 Generator sequences of a memory-2 rate-1/2 convolutional encoder. Note that the generator sequences are equal to zero after K bits. Therefore, it suffices to consider vectors of this length. To stress the dynamic nature of the encoder, the indeterminate D (for “delay”) is employed and the generators written as polynomials in D, go(D), .... Bn- 1 (D). Example 48 For the encoder in Figure 29, + +D2 go(D) = 1 D and g1(D) = + 1 D2. In general, when refemng to a convolutional code, the generators are expressed as (go;. . . g n - l ) and written in octal form. For the encoder of Example 48 the generators are (7;5). The most widely used convolutional encoder to date is a memory-6 rate-1/2 encoder with generators (171; 133). This is the so-called NASA standard code, as it was first used in the Voyager space missions. It is also used in many digital communications standards, as well as in the CCSD standard. THE ART CORRECTING OF ERROR 76 CODING As a result of the dynamical structure of a convolutional encoder, the state diagram can be represented as it evolves in time with a trellis diagram. A trellis diagram is constructed by placing the state diagram of the code at each time interval with branches connecting states between time i and time i 1,in correspondence with the encoder table. The branches of the trellis are labeled in the same way as the state diagram. + Convention: When there is no dichotomy, the input information bit does not need to appear explicitly in the branch label. For FIR encoders, the information bit u can be inferred directly from the state transition2: (SOS1 . . . sm-l) --f (usg. . . sm-2). It should be emphasized that, inthe case of a recursive systematic convolutional (IIR) encoder, this is not generally the case, as is shown in Section 5.1.1. Example 49 For the example rate-l/:! memory-2 convolutional encoder, the trellis structure is shown in Figure 32. Note that state transitions (branches) form a state s[i] to a (possibly different) state s ’ [ i l],that caused by the input bit u [ i ] = 0, are the upper branches. + t= 1 t=O t=2 t=3 t=4 t=5 t=6 Figure 32 Six sections of the trellis of a memory-2 rate-1/2 convolutional encoder. Example 50 Consider the encoder in Figure 29 and the input sequence U = (110100). Then, the corresponding output sequence can be obtained either directly from the encoder table, Table 2, or as a path in the trellis, as shown in Figure 33. It follows that the corresponding coded sequence is V = (11;01; 01;OO; 10; 11). A convolutional encoder is a linear time-invariant system, with impulse responses given by the code generators, go(D), (D);. . . &(D),where $(D) =gj[0]+gj[1]Dfgj[2]D2+-.+gj[m]Drn; for 0 5j (5.1) n. In terms of the generators of a code, the output sequences can be expressed as c m .(j) [i] = u[i - e ] g j [ e ] ; e=o This simple observation is the basis of using a trucebuck memory in a Viterbi decoder, to recover the information bits from the decoded sequence. 77 BINARY CONVOLUTIONAL CODES 11 01 00 01 10 11 Figure 33 A path in the trellis of a memory-2 rate-1/2 convolutional encoder. 0 5 j < n. The output sequences ( D ) ,0 5 j < n, as expected, are equal to the discrete convolution between the input sequence u ( D ) and the code generators go(D), g1 ( D ) ; .. . gn-l ( D ) .From this fact comes the name - “convolutional”- of the encoder [Eli2]. Observe now that Equation (5.2) above can be written as a matrix multiplication ,ij= uG; (5.3) where G is a generator matrixof the convolutional code. In particular, for a rate-1/2memorym convolutional code, where the blank entries represent zeros. Generalization to any rate 1 / n encoder is straightforward. Example 51 For the memory-2 rate-1/2 encoder in Figure 29, G= [ 11 10 11 11 10 11 11 10 11 11 10 11 ... ..l. Let 21 = (110100). Then it follows from (5.3) that V = aG = (11:01; 01; 00; 10; l l ) , which is the same output as in Example 50. + + Let V ( D ) = d 0 ) ( D ) D d 1 ) ( D )+ . .. Dn-’dn-’)(D). tween input and output sequences can be written as Then the relationship be- V ( D )= u ( D ) G ( D ) ; (5.5) where generators of a rate-l/n convolutional code are arranged in a matrix, referred to as a polynomial generator matrix, G(D) ( Go(D) g1(D) . . . gn-l(D) ) . (5.6) 78 THE ART OF ERROR CORRECTING CODING 5.I . 1 Recursive systematic convolutional codes With the use of (5.6), a memory-m rate-l/n recursive systematic convolutional code, or RSC code, has a generator matrix of the form As a short notation of this generator matrix, the generators (1;g1 /go . . . gn- /go) can be specified. Division and multiplication of the right-hand side of (5.5) by go(D) give, together with (5.6), v(D)= d(D)G’(D), (5.8) where .ci’(D)= go(D)U(D). This shows that both the nonsystematic encoder andthe systematic encoder produce the same codedsequences, but that the corresponding information sequences are scrambled by go ( D )in the RSC code. A systematic encoder is also an example of a discrete-time linear time-invariant system. Because the generator matrix contains rational functions, the encoder is an injinite-impulseresponse (IIR) linear system, as opposed toa non-systematic encoder which is ajinite impulse response (FIR) linear system. The preferred form of expressing the encoder circuit is the controller canonical form [For4]. An encoder for an RSC code, consists of a shift registers, a circuit3 for dividing by go(D) and n - 1 circuits for multiplying by 81 ( D ) ;. . . gn-l ( D ) . Example 52 A rate-1/2 memory-2 binary RSC encoder with generators (1; 5/7) is shown in Figure 34. The trellis structure of this code is the same as the nonsystematic code with generators (7; 5). However the inputloutput bits per transition are different. The statediagram of this code is shown in Figure 35. Compare with Figure 30 on page 75. Among other things, this difference in inputlouput mapping means that the all-zero sequence may not necessarily bring the state back to sosl = 00. This is evident from Figure 35. In particular, the impulse response of this encoder is (11,Ol; 01; 00; 01; 01; 00; 01: 01; 00;. . .). Figure 34 Encoder of a memory-2 rate-1/2 recursive systematic convolutional encoder. The general structure of circuits that multiply and divide bytwo polynomials can be found, among others, in [PW], Figure 7.8, or [LC], Sec. 4.7. BINARY CONVOLUTIONAL CODES 79 Figure 35 State diagram of a memory-2 rate-1/2 recursive systematic convolutional encoder. 5.1.2 Free distance The free distance df of a convolutional code is the smallest distance between any twodistinct code sequences. The length of the sequences has to be sufficiently large, much larger than the constraint length of the code. The free distance of a convolutional code can be obtained from its weight enumerator polynomial, as described in Section 5.3. There are other distances associated with a convolutional code, when the length of the sequence is of the order of the constraint length, but these are not relevant for the discussion in this book. More details on the structure of a general rate-lc/n convolutional encoder can be found in references [LC] and [Joh]. 5.2 Connections with blockcodes There is a connection between convolutional codes and block codes, as it is evident from the above description of convolutional encoders. As indicated previously, it is customary for the information sequences input to a convolutional encoder to be broken into blocks offinite length (e.g. a few thousands of bits). Generally, a fixed sequence of length m is appended at the end of each information sequence. This sequence is typically a unique word that serves to synchronize the receiver and forces the convolutional encoder to return toa known state. In the following, for simplicity of exposition, let C be a nonsystematic (FIR) convolutional code C with free distance d f , obtained from a memory-m rate-l/n convolutional encoder. Similar arguments apply to RSC codes. 5.2.1 Zero-tail construction By appending a “tail” of m zeros to an information vector .U of length ( K - m), all paths in the trellis corresponding to 2K-m codewords merge at the all-zero state. The result is a linear block ( n K ,( K - m ) , dZT) code, denoted by C ~ TIf. K is large enough, then the rateof CZT approaches the rate of C. As long as K > m holds, with K sufficiently large, even though the code sequences are restricted to end at the all-zero state, the minimum distance of CZT satisfies dZT = df . Example 53 Let C be the convolutional code obtained from the rate-1/2 memory-2 encoder in Figure 29, of free distance4 df = 5. Assume that message vectors of length K = 3 bits are encoded, each followed by m = 2 zeros. Then the zero-tail construction results in a binary That this is the value of d f will be shown in Section 5.3. 80 THE ART CORRECTING OF CODING ERROR linear block (10,3,5) code CZTwith generator matrix, G= ( 11 10 11 00 00 00 l1 10 11 00 00 00 11 10 11 + + The weight distribution sequence of this code is A ( z ) = 1 3z5 3z6 The code rate of CZTis 0.3 which is less than the rate k / n = 1/2 of C. + z7, and dZT = 5 . 5.2.2 Direct-truncation construction In this method, the codewords of a linear block ( n K ,K , dDT) code CDTassociated with a rate-k/n convolutional encoder are all code sequences associated with paths in the trellis that start in the all-zero state and, after K bits are fed into the encoder, can end at any state. The rate of CDTis the same as that of the convolutional encoder. However, theminimum distance of the resulting block code is reduced and dDT < df . Example 54 Consider the rate-l/2 memory-2 encoder in Figure 29. Information vectors of length K = 3 information bits are encoded. Then the direct truncation construction gives a binary linear block (6,3, dDT) code with generator matrix, G= (b:, : ::) 00 11 10 . The weight distribution sequence of this code is A ( z ) = 1 dDT 1 2 < d f . 5.2.3 + x’ + 3z3 + 2x4 + z5, and Tail-bitingconstruction The codewords of the tail-biting block code CTBare those code sequences associated with paths in the trellis that start from a state equal to the last m bits of an information vector of K data bits. This ensures that all codewords in CTBbegin and end at the same state. The rate of the resulting block (2K,K , dTB) code is the same as that of the convolutional encoder. However, unless K > m and K is sufficiently large, the minimum distance d T B 5 d f . Example 55 Consider the rate-l/2 memory-2 encoder in Figure 29. Assume that a block of K = 5 information bits is encoded. Then the tail-biting construction gives a binary linear block (10,5, dTB) code with generator matrix, 11 10 11 00 00 00 11 10 11 00 11 00 00 11 10 10 11 00 00 l1 The weight distribution sequence5 is A ( z ) = 1 + 5z3 + 5z4 + 6x5 + 10z6 + 5x7, and dTB = 3 < d f . This WDS was obtained with a program from the ECC web site that computes the weight distribution of a binary linear code from its generator matrix. 81 BINARY CONVOLUTIONAL CODES 5.2.4 Weight distributions In this section, a method is described to obtain the weight distribution of linear block codes derived from binary rate-l/n convolutional codes by any of the constructions in the previous sections [WV]. Let R(x) be a Zm X Zm state transition matrix with entries of the form (5.9) = &jZhiJ, Rij(2) where 6ij = 1,if and only if there is a transition from state i to state j , and S i j = 0 otherwise; and hij is the Hamming weight of the corresponding output vector (of length n). Example 56 For the convolutional encoder with the state diagram shown in Figure 30, the state transition matrix O ( x ) is given by n(x) = ( :) 1 0 x2 0 x 0 5 o x o x ' The weight distribution sequence of a binary linear block ( n ,k ) code constructed from any of the previous methods can be obtained by symbolically raising matrix n ( x )to the C-th power, denoted @(x), and combining different terms. The term n&(z)gives the weight distribution of trellis paths that start in state i and end at state j after C time intervals. For the ZT construction, the value C = k + m is used, while for both DT and TB constructions, C = k. The weight distribution sequences for each of the construction methods presented above can be computed as follows. 0 ZT construction: A ( s ) = Rk,f"(x). 0 DT construction: 0 TB construction: A ( z )= C Rtj(x). j=O Example 57 Consider again the memory-2 rate-l/:! convolutional encoder. Then 1+x5 fl3(x) = ( x2+x3 53+24 x3 + 54 53+x4 x 5 + 2 x2+23 x2 53 + 52+x3 x4+x x3+54 x5+2 5 5 + 2 2 + x 3 x5 + x 2 x2 + x 3 1 . The weight distribution sequence of the code from the DT construction in Example 54 is obtained by adding the terms in the first rowof R3 (x). References on other methods of computing the weight distribution sequence of a linear block code derived from a convolutional code are [KC], [CK] and [DFKl]. THE ART OF ERROR CORRECTINGCODING 82 X X Figure 36 The modified state diagram of a memory-2 rate-1/2 convolutional code. 5.3 Weight enumeration and performance bounds The performance of convolutional codes over memoryless channels can be estimated by using union bounds, such as those presented in Chapter 1 for block codes. It was Viterbi [Vit2]that first considered the performance of convolutional codes. See also [ViOm]. For a convolutional code, the weight enumerating sequence (WES) is defined as T ( x )= T1x + T~x’+ . . . + T , X ~+ . . . , (5.10) where the coefficient T, in T ( z )denotes the number of code sequences of weight W. Note that, in principle, the inputs to a convolutional code are infinite sequences. As a result, in general, the WES has an infinite number of terms. It is possible, however, to obtain closedform expressions for the WES of convolutional codes, as shown below. Notation: A state ( S O S ] . .. s m - l ) of the encoder willbe 1 1 CZi’ si277--l--i. denoted by S(’), where A modiJied state diagram enables us to compute the WES. This diagram has the all-zero state split into an initial state S/:)t and a final state The branches of the diagram are labeled by terms P , where W is the Hamming weight of the output sequence. Example 58 Let C be the memory-2 rate-1/2 convolutional code with generators (7,5) that has been used throughout this chapter. Figure 36 shows the modified state diagram of C. There are basically two methods to compute T ( z ) .In the first method, the modified state diagram is considered a signal flow graph. Then Mason’s rule is applied to find T ( z ) .This is the method covered in most textbooks on error correcting codes. For details, the reader is referred to [LC]. Another method, originally developed by Viterbi [Vit2],consists of assigning intermediate variables p ~ j ( zto ) each state S ( j ) ,j = 1 , 2 , . . . ,2m - 1. Each pj (x)can be expressed as a linear combination of weighted variables associated with states connected to S ( j ) , (5.1 1) BINARY CONVOLUTIONAL CODES 83 for all k E { 1;2; . . . 2m - l}, k # j , and where 6ij and h k j are defined as in (5.9). The initial state S/:)t is assigned a variable with value 1, while the WES T ( x )is assigned as a variable to the final state .Sfinal, (0) This yields a system of linear equations that can be solved to find p1 (x),from which, with the adopted notation to designate states, T ( x ) = p1(x)x% (5.12) Example 59 Consider the modified state diagram in Figure 36. The following set of equations is obtained (5.13) and T ( x )= p1 (.)x2. Equations (5.13) can be written in matrix form as (;l -;" -0" -x 1- 2 ) ( PP1 2 (x) (x)) = p3(2) (2) ; (5.14) and solving for 1-11 (x)gives Pl(2) = x3 -. 1 - 2x It follows that + + + T ( x )= -= x5 2x6 + 4x7 8x8 . . . (5.15) 1 - 22 This shows that the minimum free distance is d f = 5 for the binary 4-state rate-1/2 convolutional code with generators (7,5). x5 More detailed information about the structure of a convolutional code is obtained by labeling the modified state diagram with terms of the form xwyezm,where W is the Hamming weight of the coded outputs, e is the Hamming weight of the input vector (of length equal to k in general for a rate-k/n convolutional code), and m is the number of branches differing from the all-zero sequence. (See [LC], p. 302.) This results in a complete weight enumerator sequence (CWES), T ( z :yi z ) which can beusedto derive various bounds on the error performance of convolutional codes [Vit2], [LC], [Joh]. A weight enumerating sequence similar to the CWES was used to estimate the performance of turbo codes in [BM]. Example 60 Continuing with the example 4-state rate-1/2 convolutional code, a modified state diagram for the computation of the CWES is shown inFigure 37. The equations are i 1 -9.2 0 -xz 1 -xyz -0"" 1 - xyz with T ( z ;y; z ) = x2zp1(x).The solution is: ))I$;( = [x;z) p3(x) 84 THE ART OF ERROR CORRECTING CODING Figure 37 The modified state diagram of a memory-2 rate-1/2 convolutional code. Bounds on the bit error probability of a binary convolutional code of rate k / n can beobtained with the aid of the CWES. For transmission over a BSC channel and binary transmission over an AWGN channel [ViOm], and maximum-likelihood decoding (MLD)6,the following upper bounds hold, respectively, (5.16) (5.17) Union bounds maybeusedto estimate the bit-error rate (BER) performance of a convolutional code. The reader should be aware, however, that the bounds (5.16) and (5.17) are quite loose. Fortunately, tighter (close to the actual BER performance) bounds exist at relatively mild channel conditions, i.e., low values of p for the BSC channel and high values of E,/Na for the AWGN channel, and are given by the following [Joh]: Example 61 The union bound (5.18) on the probability of a bit error for the 4-state rate1/2 convolutional code of example 60, with transmission over a BSC channel with crossover probability p and MLD, is plotted in Figure 38. 5.4 Decoding: Viterbi algorithm with Hamming metrics The trellis of convolutional codes has a regular structure. Advantage can be taken from the repetitive pattern of the trellis in decoding. However, for linear block codes obtained An example of MLD is the Viterbi decoder, explained in the next section. BINARY CONVOLUTIONAL CODES 0.1 85 0.01 0.001 P Figure 38 Union bound (5.18) on the BER of a memory-2 rate-1/2 convolutional code with d f = 5. Transmission over BSC with crossover probability p and MLD. from terminating convolutional codes, and long information sequences, maximum-likelihood decoding is simply too complex and inefficient to implement. An efficient solution to the decoding problem is a dynamic programming algorithm known as the viterbi algorithm, also known as the viterbi decoder (VD). The VD is a maximum likelihood decoder, in the following sense. The VD finds the closest coded sequence V to the received sequence by processing the sequences on an information bit-by-bit (branches of the trellis) basis. Instead of keeping a score of each possible coded sequence, the Viterbi decoder tracks the states of the trellis. 5.4.1 Maximum likelihood decoding and metria The likelihood of a received sequence F after transmission over a noisy channel, given that a coded sequence V is sent, is given by the conditional probability n n-l P(T1V) = P(rilvi); (5.20) i=O It is easy to show that for a BSC channel with parameter p , (5.21) with d ~ ( r vi) i ; = 1,if ri # vi,and d ~ ( r ivi) ; = 0, if ri = vi.That is, d ~ ( r ivi) : = 1 is the Hamming distance between bits ri and vi. For an AWGN channel, the likelihood is given by (5.22) 86 THE ART CORRECTING OF ERROR CODING where m(.)denotes a binary modulated signal. Here, m is defined as a one-to-one mapping between bits (0: l} and real numbers { +G}. A maximum-likelihood decoder (MLD) selects a coded sequence V’ that maximizes (5.20). By taking the logarithm (any base) of (5.20), the following can be shown. For the BSC channel, an MLD is equivalent to choosing the code sequence that minimizes the Hamming distance -a7 (5.23) i=O Similarly, for the AWGN channel, it is the squared Euclidean distance c n-l dE(F;V)= 2 (F - m(v)) (5.24) ; i=O that is minimized by the coded sequence selected by the MLD. In this section, a BSC channel with crossover error probability p is considered. The AWGN channel is covered in Chapter 7. 5.4.2 The Viterbialgorithm Let Sjk) denote a state in the trellis at stage i . Each state S j k ) in the trellis is assigned a state metric, or simply a metric, M ( S (i k )), and apath in the trellis, g ( k ) .A key observation in applying the Viterbi algorithm is: yik) At time i, the most likely paths per state (the ones closest to i - e. received sequence)will eventually coincide at some time the In his paper, Viterbi [Vitl] indicated that the value of e for memory-m rate-1/2 binary convolutional codes should be C > 5m. The VD operates within a range of L received ntuples (output bits per state transition) known as the decoding depth. The value of L must be such that L > C. In the following, the Viterbi algorithm applied to a memory-m rate-l/n binary convolutional code is described, and its operation illustrated via a simple example. Some additional notation is needed: Let G [ i ] = (vo[i]vl[i] . . .vn-l [ i ] )denote the coded bits in a branch (state transition), and let F [ i ] = (ro[i]rl[i] . . . rn-l [i]) denote the output of the channel. Basic decoding steps Initialization Set i = 0. Set metrics and paths The specific way in which the initialization of the paths is performed is irrelevant, as shown later. For the sake of clarity of presentation of the algorithm, it is assumed that the paths are represented as lists that are initialized to the empty list. 1 . Branch metric computation At stage i , compute the partial branch metrics back 87 BINARY CONVOLUTIONAL CODES B M j b ) = d H ( F [ i ] ,G [ i ] ) , (5.25) b 2 C:Ii t1e[i]2~-'-', associated with the n outputs G [ i ] of every branch (or state transition) and the n received bits F[i]. 2. Add, compare and select (ACS) For each state k = 0 , 1 , . . . ,2m - 1, and corresponding pair of incoming branches, from two precursor states S&) and compare M ( S j F 2 ) + $22, + and M($??) B M i b 2 )where , bj = C::; ve[i]2"-'-', branch giving the smallest path metric and update, i = 1 , 2 . Select awinning M ( s ~ ~=)min{M(SjF>) ) + B M , ! ~ ' )M, ( s , ! ~ )+) B M : ~ ~ ) } . (5.26) 3. Path memory update For each state S i k ) ,IC = 0 , 1 , . . .,2m - 1,update the suwivorpaths g ( k )as follows, with the output of the winning branch ckj,j E { 1,2}. -(k) Yi - (k,) (Yz-1 , W k j ) , (5.27) 4. Decode symbols If i > L, then output as the estimated coded sequence where C I' is the index of the state S(") with the smallest metric. Set i = i 1 and go to decoding step 1. + yiyi, It should be stressed that this is not the only way to implement the Viterbi algorithm. The above procedure can be considered a classical algorithm. There are alternative implementations that, depending on the particular structure of the underlying convolutional encoder, may offer advantages (see, e.g., [FL2]). In addition, in the last step of the algorithm, symbol decoding can be applied to information bits directly. This is the form usually employed in the software implementations of Viterbi decoders available on the ECC web site. In hardware implementations, a method based on a trucebuck memory is favored that estimates the original information sequence, indirectly, based on state transitions. This technique is discussed later in the chapter. ACS - 7 BMG Branch Metric Generator Add, Compare and Select I paths ' Update and metrics u > Figure 39 Block diagram of a Viterbi decoder. Example 62 Consider again the memory-2 rate-1/2 convolutional encoder with generators (7,5).Note thatd f = 5 for this code. This example shows how a single error can be corrected. Suppose that G = (11,01,01,00,10,11) is transmitted over a binary symmetric channel (BSc), and that P = (10,01,01,00,10,11) is received (one error in the second position.) The operation of the Viterbi decoder is illustrated in Figures 40 to 45. The evolution of the metric values with respect to the decoding stages is shown in thefollowing table: THE ART CORRECTING OF CODING ERROR 88 Statelstage i=O i = l i=2 SjO) 0 0 0 0 1 1 0 l l Sp $2) $3) 0 1 0 i=3 1 l l l i=4 i=5 1 1 2 1 2 i=6 1 3 2 2 2 2 2 After processing six stages of the trellis ( i = S), the state with the smallest metric is S,( 0 ) with associated (survivor) path = V. One error has been corrected. Transmitted 11 01 01 00 10 11 Received 10 01 01 00 IO 11 t = 0: Figure 40 VD operation for Example 62, at i = 0 and i = 1. 5.4.3 Implementation issues In this section, some of the implementation issues related to Viterbi decoders are discussed. The techniques below apply equally to any Viterbi decoder that operates over channels with additive metrics, such as the BSc, AWGN and flat Rayleigh fading channels. Path metric initialization The Viterbi decoder can operate in the same mode from the start ( i = 0). The survivor paths can have arbitrary values, without affecting the decoder's performance. The first L decoded bits are therefore random and give no information. For this reason, the value of L contributes to the decoding delay, and is also known as the decoding depth. Moreover, provided that L is large enough ( L >> e, where t? > 5m for rate-1/2 binary codes), the decoded bit can be BINARY CONVOLUTIONAL CODES 89 10 01 10 01 t=2: Figure 41 VD operation for Example 62, at i = 2. output either from the path with the lowest metric or always output from the zero state path (g'')). The latter method is easier to implement and does not result in loss of performance. The programs on the ECC web site that implement MLD using the Viterbi algorithm work in this fashion. Also note that in Example 62 the branch labels (output) were stored inthe survivor paths. This was done in order to facilitate understanding of the algorithm. In a practical implementation, however, it is the corresponding information bits that are stored. This is discussed below, in connection with path memory management. Synchronization Branch symbols must be properly aligned with the received symbols. Any misalignment can be detected by monitoring the value of a random variable associated with the Viterbi decoder. Two commonly used synchronization variablesare: ( I ) Path metric growth, and (2) channel BER estimates. The statistics of these variables give an indication of abnormal decoding behavior. Assume that the received sequence is not properly received, i.e., the n-bit branch labels V [ i ] in the decoder are not properly aligned, or synchronized, with the received sequence F [ i ] . Example 63 Figure 46 shows an example for a rate-l/2 in which the received sequence F is not synchronized with the reference coded sequence V. In other words, not all the bits in the received subsequence F [ i ] belong to the same trellis stage in the decoder. In this case, two events that may occur are: (1) The path metrics are close to each other and grow rapidly,and (2) the estimated channel BER approaches 1/2. Figure 47 shows the block diagram of a Viterbi decoder and a BER monitor. THE ART OF ERROR CORRECTING CODING 90 10 01 01 10 01 01 1=3: (0) y 3=(10,01,11) (11 y 3=(11,01,01) (2) y 3 = (10,01,00) (31 y 3=(10,00.01) Figure 42 VD operation for Example 62, at i = 3. A synchronization stage needs to be added, external to the decoder itself, whose function is to advance the reference sequence V in the decoder until the statistics return to normal. This can be done by skipping received symbols (a maximum of n - l times) untilthe synchronization variables indicate normal decoding behavior. This is indicated in Figure 46 of Example 63 for the case of a rate-1/2 convolutional code. Metric normalization As the Viterbi decoder operates continuously, the path metrics will grow proportionally with the length on the received sequence. To avoid overflowor saturation (depending on the number representation used), the metrics need to be normalized. There are basically two methods of doing this. Both rely on thefollowing two properties of the Viterbi algorithm: 1. The MLD path selection depends only on the metric differences. 2. The metric differences are bounded. m m Threshold method To normalize the metrics, at each decoding stage, the value of the smallest metric is compared to a threshold T.1f Mmin> T then T is subtracted from all the metrics. Clearly, this does not affect the selection process, because the metric differences remain the same. This is a method that can be easily implemented in software on a generic processor. Modular arithmetic method The metrics are computed modulo N , so that they lie within the range [O: N - l], where N = 2An,,,, and Amaxis the maximum difference in survivor path metrics. BINARY CONVOLUTIONAL CODES 91 10 01 01 00 IO 01 01 00 t = 4 Figure 43 VD operation for Example 62, at i = 4. Obviously, A,,, depends on the range of values received from the channel. From the two properties of the Viterbi algorithm, it can be shown that the same MLD path is selected by the Viterbi decoder when computing path metrics with modular arithmetic. In particular, it is possible to use two's complement arithmetic7,in such a way that overflow can occur but it does not affect the selection process. For details, see [HEK, OCC]. This method is favored in hardware implementations of Viterbi decoders. Path memory management In a Viterbi decoder, survivor paths and their metrics need to be stored and updated. In a continuous operation mode, while updating the survivor paths at stage i, earlier portions of the survivor paths merge with high probability at stage i - L, where L is the decoding depth. The estimated information bits are taken from the (single) portion of the merged paths at stage i - L. There are different techniques to extract the information bits. Two of the most common are: (1) Register exchange and (2) Traceback memory. 0 Register exchange This method is the easiest to implement in software. All the survivor paths are updated at each iteration of the Viterbi algorithm. Therefore, the information bits can be read directly from the survivor paths. However, if this technique is implemented in hardware, the decoding speed would be low,due to an excessive number of times at which the path memory is read and written. To simplify control flow instructions, a circular pointer of length L can be used. Witha circular pointer, at decoding stage i, position (i - L ) in memory (to output the decoded bit) is equal to position (i 1) + 'Arithmetic using the additive group { -Zrn-I, 1 - 2rn-', . . . ,Z m - l - l} of m-bit integers [HEK]. 92 THE ART CORRECTING OF CODING ERROR 10 01 01 00 10 IO 01 01 00 10 t=5: (0) y 5=(10,01,11,00,00) (1) y 5=(11,01,01,00,10) (2) y 5=(10,01,11,00,11) (3) y 5 = (10,00,01,10,10) Figure 44 VD operation for Example 62, at i = 5 . modulo L. Traceback This technique is favored in implementations in hardware. The survivor paths are composed of decision values, which indicate state transitions in order to trace back the survivor paths and reconstruct a sequences ofstates in reverse order. The traceback memory can beorganized as a rectangular array, with rows indexed by k , for all the trellis states k = 0,1, . . . , 2m - 1. At decoding stage i , for each trellis state S,('),0 5 j 5 2m - 1, the traceback memory (or TB RAM) is written with the rightmost bit of the previous state S!::, l E { 1,2}, associated with the winning branch. The traceback method trades off memory for decoding speed, since it writes only one bit per state per decoding stage, instead of L bits per state per decoding as in the register exchange method. The information bits are decoded by reading in reverse order the state transitions with an encoder replica (or looking up a state transition table). More memory (number of columns if organized in a rectangular array) is required for continuous operation because, at the same time as information bits are being read out (decoded), new decision values are being written. Example 64 Continuing with Example 62, using the traceback technique, at the end of the i = 6 received pair of bits from the channel, the traceback memory would have the contents shown in Table 3. The traceback memory is read starting from the last bit (e.g., use a LIFO), i = 6 (in general T = L). The row address is given by the state with the best metric. This is state S(') in the example. Read the transition bit b6 (in general, b ~ )The . row address (ROW) at i = 5 (i = L - l),to read the next transition bit bg, is obtained by shifting the address IC at i = 6 BINARY CONVOLUTIONAL CODES 01 10 01 93 00 10 11 00 10 11 t = 6: 01 01 10 (0) y (11,01,01,00,10,11) y(1),=(10,00,01,10,10,01) y(2) 6=(10,01,11,00,w,ll) y(3) 6=(10,00,01,10,10,10) Figure 45 VD operation for Example 62, at i = 6. SKIP ... ...ro[i] Synchronized Reference ... vo[i] 1- r,[i] vo[i+l] A’ rl[i+l] ro[i+2] vl[i+l] v,[i] ‘ A rl[i+2j vo[i+2] vl[i+2] ... Figure 46 Example of branch misalignment in a Viterbi decoder. ( i = L ) and appending the previous decision bit b6 ( b ~ )This . can be stated in the following C language instruction: ROW[j] = ((ROW[j+l] << 1 ) && MASK) TB-RAM[ROW[j+l]] [j+l]; (where MASK = 2m - 1.) Continuing in this way, the following state sequence is obtained: From this sequence, the corresponding information sequence can be recovered, by reversing the state transitions: 6 = ( l 1 0 l 0 0 ). For high-speed continuous VD operation with the traceback method, the traceback memory must be partitioned into several blocks. In this way, while one block is being written with decisions at the current stage i , another block is read (decoded) at a time i - L, L > C, and another (possibly more than one) block for intermediate stages is used for performing 94 THE ART OF ERROR CORRECTINGCODING Viterbi decoder estimator Delay Figure 47 Channel error rate estimation for a BSC channel. Table 3 Tracing back the path that ended in state S(’). i=l i=2 i=3 i=4 i=5 i=6 0 0 1 0 2 1 0 3 1 1 1 1 1 1 0 0 0 0 1 1 0 0 0 1 1 1 0 1 traceback tracing. This memory partitioning improves speed but increases the latency (or decoding delay) of the decoder [Col, BABSZ]. ACS For memory-m rate-l/n binary convolutional codes codes, the basic trellis element is a butterfly, as shown in Figure 48. The add-compare-select (ACS) operation is implemented using this structure over 2m-1 pairs, i.e., j = 0 , 1 , 2 , . . . ,2m-1 - 1. Therefore, the ACS operation can be done either serially (a loop on j in software) or in parallel, with 2m-1 ACS units, one per butterfly. If the code is antipodal then the generators of the code have a one in the first and last positions. In this case, the labels of the branches incident to a state S(2j) are the same as those incident to state S(’j+l) in a butterfly. Moreover, the label of a branch incident to a state S(’j) is equal to the complement of the label of the other branch’. Using these facts, a clever technique [K21 based on branch metric differences was proposed. For rate-l/n codes, the result is a Viterbi decoder with a compare-add-select (CAS) architecture, and lower complexity of implementation than the conventional ACS approach. 5.5 Puncturedconvolutionalcodes Punctured convolutional codes were introduced in [Cla]. Puncturing is the process of systematically deleting, or not sending, some output bits of a low-rate encoder’. Since the trellis structure of the low-rate encoder remains the same, the number of information bits per sequence does not change. As a result, the output sequences belong to a higher-rate punctured convolutional ( P C ) code. The following discussion focuses on codes obtained from a binary + The complement of a bit a is 1 a mod 2. Sometimes referred to as the mother code. BINARY CONVOLUTIONAL CODES 95 Figure 48 Trellis butterfly of a memory-m rate-l/n binary convolutional code. memory-m rate-l/n convolutional code. rate- lln encoder matrix Framing Figure 49 Encoder of a punctured code based on a rate-l/n code. A puncturingmatrix P specifies the rules of deletion of output bits. P is a k x np binary matrix, with binary symbols pij that indicate whether the corresponding output bit is transmitted = 1)or not (pij = 0). Generally, the puncturing rule is periodic. A ratek / n , PC encoder based on a rate-l/n encoder, has a puncturing matrix P that contains l! zero entries, where np = kn - l!, 0 5 l! kn. Example 65 A rate-2/3 memory-2 convolutional code can be constructed by puncturing the output bits of the encoder of Figure 29, according to the puncturing matrix P= (; ;). The corresponding encoder is depicted in Figure 50. A coded sequence 6 = (... of the rate-1/2 encoder is transformed into code sequence up = (. i.e., every other bit of the second output is not transmitted. One of the goals of puncturing is that the same decoder can be usedfor a variety of high-rate codes. One way to achieve decoding of a PC code using the Viterbi decoder of the low-rate code, is by the insertion of “deleted” symbols in the positions that were not sent. This process is known as depuncturing. The “deleted” symbols are marked by a special flag. This can be done, for example, using an additional bit and setting it to 1 in the position of an erased bit. If THE ART OF ERROR CORRECTING CODING 96 100011 U 110100 6 input bits 111001 v = (ll,Ol,Ol,OO,lO,l1) 12 output bits v = (11,0,01,0,10,1) 9 output bits P -- lXlX0X rate-1/2 rate-2/3 a position is flagged, then the corresponding received symbol is not taken into account in the branch metric computation’0. In a software implementation, an alternative method is for the decoder to check the entries pm,j of matrix P in the transmission order, m = 0; 1;.. . n - 1, j = i; i 1;.. . i k - 1. If pm,j = 0, then the current received symbol is not used to update the branch metric. The decoder advances a pointer to the next branch symbol in the trellis and repeats the test on p,j. Otherwise, pm,j = 1 and the received symbol are used to update the branch metric. Every time the branch pointer is advanced, a check is made to determine if all the symbols in that branch have been processed. If they have, then the ACS operation is performed. This method is used in some of the programs that implement Viterbi decoding ofPC codes on the ECC web site. Table 4 shows puncturing matrices employed with the de-facto standard memory-6 rate1 / 2 convolutional code with generators (go; 91) = (171; 133). In the table, )v: indicates the output, at time m, associated with generator s i , i = 0; 1. Other puncturing matrices can be found in [YKH]. + ~~ lo Some authors write “the branch metric is zero” as an equivalent statement. + 97 BINARY CONVOLUTIONAL CODES Example 66 (Continuation of Example 65.) Let U be the same asin Example 62, and suppose transmission over a BSC introduces an error in the second position. Coded symbols dl)[i];i = 0; 2: 4;.. ., are not transmitted. The decoder knows this and inserts deleted symbols, denoted by E. When computing branch distances, the E symbols are ignored. The operation of a Viterbi decoder with deleted symbols is depicted in Figures 5 1 and 52 for i = 1 and i = 2, respectively. It is left as an exercise to finish the example. Transmitted 11 0 01 0 10 1 Received 10 m 01 m 10 1E t i=O t f i = 1: 10 Branch metric computation: (0) BM1 =d(00, 1 0 ) = 1 (2) BMI =d(10,10)=0 (0) (0) M(S (1) BM1 = d ( 0 1 , 1 0 ) , 2 (3) B M I= d ( l l , 1 0 ) = 1 (0) ) = min [ M(S O)+BM (1) (3) M(S O)+BM ] = 1, select (flip a coin) branch from state 0 Updatemetric M(S (0) )=1 Updatepath y (0) = (00) Figure 51 VD operation for a rate-213 memory-2 PCC, i = 1. 5.5.1 Implementation issues related to punctured convolutional codes For a system that employs punctured convolutional codes, the re-synchronization process will take longer, as the output symbols are spread over severaltrellis branches. Moreover, practical systems use a multiplicity of puncturing patterns of different lengths. The decoder therefore may have to make a guess as to the specific rate and puncturing pattern being received. Other points to consider when implementing PC codes are: 1. The decoding depth L must be increased as more output bits are punctured. This is because high-rate codes (e.g., rate 7 / 8 ) have low minimum Hamming distance THE ART OF ERROR CORRECTING CODING 98 Transmitted 11 0 01 0 IO 1 Received 10 OE 01 OE IO IE 10 OE i=2: Figure 52 VD operation for a rate-2/3 memory-2 PCC, i = 2. between coded sequences (e.g., dmin = 3) and therefore the survivor paths take longer to converge. 2. An additional level of synchronization with the received sequence is needed, to align the puncturing pattern. The fact that the puncturing patterns have different lengths for different rates may be solved by employing the same puncturing period. This is one of the features of rate-compatible PC codes, or RCPC codes. 5.5.2 RCPC codes RCPC codes were introduced by Hagenauer [Hag]. These codes are constructed from a lowrate code by periodic puncturing. Let M denote the puncturing period. Then, as before, the puncturing matrices are n x M binary matrices. However, in general, M > kn - l . To construct a family of RCPC codes, it is required that the branch outputs of a high-rate PC code are used by the lower rates PC codes. This is achieved as follows. Let P ( H )and P ( L )denote matrices of a high-rate code CH and a low-rate code CL, respectively, both derived from the same lowest-rate code C. Then codes CH and CL are rate-compatible if the following condition holds: pi;’ 1+ = 1. An equivalent condition is z 0 +p!?) = 0. 23 BINARY CONVOLUTIONAL CODES 99 Example 67 Let C be a rate-1/2 convolutional code. Let M = 4. Then matrices P(l) through P(4) below generate RCPC codes of rates 4/5,4/6,4/7 and 4/8(=1/2), respectively. RCPC codes [Hag] find applications in automatic repeat request (ARQ) systems, due to their incremental redundancy nature, as well as in constructing variable-rate or unequal error protection codes. The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) Modifying and combining codes In this chapter, several techniques are presented that allow modification of an existing code, or the combination of several codes, in order to achieve flexibility in the design of error correcting codes. Many of the best codes known to date have beenobtained not from members of a known family of codes, but from modifying and combining codes[Bro]. 6.1 Modifyingcodes In the following, let C denote a linear block (n,k,d ) code overGF(q) with generator matrix G and parity-check matrix H . 6.1.I Shortening Shortened cyclic codes were described in Section 3.1.5. Let S be an integer, 0 5 S < k . In general, a linear shortened (n - S, k - S , d,) code C, has distance d, 2 d. A generator matrix of C, can be obtained fromthe generator matrix G of theoriginal code C as follows. Assume that G is in systematic form, that is, Then a ( k - S ) x ( n - S) generator matrix G, of the shortened code C, is obtained by removing S columns of the identity matrix Ik and the S rows that correspond to where the selected columns elementsare nonzero. Thisis best illustrated by an example. Example 68 Consider the Hamming (7,4,3) code, presented in Example 13, page 23, of Chapter 2. This codehas G= ( 1 0 0 1 0 0 0 0 1 1 0 1 1 1 0 0 0 1 0 1 1 (Compare with the H matrix in Example 13.) To construct a shortened (5,2,3) code, any two among thefour leftmost columns of G can be removed. Suppose that the first and second columns of G are removed, and that the first and second rows, corresponding to the nonzero elementsof the columns, are removed. These rows and columns are indicated by boldface types in (6.2). The remaining entries of G form THE ART OF ERROR CORRECTING CODING 102 the matrix It isinstructive to examine the standard array of the shortened (5,2,3) code of Example 68, to understand the enhancement in the error correcting capability of a shortened code, with respect to the original code. Example 69 The standard array for the shortened ( 5 , 2 , 3 ) code of Example 68 is given in Table 5 . Table 5 Standard array for a shortened ( 5 , 2 , 3 ) code. S - u=oo 000 110 01 1 l00 010 00 1 101 111 00000 10000 01000 00100 00010 00001 11000 01100 u=o1 u=10 10110 00110 01011 11011 00011 01111 01001 01010 10011 001 11 111 10 10010 10100 10111 01 110 11010 U = l l 11101 01101 10101 11001 11111 11100 00101 10001 From the standard array, it follows that, although the minimum distance of the code is d, = 3, there are two error patterns of Hamming weight two, namely 11000 and 01100, that can be corrected. Note that shortening a code reduces the length and dimension of the code, while at the same time maintains the same redundancy, so that more error patterns can be corrected. This can be explained from the Hamming bound for a t-error-correcting linear block ( n ,k , d ) code, inequality (1.24) which is repeated here for convenience, t 2n-k 2 2 (;). With respect to the original (n,IC, d ) code, the shortened ( n - S , k - S , d,) has the same redundancy. Therefore, the left-hand side of (6.3) has the same value. In other words, the number of cosets doesnot change. On the other hand, for S > 0, the right-hand side becomes smaller. In other words, the number of error patterns of Humming weight up to t decreases. If the original code does not meet the Hamming bound (6.3) with equality (in the binary case, only the Hamming codes, the Golay code, the single parity-check codes andthe repetition codes do), then the quantity 2n-k )(: represents the additional patterns of weight greater than t that the original code can correct. The number of additional error patterns of Hamming weight greater than t that the shortened code can correct is given by cl=, A, = 2n-k - 5 (" e ') !=O In-' 5 (i)] 2 [ (i)(" ')] t - - = - , (6.4) t=0 which is equal to the difference in the volume of Hamming spheres of radius t when going from n to n - S coordinates (or dimensions). 103 MODIFYING AND COMBINING CODES 6.I .2 Extending In general, extending a code C means adding E parity-check symbols. The extended ( n E, k , dext)code Cexthas minimum distance dext 2 d. In terms of the parity-check matrix H of the code C, E rows and columns are added to it, to obtain an ( n - k E ) x (n + E ) extended parity-check matrix, + + ... H ... hn--ktt,t The most common way of extending a code is by adding an overall parity-check symbol. In this case, the following parity-check matrix is obtained, The resulting code Cext is an ( n + 1;k , dext)code. If the minimum distance of the original code is odd, then dext = d 1. + Example 70 Let C be the Hamming (7,4,3) code. Then the extended (8,4,4) code Cexthas parity-check matrix / 1 1 1 1 . 1 1 1 1 \ 0 1 1 1 0 1 0 0 Hext = 0 0 1 1 1 0 1 0 0 1 1 0 1 0 0 1 J After permutation of columns, this is also the generator matrix of the Example 16, which is a self-dual code'. RM1,3 code of 6.1.3 Puncturing A puncturing technique was discussed in Section 5.5, in connection with convolutional codes. More generally, puncturing of linear block codes consists of the removal of parity check symbols, to obtain a linear block (n - p , IC, d,) code C, with minimum distance d, 5 d. The rate of the code increases, since the dimension does not change, and redundancy (number of parity-check symbols) is reduced. Puncturing is achieved by removing certain columns of the parity-check matrix H of the original code, to obtain matrix H p of C,. This technique is similar to shortening of the dual code. If H = ( P O 172 ... P k - l l I n - k ) 7 See page 7. 104 THE ART OF CORRECTING ERROR CODING denotes the parity-check matrix of C, then removing any p columns among the n - IC rightmost columns from H , and p rows corresponding to the nonzero values of the selected columns, results in an ( n - IC - p ) x ( n - p ) matrix where Ij denotes a column of weight equal to one, and p> denotes a column after the removal of the p rows described above. Example 71 Consider the ( 5 ;2 ; 3) code C, from Example 68. Its parity-check matrix is equal to (:1 : 1) H , = l l O l O . Deleting the third column and the top row of H , gives the parity-check matrix H p of a punctured (4: 2 ; 2) code C,, which is the same LUEP code as in Examples 3 and 6, up to a permutation of the first and second code positions. 6.1.4 Augmenting and expurgating There are two additional techniques to modify an existing linear code that in general may produce nonlinear codes. These techniques are augmenting and expurgating and are very useful in analyzing families of linear block codes, such as the Reed-Muller and BCH codes. Augmenting a code means adding more codewords to it. An obvious way to do this and still get a linear code is to add more (linearly independent) rows to the generator matrix. This is equivalent to forming a supercode by taking the union of cosets of code C. The resulting code C,,,, has dimension (n;IC 6;d,,,), where 6 is the number of rows added to G. The minimum distance of Caugis daug5 d. Let G = (3: . . . ) T be a generator matrix of C, where gj denotes a vector of length n, 0 5 j < n, and denotes the transpose of a vector 3. Then a generator matrix of an augmented code is given by + ' x Augmenting techniques are closely related to coset decomposition [For61 of codes. This is discussed below in connection with the direct-sum of codes and the lulu v]-construction and related techniques. A common method to augment a linear block (n; k ;d) code C is to add the all-one codeword to it (if it is not already in the code.) Alternatively, if the all-one codeword belongs to C, then + c = c. U{i + co), for an ( n ;IC - 1;do) subcode COC C, where do = d - 1. 105 MODIFYING AND COMBINING CODES Table 6 Basic techniques for modifying a code. Technique Shortening Lengthening Extending Puncturing Augmenting Expurgating Action Removing information symbols Adding information/parity-check symbols Adding parity-check symbols Removing parity-check symbols Adding codewords (cosets) Removing codewords (cosets) Code parameters (n-C,k-l,d, > d ) ( n + C, IC l ,d, 5 d ) ( n + e, IC, dext 2 4 .( - e, k,d p 5 4 (n,IC + e, daug 5 4 ( n , - e, dexp 2 d ) + Expurgating is the process of removing codewords from a code. In general, this will produce a nonlinear code. One way to get a linear code is to remove rows from the generator matrix G of the original code. This results in an (n,IC - C,de,,) code Cexpwith dexp 2 d. It is interesting to note that if the code is systematic, then expurgating is equivalent to shortening. Example 72 Consider the Hamming (7,4,3) code, with G= ( 1 0 0 0 1 0 1 0 1 0 0 1 1 1 0 0 0 1 0 1 1 Removing the first rowresults in a (7,3,3) code, for which the firstbit is always zero. In other words, the code isequivalent to a (6,3,3) code. It is important to note that when expurgating a code, the choice of the generator matrix, or the basis of the original code, is important in order to obtain codes with high minimum distance. This is illustrated in the following example. Example 73 As shown inExample 39, the Hamming (7,4,3) code contains seven codewords of weight 3 and seven codewords of weight 4. Selecting four linearly independent codewords, three of weight 4 and one of weight 3 , the following generator matrix is obtained, G’= ( 1 0 0 0 1 0 1 1 1 0 0 1 1 0 (6.10) 0 1 1 0 0 1 1 Matrix G’ generates the same Hamming (7,4,3) of the previous example. However, if the top row of G’ is removed, the generator matrix of a maximum-length-sequence (or simplex) (7,3,4) code is obtained. e The code lengthening technique is performed by adding columns and e rows to the generator matrix, and C additional parity-check symbols, so that C additional information symbols are introduced. In particular, one way to lengthen a code is to add one information symbol and one overall parity-check symbol [MS]. Table 6 summarizes the techniques presented in this section. THE ART OF CORRECTING ERROR 106 CODING 6.2 Combining codes In this section, several methods of combining codes are presented. Code combining techniques are very powerful, as evidenced by the appearance in 1993 of turbo codes [BGT]. In the following, unless otherwise specified, let Ci denote a linear block (ni,ICi, d i ) code, i = 1,2. 6.2. l Time-sharing of codes Consider two codes C1 and C2. Then the time-sharing of C1 and C2 consists of transmitting a codeword C1 E Cl followed by a codeword c2 E C Z , IC,lC,l = {(c1,c2) : ci E cz,i = 1,2}. (6.11) The result of time-sharing of m linear block (ni,k i , d i ) codes, i = 1 , 2 , . . . , m, is an (n,IC, d ) code, with parameters m m n= E n , , IC = c k i , and d = i=l i=l min { d i } . (6.12) lsilm Let Gi denote the generator matrix of component code Ci, for i = 1 , 2 ,. . . ,m. Then the generator matrix of the code obtained from time-sharing is (6.13) where the blank entries represent zeros. Time-sharing is sometimes referred to as “direct-sum’’ [MS] or “concatenation” [Rob]. In this book, however, concatenation of codes has a different interpretation, and is discussed in Section 6.2.4. Example 74 Let C1 be a repetition (4,1,4) code and C2 be a Hamming (7,4,3) code. Then time-sharing of C1 and C2 results in a linear block (11,5,3) code with generator matrix GTS = 0 G2 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 1 1 1 0 0 0 1 1 1 0 1 1 0 1 The time-sharing technique has been widely used in communication systems that require a variable amount of error protection, or unequal errorprotection, using the RCPC codes of Section 5.5.2. See [Hag]. Also note that time sharing m times the same code is equivalent to repeating the transmission of a codeword m times. More on this when product codes are discussed, in Section 6.2.3. 107 MODIFYING AND COMBINING CODES 6.2.2 Direct-sums of codes Let Ci denote a linear block (n;ki, d i ) code, 1 5 i 5 m. A direct-sum code CDSis defined as CDS = {fila = fil fil ' ' ' ; ,V fii E ci, i 1;2 , . . . ; m }. + + + 1 This technique may increase dimension. However, it generally reduces the distance. Let Gi denote the generator matrix of component code Ci, for i = l;2; . . . :m. Then the generator matrix of the code CDS,obtained from the direct-sum of these component codes, denoted CDS= Cl C2 ... C,, is + + + (6.14) GDS= Code CDSis a linear block (n:k7d ) code with k 5 IC1 + k2 + . . . + I C ,and d 5 mini{&}. Example 75 Let Cl be the repetition (4; 1: 4) code and CZbe a linear block (4; 2; 2) code with generator matrix (This code consists of codewords formed by sending twice a 2-bit message.) Then code CDS= C1 C, is a single parity-check (4: 3; 2) code, with generator matrix, + 1 1 1 1 G D S = (1 0 1 0 ) . 0 1 0 1 The direct-sum technique can be used not only to combine codes of smaller dimension, but also to decompose a code into a union of subcodes Ci C C , such that C can be expressed as the direct sum of the component subcodes. More on this is discussed in Section 6.2.4. Trivially, every linear block (n;k , d ) code C with generator matrix G can be decomposed into k linear block (n;1,d i ) subcodes Ci, 1 5 ,i 5 k . Each subcode has a generator matrix equal to a row g i , of G , 0 5 i < k . However, there are known construction techniques that allow decomposition of codes into subcodes of larger dimension and known minimum distances. This is the topic of the next section. The ) U ~ U+ v)-constructionand related techniques Combining time-sharing and direct-sum gives interesting construction methods. The first one is the 1u1u vl-construction [Plo], defined as follows. Based on two codes, C1 and CZ,of A lengthn = nl = n2,the Iulu+vl-constructiongivesacode*C = IC1IC1+C2I withgenerator matrix + (6.15) ~ If the lengths are not equal, say, 7 ~ 1> 712, then append c2. TLZ - nl zeros to the end of codewords of code 108 THE ART OF CORRECTING ERROR CODING + The parameters of C are (2n; IC1 kz d). That the minimum distance of C is d = min(2dl dz} can beproven by noticing that for twobinary vectors 5 and 1, w t ( 5 y) 5 wt(5) - wt(y). The lulu v(-constructionallows to express RM codes in a convenient way: + + Reed-Muller codes RM(r + l ; m + 1) = IRM(r + l;rn)lRM(r + 1;rn) + RM(r;m)l. (6.16) This interpretation of RM codes has been used to develop efficient soft-decision decoding methods taking advantage of the “recursive” structure of this class of codes (see [For6, SB] and references therein). In Section 6.2.4, this construction will be shown to be a particular case of a more powerful combining technique. The IuIu v)-construction is also called the squaring construction in [For6]. + Example 76 Let C1 be RM(1; 2), an SPC (4,3,2) code, and C2 be RM(0; a), the repetition (4; 1:4) code. Then C = IC1JC1 CZ)is RM(1;3), an extended Hamming (8:4:4) code with generator matrix 1 0 0 1 1 0 0 1 0 1 0 1 0 1 0 1 G= + ( 0 0 0 0 1 1 1 1 Construction X + This is a generalization of the lulu v)-construction [Slo]. Let Ci denote a linear block (ni;ki,d i ) code, for i = 1;2: 3. Assume that C3 is a subcode of C2, so that 723 = nz, IC3 5 kz and d3 2 dz . Assume also that the dimension of Cl is ICl = kz - IC3. Let ( G; G$ ) and G3 be the generator matrices of code CZ 3 C, and subcode C3, respectively. Note thatG2 is a set of coset representatives of C3 in C2 [For6]. Then the code CX with generator matrix (6.17) is a linear block (nl + 122; kl + IC2; d x ) code with d, = min(d3; dl +4 ) . Example 77 Let C1 be a single-parity check (SPC) (3; 2; 2) code, and CZbe an SPC (4; 3; 2) code whose subcode is C3, a repetition (4: l;4) code. Then 1 0 1 1 (::)=(o 1 0 0 1 1 1 01 1 l); and G3 = ( 1 1 1 l ). Construction X results in code CX = 1 Cl JCz+ C, 1 with generator matrix l 0 1 1 0 0 1 0 0 0 1 1 1 1 and is a maximum-length-sequence (7; 3: 4) code. This code is equivalent to thecode obtained from the Hamming (7; 4: 3) code by expurgating one message symbol, as in Example 73. MODIFYING AND COMBINING CODES 109 ConstructionX3 Extending further the idea of using coset representatives of subcodes in a code, this method combines three codes, one of them withtwo levels of cosetdecomposition into subcodes, as follows [Slo]. Let C3 be a linear block (n1 k 3 ; d3) code, where k3 = k2 a23 = kl a12 ~ 2 3 C3 . is the union of 2a23 disjoint cosets of a linear block (nl ,k2, d2) code, C2, with k:! = kl u12. In turn, C2 is the union of 2a12disjoint cosets of a linear block ( n l ,kl d l ) code, C l .Then each codeword in C3 can be written as5i+jji+V, with V E C l , where Zi is a coset representative of C2 in C3 and y i is a coset representative of Cl in C,. Let C4 and C5 be two linear block (n4;u33, d 4 ) and (n5 u12, d5) codes, respectively. The linear block (nl n4 725, k3, d ~ 3code ) Cx3 is defined as + + + + + + and has a minimum distance dx3 = min { d l is where ( G1 ), ( G: G: ) T and ( G: C, and C,, respectively. d2 G$ G: + d4, d3 + d5) . A generator matrix of (3x3 )T are the generator matrices of codes Cl, Example 78 Let C l , C2 and C3 be (64,30,14), (64,36, 12)and (64,39; 10) extended BCH codes, respectively, and let C4 and C5 be (7; 6.2) and (7; 3,4) SPC and maximum-length codes, respectively. Construction X3 results in a (78: 39,14) code. This code has higher rate (four more information bits) than a shortened (78: 35,14) code obtained from the extended BCH (128,85,14) code. Generalizations of constructions X and X3 and good families of codes are presented in [Slo, MS, Kasl, Sug, FL31. The application of these techniques to construct LUEP codes was considered in [Van, MH]. 6.2.3 Products of codes In this section, the important method of code combination known as product, is presented. The simplest method to combine codes is serially. That is, the output of a first encoder is taken as the input of a second encoder, and so on. This is illustrated for two encoders, in Figure 53. This is a straightforward method to form of a product code. Although simple, this direct product method produces very good codes. Very low-rate convolutional codes can be constructed by taking products of binary convolutional codes and block repetition codes. Example 79 Consider the standard memory-6 rate- 1/2 convolutional encoder with generators (171, 133)and minimum distance d f = 10, connected in series with time-sharing of repetition (2: 1:2) and (3: 1,3) codes, namely 1(2: 1,2)\(2;1;2)1 and l(3; 1,3)1(3,1;3)l. In other words, every coded bit is repeated two or three times, respectively. These schemes produce a memory-6 rate-114 code and a memory-6 rate-1/6 code, with generators (171; 171,133,133) and (171; 171: 171; 133,133,133), anddf= 20anddf = 30, respectively. These codes are optimal [Dho] in the sense that they havethe largest free distance THE ART OF ERROR CORRECTING CODING 110 Figure 53 Block diagram of an encoder of a product code. 012 k,-l n , -1 0 1 2 horizontalchecks B verticalchecks m checks-on-checks Figure 54 Codeword of a two-dimensional product code. for a given number of states. This seems to be the first time that they have been expressed in terms of these generators. However, exceptfor the case of twoencoders where the secondencoder is the time-sharing of repetition codes, important questions arise when considering aserial connection3 between two encoders: How should the output of the first encoder be fed into thesecond encoder? In what follows, let C1 denote the outer code and C2 denote the inner code. Either C1 or C2 or both can be convolutional or block codes. If G1 and G2 are the generator matrices of the component codes, then the generator matrix of the product code is the Kronecker product, G = G1 @ G2. In 1954, Elias [Elil] introduced product or iterated codes. The main idea is as follows. Assume that both C1 and C2 are systematic. The codewords of the innercode C1 are arranged as rows of a rectangular array with 721 columns, one per code symbol in a codeword of C,. After k2 rows have been filled, the remaining 722 - IC2 rows are filled with redundant symbols produced, on a column-by-column basis, by theouter code C,. The resulting 722 x n1 A rectangular array is a codeword of the product code Cp = C1 @ C,. Figure 54 depicts the structure of a codeword of a two-dimensionalproduct code. Extension to higher dimensions is straightforward. This is also known as serial concatenation. However, in this chapter the term concarenafion is used with a different meaning. MODIFYING AND COMBINING CODES 111 Block interleaver ... .~ ~~ ~ n k, codewords of length n l 0 Encoder C, n2 codewords of length n l n I messages of length k, Figure 55 A two-dimensional product encoder with a block interleaver. j=O j=1 j=2 j=3 j=4 i=o i= 1 Figure 56 A 2-by-5 block interleaver. The array codewords are transmitted on a column-by-column basis. With reference to the initial description of a product code, Figure 53, Elias two-dimensional product codes can be interpreted as connecting the two encoders serially, with an interleaver in between. This is shown schematically in Figure 55. As defined by Ramsey [Ram], an interleaver is a device that rearranges the ordering of a sequence of symbols in some one-to-one deterministic manner. For the product of linear block codes, naturally, the device is known as a block interleaver. The interleaver describes a mapping mb(i,j) between the elements ai,j in a IC2 x n1 array, formed by placing IC2 codewords of C1 as rows, and the elements u ~ ~ of ( an ~ ,information ~ ) vector U. = ( U 0 U1 . . . un*nz-l). The one-to-one and onto mapping induced by a m1 x 7132 block interleaver can also be expressed as apermutation Il : i e ~ ( i acting ) , on the set of integers modulo 71317722. Writing the array as a one-dimensional vector, ii, by time-sharing of (in that order) the first to themlth row of the array, U=(UO UI ... um1mz-1), the output of the block interleaver is read, via H, as where i ~ ( i=)m2(i mod m l ) + 1--J (6.19) m1 Example 80 Let Cl and C, be linear block SPC (5,4,2) and (3,2,2) codes, respectively. This results in a (15,8,4) product code with the block interleaver shown in Figure 56. The permutation is given by i ~ ( i=)5(i mod 2) + 1-1, 2 and the vector ii = ( U O ,211, u2, us,. . . , u g ) is mapped onto THE ART OF ERROR CORRECTING CODING 112 1 10 1 0 0 1 2 3 4 5 6 7 8 9 2 5 3 1 6 4 ' 5 2 7 6 3 7 8 8 4 91 9 1 Figure 58 Mapping mb(i,j ) of a 3-by-5 block interleaver. This is illustrated in Figure 57. The sub-vectors U i = (21&+5 ), 0 5 i < 5, constitute information vectors to be encoded by Cl. Traditionally, codewords are interpreted as twodimensional arrays, at,^ . . . ae,4 ) E Cl, for C = 0,1,2, and the columns Cz! for C = 0,1, . . .4. The underlying ordering is depicted in Figure 58. where the rows (at,-, ( ao,! a1,t ~ 2 , )e E The one-dimensional notation gives the same vector, Example 81 Let Cl and C2 be two binary SPC (3,2,2) codes. Then Cp is a (9,4,4)code. Although this code has one more redundant bit than an extended Hamming code (or the RM( 1,3) code), it can correct errors very easily by simply checking the overall parity of the received rowsand columns. Let the all-zero codeword betransmitted over a BSC channel and suppose that the received codeword is F= (1 1) 1 0 0 . 113 MODIFYING AND COMBINING CODES i=O 0 . 6 12 '\ '\ i=2 5 . 3 7 ;'l3 i=l 1. 10 11 2 8 ,, 9 \ 4. 14 ': Figure 59 Cyclic mapping m, for n1 = 5 , 7 2 2 = 3. Recall that the syndrome of a binary SPC ( n ,n - 1 , 2 ) code is simply the sum of the n bits. The second row and the first column will havenonzero syndromes, indicating the presence of an odd number of errors. Moreover, since the other columns and rows havesyndromes equal to zero, it is concluded correctly that a single error must have occurred in the first bit of the second row (or the second bit ofthe first column). Decoding finishes upon complementing the bit in the located error position. The code in Example 81 above is a member of a family of codes known as array codes (see, e.g., [Kas2, BL]). Being product codes, array codes are able to correct bursts of errors, in addition to single errors. Array codes have nice trellis structures [HMI, and are related to generalized concatenated (GC) codes [HMF], which are the topic of Section 6.2.4. Let Ci be a linear block (ni,ki,d i ) code, i = 1,2. Then the product Cp = C1 63 C2 is a linear block (nln2, ICl IQ, d p ) code, where d p = dl d2. In addition, Cp can correct all bursts of errors of length up to b = max(nlt2, nztl}, where ti = L(di - 1)/2], for i = 1,2. The parameter b is called the burst error correcting crzpability. Example 82 Let Cl and C2 be two Hamming (7,4,3) codes. Then Cp is a (49,16,9) code that is capable of correcting up to 4 random errors and bursts of up to 7 errors. If the component codes are cyclic, then the product code is cyclic [BW]. More precisely, let Ci be a cyclic (ni,Ici, d i ) code with generator polynomial i = 1,2. Then the code Cp = C1 63 C2 is cyclic if the following conditions are satisfied: si(.), 1. The lengths of the codes Ci are relatively prime, i.e., an1 + bn2 = 1,for two integers a and b; 2. The cyclic mapping mc(i,j ) that relates the element a i j in the rectangular array of Figure 54 with a coefficient t ~ ~ ~of (a code ~ , polynomial ~ ) U ' (). = ZIO 01 . .. tIn,,,-1.n1n2--l E C p , is such that + + + m c ( i , j )= [ ( j - i) . bnl + i] mod n l n ~ , (6.20) form,(i,j) = 0 , 1 r . . . , n l n 2 - 1. When these two conditions are satisfied, the generator polynomial of the cyclic code Cp is given by g(.) = GCD + ( g l ( ~ ~ ~ ~ ) g ~ 1() .5 ~ ~ ~ ) , 2 (6.21) ~ ~ ~ ~ THE ART OF ERROR CORRECTING CODING 114 Figure 60 Codeword of an block interleaved code of degree I = nz. Example 83 An example of the cyclic mapping for n1 = 5 and 722 = 3 is shown in Figure 59. In this case, (-1)5 + (2)3= 1,so that a = -1 and b = 2. Consequently, the mapping is given by m c ( i , j )= ( 6 j - 5i) mod 15. As a check, if i = l and j = 2, then m c ( l , 2 ) = (12 - 5) mod 15 = 7 ; if i = 2 and j = 1, then mc(2,1) = (6 - 10) mod 15 = -4 mod 15 = 11. The mapping m c ( i , j )indicates the order in which the digits of the array are transmitted [BW]. This is not the same as the column-by-column order of the block interleaver for a conventional product code. The mapping described by (6.20) is referred to as a cyclic interleaver. Other classes of interleavers are discussed in Section 6.2.4. With the appearance of turbo codes [BGT] in 1993, there has been intense research activity in novel interleaver structures that perform a pseudo-random arrangement of the codewords of C l , prior to encoding with C,. In the next section, interleaved codes are presented. Chapter 8 discusses classes of interleaver structures that are useful in iterative decoding techniques of product codes. Block interleaved codes A special case of product code is obtained when the second encoder is the trivial (n2,122,1) code. In this case, codewords of C1 are arranged as rows of an na-by-nl rectangular array and transmitted column-wise, just as in a conventional product code. The value I = 722 is known as the interleaving degree [LC] or interleaving depth. The resulting block interleaved code, henceforth denoted as canbe decoded using the same decoding algorithm of Cl, after reassembling a received word, column-by-column and decoding it row-by-row. Figure 60 shows the schematic of a codeword of an interleaved code, where ( W ~ , O W ~ , J . . . ) E Cl, for0 5 i < 722. If the error correcting capability of C1 is tl = L(dl - 1)/2], then can correct any single error burst of length up to b = tln2. This is illustrated in Figure 61. Recall that the transmission order is column by column. If a burst occurs, but it does not affect more than bl positions per row, then it can be corrected by Cl. The maximum length of such a burst of errors is 712 times b l . Moreover, if code C1 can already correct (or detect) any single burst of length up to b l , then C$nz)can correct (or detect) any single burst of length up to bln2. If C1 is a cyclic code, then it follows from (6.21) that is a cyclic code with generator polynomial g l ( z n z ) [PW, LC]. This applies to shortened cyclic codes as well, andthe following result holds ([PW], p. 358): MODIFYING AND COMBINING CODES 115 Figure 61 A correctable error burst in a block interleaved codeword. Interleavingashortenedcyclic (n:IC) codetodegree e producesa is e times shortened (ne,ke) code whose burst error correcting capability that of the original code. Finally, note that the error correcting capability of a product code, t p = L(dld2 - 1)/2], can only be achieved if a carefully designed decoding method is applied. Most of the decoding methods for product codes use a two-stage decoding approach. In the first stage, an errors-only algebraic decoder for the row code C1 is used. Then reliability weights are assigned to the decoded symbols, based on the number of errors corrected. The more errors are corrected, the less reliable the corresponding estimated codeword 61 E C1 is. In the second stage, an errors-and-erasures algebraic decoder for the column code C, is used, with an increasing number of erasures declared in the least reliable positions (those positions for which the reliability weights are the smallest), until a sufficient condition on the number of corrected errors is satisfied. This is the approach originally proposed in [RR, Well. The second decoding stage is usually implemented with the GMD algorithm, which is discussed in Section 1.6. More on decoding of product codes can be found in Chapter 8. 6.2.4 Concatenatedcodes In 1966, Forney [Forl] introduced a clever method of combining two codes, called concatenation. The scheme is illustrated in Figure 62. Concatenated codes4 that are based on outer Reed-Solomon codes and inner convolutional codes have been to date’ perhaps the most popular choice of ECC schemes for digital communications. In general, the outer code, denoted Cl, is a nonbinary linear block ( N ,K: D)code over GF(2’). The codewords of C1 are stored in an interleaver memory. The output bytes read from the interleaver are then passed through an encoder for an inner code, C2. The inner code C2 can be either a block code or a convolutional code. When block codes are considered, and C2 is a binary linear block (n:IC, d ) code, the encoder structure is shown in Figure 62. Let C = C1 * C2 denote the concatenated code with C1 as the outer code and C2 as the inner code. Then C is a binary linear block ( N n ,Klc, Dd) code. The purpose of the interleaver between the outer and the inner code is two-fold. First, it serves to convert the bytes of size IC into vectors of the same dimension (number of information bits) as the inner code, be it binary or nonbinary, a linear block (n;IC’: d ) code or a rateAlso referred to by some authors as cascaded codes. Before the arrival of turbo codes and LDPC codes. 116 Outer encoder K messagesL of length k bits C, (N,K,D) GWk 1 - THE ART OF ERROR CORRECTING CODING Inner encoder n -- C 2 I I I I I I I I 1 byte N codewords of length n bits (n,k,d) W2) ' k bits Figure 62 An encoder of a concatenated code. I / D 2a O2 3 4 D D D D \ 3 D 40 Figure 63 Aconvolutionalinterleaver. k'ln convolutional code, for which in general k' # k. On the other hand, as discussed in the previous section, interleaving allows breaking of bursts of errors. This is useful when concatenated schemes with inner convolutional codes are considered, because the Viterbi decoder tends to produce bursts of errors [CY, Mor]. There are several types of interleavers that are used in practice. The most popular appears to be theconvolutional interleaver [ForS], which is a special case of a Ramsey interleaver [Ram]. The basic structure of a convolutional interleaver is shown in Figure 63. The deinterleaver structure is identical, with the exception that the switches are initially in position M and rotate in the opposite direction. An important advantage of concatenated codes (and product codes) is that decoding can be based on the decoding of each component code. This results in a dramatic reduction in complexity, compared to a decoder for the entire code. Example 84 Let Cl be a (7:5 : 3) RS code6 with zeros (1;a } , where a is a primitive element of G17(2~),and cy3 + a + 1 = 0. Let C2 be the maximum-length-sequence (7:3: 4) code of Example 77. Then C = C1 * C2 isa binary linear block (49: 15,12) code. This code has six information bits less than a shortened (49: 21,12) code obtained from the extended BCH (64,36;12) code. However, it is simpler to decode. Let V ( x ) = (x4 a 4 ) 9 ( x )= a5 x + a4x2 ax4 a3x5+ x6 be a codeword in the RS (7:5: 3) code, where g(x) = x2 + a3x + a. + + RS codes are the topic of Chapter 4. + + MODIFYING AND COMBINING CODES 117 a5 1 a4 0 a a3 1 1 0 1 0 0 0 0 1 0 1 0 1 1 0 1 1 0 0 0 1 1 0 1 1 0 0 1 1 0 1 1 0 1 0 1 0 0 0 0 1 1 0 1 1 0 0 1 0 1 Figure 64 A codeword in the concatenated code C1 * C2, with C1 the RS(7,5,3) code over G F ( L ? and ~ ) C2 a binary cyclic (7,3,4) code. Using the table on page 41, the elements of GF(23)can be expressed as vectors of 3 bits. A 3-by-7 array whose columns are the binary vector representations of the coefficients of the code polynomial a(z) is obtained. Then encoding by the generator polynomial of C, is applied to the columns to produce 4 additional rows of the codeword array. For clarity, the following systematic form of the generator matrix of C2 is used, which is obtained after exchanging the third and sixth columns of G in Example 77, G’= 1 1 0 0 1 0 1 1 0 1 0 0 1 1 1 . 0 0 1 1 1 0 1 ( Figure 64 shows the codeword array corresponding to V E Cl. 6.2.5 Generalizedconcatenatedcodes In 1974, B l o b and Zyablov [BZ] andZinov’ev [Zin] introduced the powerful class of generalized concatenated (GC) codes. This is a family of error correcting codes that can correct both random errors and random bursts of errors. As the name implies, GC codes generalize Forney’s concept of concatenated codes, by the introduction of a subcode hierarchy (or subcode partition) of the inner code Cl and several outer codes, one for each partition level. The GC construction combines the concepts of direct-sum, or coset decomposition, and concatenation. Before defining the codes, some notation is needed. A linear block (n;IC, d) code C is said to be decomposable with respect to its linear block (n;ki; d i ) subcodes Ci, 1 5 i 5 M , if the following conditions are satisfied: (S~~)C=C~+CZ+-..+C,~~; ( D ) F o r . E-i i E C i , l I i I M , v l + V 2 + . . . + V ~ = O i f a n d o n l y i f 4 = V 2 = . . . = a,q4 = 0. For 1 5 i 5 M , let C I be ~ a linear block ( n ICli) ~ code over G F ( q )such that (DI)for21iECliwithlIi<M,21l+212+...+21lM=O,ifandonlyif21i=O forl<i<M. 118 THE ART OF CORRECTING ERROR CODING Let Si denote the minimum Hamming distance of the direct-sum code C I ~ + C I ~ +. ~ .+CIM. +. Let Coi be a linear block (no;koi: doi) code over GF(q"'")and let C: = Coi * CIi. The generalized concatenated code C is defined as the direct-sum A C=Cy-+C,*+...+C&. (6.22) Then, the condition (D) on C follows from the condition (DI). The minimum Hamming distance d of C is lower bounded [TYFKL] as (6.23) As with (single level) concatenated codes, a main advantage ofGC codes is that multistage decoding up to the distance given by the right-hand side of (6.23) is possible [TYFKL, MFKL]. Unequal error protection Another advantage of this class of codes is that it isrelatively easy to coordinate the distances of the outer and inner codes to obtain linear block or convolutional codes with unequal error protection (UEP) capabilities. If the direct-sum conditions above are satisfied, and in addition the products of the minimum distances satisfy the following inequalities, &dol 2 S2d02 2 . . . 2 S M d o M ; (6.24) then codewords in correspondence with k o i k ~ isymbols over G F ( q ) willhavean error correcting capability, [(Sidoi - 1)/2], that decreases with the level i , for 1 5 i 5 M . As a result, the messages encoded in the top (low values of i ) partition levels will have enhanced error correcting capabilities, compared to those associated with the lowest partition levels. Constructions of this type are reported in [DYS, MH]. A construction Let a linear block (721, k1, d l ) code C1 over GF(q) be partitioned into a chain of M ( n I , ki, d i ) subcodes Ci, i = 2; 3; . . . M 1, such that + Cl 3 c 2 3 . . . 3 CM+1; A A where, for convenience, C M += ~ {O}, and dM+l = m. Let C I = ~ [Ci/Ci+l]denote a linear block ( n ~k ~; iS;i ) subcode of Ci, which is a set of coset representatives of Ci+1 in Ci, of dimension k ~ = i ki - ki+l and minimum Hamming distance Si 2 di. Then C1 has the following coset decomposition [For61 Cl = Cl1 + Cl2 + . . + CIM. ' (6.25) Let Coi denote a linear block (no,koi dei) code Coi over GF(qk1< ), where k ~= i dim(Ci/Ci+l)= ki - ki+l, i = 1;2 ; . . . M . Then the direct sum of concatenated codes c=CO,*CIl+Co2*CI2+"'fCOM*CIM. is an non^; k i d ) linear block code of dimension andminimum respectively [BZ], c Hamming distance, M k= k~ikoi; i=l and d 2 l<i<M min {Sidoi}. (6.26) MODIEYING AND COMBINING CODES 119 Figure 65 Encoder structure of a generalized concatenated code with M partition levels. Note that equality holds in (6.26)when Cri, 1 5 i 5 M , contains the all-zero codeword. As for the choice of component codes. Note that, since in general the dimensions of the coset representatives are distinct, the outer codes can be selected as RS codes or shortened RS codes. Binary RM codes are good candidates as inner codes, because they have the following subcode property, RM(r, m) C RM(r 1, m), + for 0 5 T < m, and m > 2. Example 85 Consider the r-th order RM codes of length 8, RM(r, 3). Then RM(3,3) 3 RM(2,3) 3 RM(1,3) 3 RM(O,3) 3 {O}, and it follows from (6.25)that This decomposition can bealso appreciated from the generator matrix of RM(3,3),expressed as 0 0 0 0 0 0 0 1 - 0 0 0 0 0 0 l 1 0 0 0 0 0 1 0 1 0 0 0 1 0 0 0 1 > 0 0 0 0 1 1 1 1 0 0 l 1 0 0 l l 0 1 0 1 0 1 0 1 1 \l 1 1 1 1 1 1 11 where Gi has been defined as the generator matrix of the set of coset representatives of RM(3 - i, 3) in RM(3 - i + 1,3), or [RM(3 - i 1, m)/RM(3- i, m)], for i = 1,2,3, and where G4 is the generator matrix of RM(0,3). + THE ART OF CORRECTING ERROR 120 CODING According to this partition of RM(3,3), a GC code can be designed with up to four levels. Note that the outer codes should be over GF(2), for the first and fourth partition levels, and over GF(23) for the second and third partition levels. Also, some of the subcodes of RM(3,3) themselves can be usedas inner codes to obtain a GC code with a reduced number of levels. If RM(2,3) is used as the inner code of a GC code, then the number of partition levels is three, as the generator matrix of RM(2,3) is obtained from that of RM(3,3) by removing G1 (the top row). Let RM(2,3) be selected as the inner code, and an RS (8,1,8)code Col over G F p 3 ) , an RS (8,5,4) code C02 over GF(23), and a binary linear SPC (8,7,2) code C03 be selected as outer codes. This gives a binary linear GC (64,25,16) code which has one more information bit than thebinary extended BCH (64,24,16) code. It is now shown how Reed-Muller codes can be expressed as GC codes. Recall from Section 6.2.2, Equation (6.16) on page 108, that the (T + l)-th order RM code of length 2m+1 can be expressed as RM(T 1 , m 1) = IRM(T l,m)JRM(T 1,m ) RM(r, m ) [ Let . G(T,m ) denote the generator matrix of RM(r, m ) .Then + G(. + + 1 , m + l) L + = (G(.+Ol,m)G(T = G(r+l,m) Encoder RM(r,m) - i Encoder M ( r + 1,m) + + + 1,m) G(r, m) (i 2) (,,0 +G(T,~) 0 (6.27) - - 0 00 1-11 + 0 - 00 1-01 Figure 66 An encoder for the binary R M ( r 1,m GC code. + 1)code when viewed as a two-level This expression holds for each r-th order RM code, with r 2 l and m > 1, so that a recursion is obtained. From (6.27) it follows that RM(T 1,m 1) is a GC code with inner codes the (2,2,1) code and its partition into cosets of the repetition ( 2 , 1 , 2 ) code, and outer codes RM(T,m ) and RM(T 1,m ) ,respectively. An encoder for RM(T 1,m 1)is shown in Figure 66. This recursive structure of RM codes is advantageous when designing softdecision decoding algorithms for RM codes7, which can be based on simple repetition and parity-check codes. This is precisely what was done in [SB]. + + + + ~~ Soft-decision decodingis presented in Chapter7. + The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) 7 Soft-decision decoding In this chapter, decoding with soft information from the channel is considered. For simplicity of exposition, binary transmission over an AWGN channel is assumed. In Chapter 9, the general case of multilevel transmission is discussed. To provide a motivation for the use of soft decision decoding, it can be argued that the noise environment in data retrieval or signal reception is continuous in nature, not discrete. This means that the received symbols are (quantized) real numbers (voltages, currents, etc.), and not binary or GF(2m)symbols. When making hard decisions on the received symbols, on a symbol-by-symbol basis, errors may be introduced. This is illustrated in Figure 67. Binary sequence Real sequence Binary sequence Figure 67 Example of an error introduced by hard-decision decoding. Basically, there are two methods of decoding an error correcting code, based on a received real-valued sequence: Hard-decision decoding (HDD): When hard decisions are made on the channel values, errors are introduced. The goal of HDD is to correct binary errors induced in the hard-decision process. The first part of the book was devoted to different techniques of HDD for linear block codes, cyclic codes and convolutional codes. Soft-decision decoding (SDD): In this case, received values from the channel are directly processed by the decoder in order to estimate a code sequence. A special case of SDD is maximum-likelihood decoding (MLD), in which case the most-likely code sequence is output by the decoder. An important point to keep in mind here is that SDD needs to know the statistics of the channel noise. In general, SSD is computationally more intensive than HDD. There are two main reasons for this. One is that SDD needs to operate on real numbers. In a practical application, these 122 THE ART OF ERROR CORRECTING CODING numbers will be quantized to a finite number of bits. In somebinary transmission systems over an AWGN channel, it is known that 8 quantization levels, or 3 bits, are adequate to achieve practically the same performance as with unquantized channel values [Mas3, OCC]. The second reason for the increased complexity in SDD is that the a-posteriori statistics of the coded symbols, given the received values, need to be computed. However, in returnfor the increase in implementation cost there is potentially much better performance to be attained. As shown in Chapter 1, with binary transmission over an AWGN channel, to get the same error performance, SDD requires 2 to 3 dB less signal-to-noise power ratio than HDD. This is to say that, with SDD, the transmitted power can be 50% to 63% lower compared to HDD which translates into smaller transmit antennas or, alternatively, smaller receive antennas for the same transmission power. 7.1 Binary transmission over AWGN channels For AWGN channels, the a-posteriori probability of the received values, ri, given that code symbols wi are sent, is given by As shown in Chapters 1 and 5, the MLD metrics become the squared Euclidean distances, D2 ( r i ? m ( v i ) )= ( ~ -i m ( ~ i ) Let )~E . denote the energy of a transmitted symbol. The mapping rule or modulation employed is BPSK', G. which can be expressed as m(vi) = (-l)'% With BPSK transmission over an AWGN channel, decoding metrics for binary transmission can be simplified by noticing that if vi = 1 is transmitted then Taking the natural logarithm and removing constant terms, it follows that the log-likelihood metric -(r+l)' is proportional to - r . If vi = 0 then themetric is proportional to T . Therefore, the following is important to note: With binary transmission over an AWGN channel, metric computationis reduced to changing the signs of the received values. 7.2 Viterbi algorithm with Euclidean metric The VD can be used to decode convolutionally coded binary data, BPSK modulated, and transmitted over an AWGN channel. With respect to the hard-decision VD algorithm, studied in Section 5.4, two changes are required: Binary phase-shift keying. SOFT-DECISION DECODING 123 1. Branch-metric generator (BMG) stage For an AWGN channel, as shown in the previous section, the metric is proportional to the correlation between the received and the candidate code sequence. Therefore, instead of Euclidean distances, correlation metrics can be used. 2. Add-compare-select (ACS) stage: Instead of minimizing the distance, the VD seeks to maximize the correlation metric. Example 86 Consider a memory-2 rate-1/2 convolutional encoder with generators (7,5). Suppose that the information sequence is ti = (110100).The corresponding path is shown in Figure 68. 01 11 00 01 11 10 Figure 68 A path in the trellis corresponding to U. = (110100). The transmitted sequence (normalized with respect to a)is t(v) = (-1, -1,1, -1, l, -1,1,1, -1,1, -1, -1). Let the received sequence, after transmission over an AWGN channel and quantization to 3 bits, be: r = (-4, -1, -1, -3, +2, -3, +3, +3, -3, +3, -3, + l ) . Note that the hard-decision received sequence (given by the sign bits) is: Ff€ = ( 1 1 , ~ 1 , 0 1 , 0 0 , 1 0m), , so that the hard-decision detection process introduces two bit errors. The operation of the Viterbi decoder is illustrated in Figures 69 to 74. The evolution of the metric values with respect to the decoding stages is shown in the following table: State/Stage SjO’ Sp $1 $3) i=O 0 0 0 0 i=l i=2 +5 +7 +5 +3 +5 +3 +9 +7 The decoded information sequence is ii = ( 1 l corrected. i=3 +6 +l2 +8 +l4 0 i=4 i=5 +l2 +l4 +l8 +l4 +l4 +24 +l4 +20 1 0 i=6 +26 +l8 +22 +24 0 ), andtwo errors havebeen All the implementation issues related to Viterbi decoding, discussed in Sections 5.4.3 and 5.5.1, apply in the case of soft-decision. In particular, metric normalization must be carefully taken into account. THE ART OF CORRECTING ERROR CODING 124 Transmitted -I - l Received -4 - I -I +I + l -1 +l + I -1 + l -1 - l -1 -3 +2 -3 +3 c3 -3+3 -3 + I i=O 1 = change sign i = 1: Branch metric computation: IO) B M , =d(00,4-1 ) = - 5 121 BMI =d(10;4-1 BM, =d(01,-4-1 )=-3 13) )=+5 ) = + 3 BMI=d(11;4-1 (01 Compare Select = +5. select branch from state I (01 Update metric M(S I ) = +5 Updatepath y (0, - ( 1 1) Figure 69 Soft-decision Viterbi decoder operation at i = 1. Transmitted Received -1 -1 +l - 1 +l -1 +l +l - 1 +l -1 -1 4- 1 -1 -3 +2 -3 +3 +3 -3 +3 -3 + l i = 2: Figure 70 Soft-decision Viterbi decoder operation at i = 2. 125 SOFT-DECISION DECODING Transmitted -1 -1 + l -1 + l -1 +l +l -1 + l -1 -1 Received -4 -1 -1 -3 +2 -3 +3 +3 -3 +3 -3 +l -4 - 1 -1 -3 i = 3: +7 +2-3 Figure 71 Soft-decision Viterbi decoder operation at i = 3. Transmitted -1 - 1 +l -1 +l - 1 +l +l -1 +l -1 - 1 Received 4-1 -1 -3 +2 -3 +3 +3 -3 +3 -3 +l -4 - 1 -1 -3 +2-3 i=4: +6 +3+3 (0) y 4 = (10.1 1,00,00) (1) y 4=(ll,ll,ol,ol) (2) y 4 = (11,01,01.00) (3) y~=~ll,ll,ol,lo) Figure 72 Soft-decision Viterbi decoder operation at i = 4. I26 THE ART OF ERROR CORRECTING CODING Transmitted -1 -1 + l -1 + l -1 +l +l -1 + l -1 - 1 Received -4 -1 -1 -3 +2 -3 +3 +3 -3 +3 -3 + l -4 -1 -1 -3 +2 -3 +3 +3 i = 5: r,, +3 -3 (0) y5=(11,11,01,01,11) (1) y ,=(ll,Ol,Ol,00,lO) (2) y 5=(11,11,01,01,00) (3) y5=(11,11,O1,10,1O) Figure 73 Soft-decision Viterbi decoder operation at i = 5 . 127 SOJT-DECISION DECODING Transmitted -1 -1 + l -1 +l -1 +l +I -1 + I -I -I Received -4 -1 -1 -3 +2 -3 +3 +3 -3 +3 -3 + l -4 -1 - I -3 +2 -3 +3 -3+3 i=6: +3 +,4 -3+1 (0) y ,=(11,01,01,00,10,1 (1) y 6=(11,11,01,01,00,11 (2) y ,= (11,01,01,00,10,oc (3) y6=(11,11,01,10,10,1~ Figure 74 Soft-decision Viterbi decoder operation at i = 6. 128 THE ART OF CORRECTING ERROR CODING 7.3 Decoding binary linear block codes with a trellis The Viterbi algorithmcan also be applied to linear blockcodes. A syndrome trellis for a binary linear ( N ;K ) block code C can be constructed from its parity-check matrix as follows [Wol]. Let (v1 v2 . . . UN) denote a codeword of C. At time i, 1 5 i 5 N , states in the trellis are specified by thepartial syndromes: ~ i j=1 where the sum is over G F ( 2 ) ,hj is the j-th column of H , and 3; 2 (0 0 . . . 0). The maximum number of states in the syndrome trellis min is { 2 K , 2 N - K } . The syndrome trellis has the property that it has the smallest possible number of states. A trellis satisfying this condition is said tobe a minimal trellis. Example 87 Consider a binary cyclic Hamming (7;4; 3) code with g(z) = z3+ z + 1. Then h ( z ) = 1 + z + x2 + z4, and a parity-check matrix for C is i 1 1 1 1 0 1 0 0 H = 0 1 1 1 0 1 0 . 0 0 1 1 1 0 1 To label a state = (so s1 s2) an integer Ij is assigned, such that Ij = so + 51 x 2 + S2 x 22. The trellis has maximum of 8 states (since 2n-k = 23) at times i = 3 and i = 4, and is shown in Figure 75. Figure 75 Trellis-structure of a Hamming ( 7 , 4 , 3 ) code. It is interesting to note that with a syndrome trellis there is no need to label the branches with the codedbits. A transition between two states with the same label corresponds to coded bit 0, as seen from Equation (7.3):If the coded bit is0, then the sum does not change. 129 SOFT-DECISION DECODING Some observations about the trellis structure of linear block codes are the following: In general, the trellis of a binary cyclic code has an irregular structure. As a result, implementation of the Viterbi algorithm will be more intricate thanin the case of convolutional codes. For some classes of codes, such as extended2 BCH codes and Reed-Muller codes, the trellis can be divided into sections. This results in a more regular and symmetric trellis structure with many parallel subtrellises, which may be used to build very high-speed Viterbi decoders for block codes [LKFF, HMI. 7.4 TheChase algorithm The Chase algorithm [Cha] is a suboptimal decoding procedure that uses a set or list of most likely error patterns. These error patterns are selected based on the reliability of the received symbols. Each error pattern is added to the hard-decision received word anddecoded using a hard-decision decoder.Each decoded codeword is scored by computing its metric with respect to the received (soft-decision) sequence. The codeword with the best metric is selected as the most likely. Let C be a binary linear ( N ;K ;d ) block code, capable of correcting any combination of t = [ ( d - 1)/2] or less random bit errors. Let F = ( T I 7-2;. . . ,T N ) be the received word from the output of the channel, ri = wi,where wi is a zero-mean Gaussian random variable with variance N0/2, i = 1;2, . . . N . The sign bits of the received values represent the hard-decision received word, + where sgn(x) = 0; if x 2 0; 1; otherwise. The reliabilities of the received channel values, for binary transmission over anAWGN channel, are the amplitudes Iril. The received symbol reliabilities are ordered with a sorting algorithm (e.g., quick sort). The output of the algorithm is a list of indexes I j , j = 1;2; . . . ; N , such that IT11 I I 1% I L . . . 5 ITIN I. In the first round of decoding, the hard-decision received word ZO is fed into a hard-decision decoder. Let V0 denote the decoded codeword. Then the metric of V0 with respect to the received word F N rj i (7.5) j=1 is computed and its value stored as the maximum. Chase classified his algorithm into three types, according to the error pattern generation. 0 npe-I Test all error patterns within radius ( d - 1)from the received word. Extending codes is covered in Chapter 6. THE ART OF ERROR CORRECTING CODING 130 0 0 Type-I1 Test the error patterns with at most [ ( d - 1)/2] errors located outside the bits having the Ld/2] lowest reliabilities. Compared to Type-I, the performance of Chase TypeI1 algorithm is only slightly inferior, while at the same time having significantly reduced number of test patterns. Type-I11 Test those error patterns with i ones in the i least reliable bit positions, with i odd and 1 5 i 5 d - 1. It should be noted that this algorithm is closely related to the GMD decoding algorithm (see also [ML], p. 168). GMD decoding of RS codes is the topic of Section 7.6. Because of its reduced complexity and good performance, among the three types above, Chase algorithm Type-I1 is the most popular and described next. A flow diagram of Chase algorithm is shown in Figure 16. Chase type-I1 algorithm 0 0 0 For i = 1;2; ...: 2t - 1, an error pattern ei is added to the hard-decision received word: Zi = ei 8 Zo. The error patterns are generated among the t least reliable positions. That is, positions { I l ;Iz: . . . I t } , for which reliabilities (amplitudes) are the smallest. Each Zi is input to a hard-decision decoder, producing a codeword V i , i = 1;2; ...: 2t - 1. The metric is computed according to Equation (7.5) and if maximum, codeword 6i stored as the most likely. r Identify the T least reliable positions I Generate an error pattern =- e i in the T least reliable C -. . . .- ... postitions I I Stare r j Repeat 2' times Find a codewordTiusing hard-decision decoding - Compute the metric ..... output v Figure 76 Flow diagram of Chase type-I1 decoding algorithm. If desired, the algorithm can output a list of codewords. This is a useful feature in producing soft-outputs for use of the Chase algorithm in iterative decoding of block turbo codes, see Section 8.2.3. 131 SOFT-DECISION DECODING 7.5 Ordered statistics decoding The ordered statistics decoding (OSD) algorithm [ E l ] is similar to the Chase algorithm in the sense of creating a list of error patterns and using hard-decision decoding. However, unlike the algorithm in the previous section, which deals with least reliable code positions, OSD considers a list of codewords drawn from a set of (permuted) most reliable information positions. Assume that an ( N ,K ) linear block code C, with generator matrix G and minimum distance d H 2 2 t 1, is usedwith binary transmission over an AWGN channel. Let ?I = ( c l ,ea, . . . ,C N ) denote a codeword of C, and let F = ( T I , T Z , . . . ,T N ) be the received sequence. Asin the Chase algorithm above, the decoding process begins by reordering the components of the received sequence in decreasing order of reliability value. Let + (?h,y2>'..,!/N) denote the resulting sequence, where Iy1 I 2 1yzl 2 . . . 2 permutation mapping X 1 such that y = X 1 [F] . The next step is to permute the columns of G using X 1 : 1 1 ~ ~ This 1 . reordering defines a g; where denotes the j-th column of G'. A most reliable basis is obtained in the next step of the algorithm as follows: Starting from the first column of G', find the first K linearly independent (LI) columns with the largest associated reliability values. Then, these K L1 columns are used as the first K columns of a new matrix G , maintaining their reliability order. The remaining ( N - K ) columns of G" are also arranged in decreasing reliability order. This process defines a second permutation mapping X2, such that G" = X2 [G']= X2 [ X , [ G ] ] . Applying the map X2 to 9, results in the reordered received sequence: 2 = X 2 [g] = (Zl,Z2,.",Zk,Zkfl,.'.,ZN), with 1211 2 1221 2 . . . 2 1 ~ ~ and 1 , IZk+l) 2 . . . 2 1 ~ operations on G ' , a systematic form G1 can be obtained: G1=(Ik p)= 1 0 ... 0 0 1 ... 0 .. .. .. . . . . . p2,J .'. 1 pk,l ( 0 0 ~ By 1 . performing elementary row p1,1 . ... ... . ..' .. P1,N-K P2,N-K pk,N-K 1. It is easy to see that the code generated by G1 is equivalent to G , in the sense that the same codewords are generated, with permuted code bit positions. The next step is to perform hard-decision decoding based on the IC most reliable (MR) values of the permuted received sequence 2 . Let Ga = ( 2 1 1 , u2, . . . ,U K ) denote the result. Then, the corresponding codeword in C1 can be computed as v = CaG1 = ( V 0 V1 '.. VK-1 VK ..' UN). 132 THE ART OF CORRECTING ERROR CODING The estimated codeword G H D in C can be obtained from G, via the inverse mapping as v H D = X,' [X;' [G]] . (7.6) x,'x;', Based on the hard-decision-decoded codeword G, the algorithm proceeds to re-process it until either practically optimumor apredefined desired error performance is achieved. Order-Q re-processingis defined as follows: For 1 5 i 5 l,make all possible changes of i of the k MR bits of G. For each change, find the corresponding codeword G' in C1 and map it onto a binary real sequence z', via the mapping (0, l}e {+l,-l}. For each candidate codeword generated, compute the squared Euclidean distance, Equation (1.33) of Chapter 1, between the generated sequence 2' and the reordered received sequence Z . After all the C&,):( codewords have been generated and their distances compared, the algorithm outputs the codeword V* which is closest to Z. From G*, using the inverse map (7.6), the most likely code sequence ahL is computed. Re-processing can be done in a systematic manner to minimize the number of computations. In particular, as mentioned in Section 7.1, with binary transmission over an AWGN channel, there is no need to compute the Euclidean distance but rather the correlation between the generated codewords and the reordered received sequence. Thus the computation of the binary real sequence 2' is not needed. Only additions of the permuted received values zi,with sign changes given by the generated V*, are required. 7.6 Generalized minimum distance decoding In 1966, Forney [For31 introduced generalized minimum distance (GMD) decoding. The basic idea was to extend the notion of an erasure, by dividing the received values into reliability classes. The decoding strategy is similar to the Chase algorithm Type-111, with the use of erasure patterns. The GMD decoder works by declaring an increasing number of erasures within the d - 1 least reliable symbols, and testing a suficient condition for the optimality of the decoded word, until thecondition is satisfied or a maximum number of erasures have been considered. Let C be a linear block ( N ,K , d ) code. Assume that there isan errors-and-erasures decoder that is capable of decoding any combination of e errors and S erasures within the capability of the code, i.e., 2e S 5 d - 1. Such decoders were presented in Section 3.5.6 for BCH codes and in Section 4.3.2 for RS codes. Let F = ( T I , r2, . . . ,r ~ be) the receivedword from the output of the channel, ri = wi,where wi isa zero-mean Gaussian random variable withvariance N0/2, i = 1,2, . . . , N . It is assumed that F has been normalized (clipped) so that its components lie in the range [- 1,+l].As before, the sign bits of the received values represent the harddecision received word. + + As with Chase algorithms and the OSD algorithm, the reliabilities of the received channel values are sorted, producing a list of indexes I j , j = 1 , 2 , . . . , N , such that 133 SOFT-DECISION DECODING In the first round of decoding, the hard-decision received word E is fed into an errors-only algebraic decoder. Let 6 denote the resulting estimated codeword. The correlation metric of 6 with respect to the received word F N v= Z(-l)fij x (7.7) Tj, j=1 is computed. If the following sufficient condition is satisfied then 6 is accepted as the most likely codeword and decoding stops. Otherwise, a new round of decoding is performed. This is accomplished by setting S = 2 erasures, in positions I1 and I2, and decoding the resulting word with anerrors-and-erasures decoder. The correlation metric between F and the estimated codeword 6 is computed, as in (7.7), and then the sufficient condition (7.8) tested. This GMD decoding process continues, if necessary, everyround increasing the number of erasures by two, S = S 2, until the maximum number of erasures (smax= d - 1) in theleast reliable positions are tried. If at the end of GMD decoding no codeword is found, the output can be either an indication of a decoding failure or the hard-decision decoded codeword 60 obtained with S = 0. + 7.6.1 Suficient conditions for optimality The condition usedin GMD decoding can be improved and applied to other decoding algorithms that output lists of codewords, such as Chase and OSD. These algorithms are instances of list decoding algorithms. The acceptance criteria (7.8) is too restrictive, resulting inmany codewords rejected, possibly including the most likely (i.e., selected by true MLD) codeword. Improved sufficient conditions on the optimality of a codeword have been proposed. Without proofs, two suchconditions are listed below. Before their description, some definitions are needed. Let f represent a BPSK modulated codeword, f = m ( 6 ) , where 6 E C and z i = (-l)ui, for 1 5 i 5 N . See also (7.2). Let S, = {i : sgn(zi) # sgn(ri)} be the set of error positions, U = { I j , j = 1 , 2 , . . . ,d } the set of least reliable positions, and T = {i : sgn(zi) = sgn(ri), i E U } . Then the extended distance or correlation discrepancy between a codeword 6 and a received word F is defined as [TP], IES. Improved criteria for finding an optimum codeword are based on upper bounds on (7.9) and by increasing the cardinality of the sets of positions tested. Two improvements to Fomey’s conditions are: 0 Taipale-Pursley [TP] There exists an optimal codeword such that (7.10) (ET THE ART OF ERROR CORRECTING CODING 134 0 Kasami et al. [KKTFL] There exists an optimal codeword ZOpt such that de(Gpt, < c Irzl, (7.1 1) &TK whereTK = { i : sgn(zi) # sgn(ri), i E U } . Good references to GMD decoding, its extensions, and combinations with Chase algorithms are [KNIH],[Kam],[TKK],[FL4] and [TLFL]. 7.7 Listdecoding List decoding was introduced by Elias andWozencraft (see [Eli3]). Most recently, list decoding of polynomial codes has received considerable attention, mainly caused by the papers written by Sudan and colleagues [Sud, Gur] on decoding RS codes beyond their error correcting capabilities. The techniques used, referred to as Sudan algorithms,use interpolation and factorization of bi-variate polynomials over extension fields. Sudan algorithms can be considered extensions of the Welch-Berlekamp algorithm [Bed]. These techniques have been applied to soft-decision decoding of RS codes in [KV]. 7.8 Soft-output algorithms The previous sections of this chapter have been devoted to decoding algorithms that output the most likely coded sequence or codeword (or list of codewords). However, since the appearance of the revolutionary paper on turbo codes in 1993 [BGT], there is a need for decoding algorithms that output not only the most likely codeword (or list of codewords), but also an estimate of the bit reliabilitiesfor further processing. In the field of error correcting codes, soft-output algorithms were introduced as early as 1962, when Gallager [Gal] published his work on low-density parity-check (LDPC) codes3, and later by Bahl et al. [BCJR]. In both cases, the algorithms perform a forward-backward recursion to compute the reliabilities of the code symbols. In the next section, basic soft-output decoding algorithms are described. Programs to simulate these decoding algorithms can be found on the ECC web site. In the following sections, and for simplicity of exposition, it is assumed that a linear block code, constructed by terminating a binary memory-m rate-l/n convolutional code, is employed for binary transmission over an AWGN channel. It is also assumed that the convolutional encoder starts at the all-zero state Sf)and, after N trellis stages, ends at the all-zero state S:"' 7.8.I Soft-output Viterbi algorithm In 1989, the Viterbi algorithm wasmodified to output bit reliability information [HH]. The soft-output viterbi algorithm (SOVA) computes the reliability, or soft-output, of the information bits as a log-likelihood ratio(LLR), (7.12) LDPC codes are coveredin Chapter 8. SOFT-DECISION DECODING 135 where F denotes the received sequence. The operation of a SOVA decoder can be divided into two parts. In the first part, decoding proceeds as with the conventional VA, selecting the most likely coded sequence, 6, in correspondence with the path in the trellis with the maximum (correlation) metric, at stage n. (See Section 5.4.) In addition, the path metrics need to be stored at each decoding stage, and for each state. These metrics are needed in the last part of the algorithm, to compute the soj? outputs. In the second part of SOVAdecoding, the Viterbi algorithm transverses the trellis backwards, and computes metrics and paths, starting at i = N and ending at i = 0. It should be noted that, in this stage of the SOVA algorithm, there is no need to store the surviving paths, but only the metrics for each trellis state. Finally for each trellis stage i , 1 5 i 5 N , the soft outputs are computed. Let M,, denote the (correlation) metric of the most likely sequence 6 found by the Viterbi algorithm. The probability of the associated information sequence ii given the received sequence, or a-posteriori probability(APP), is proportional to Mmax,since Pr{ClF} = Pr{GlF} - eMm*x. (7.13) Without loss of generality, the APP of information bit ui can be written as Pr{ui = I ~ F } eM;(’); A where Mi(1) = Alma. Let Mi(0) denote the maximum metric of paths associated with the complement of information symbol zli. Then it is easy to show that Therefore, at time i , the soft output can be obtained from the difference between the maximum metric of paths in the trellis with fii = l and the maximum metric of paths with fii = 0. In the soft-output stage of the SOVA algorithm, at stage i, the most likely information symbol ui = a, a E (0; l}, is determined and the corresponding maximum metric (found in the forward pass of the VA) set equal to Mi(ui). The path metric of the best competitor, Mi(ui @ l),can be computed as [VUC], (7.15) where ICl, IC2 E (0; 1:2;. . . 2m - l}, Mf(Sj??) is the path metric of the forward survivor at time i - 1and state S ( k 1 ) , - BM,!bl)(ui@ 1) is the branch metric at time i for the complement information associated with a transition from state E d k 1 ) to S ( k 2 )and , - M b ( S j k 2 )is ) the backward survivor path metric at time i and state S(”). - Finally, the soft output is computed as R ( u ~ ) Mi(1) - Mi(0). (7.16) Example 88 Let C be a zero-tail (12: 4) code obtained from a memory-2 rate- 1/2 convolutional code with generators (7; 5 ) . The basic structure of the trellis diagram of this 136 THE ART OF ERROR CORRECTING CODING Transmitted -I:l +I;I +l;l +1,+1 -l,+l -1;l Received 4 - 1 - 1 .-3 +2:3 +3,+3 -3,+3 +3,+1 Figure 77 Trellis diagram used in SOVA decoding for Example 88. code is the same as in Example 86. Suppose that the information sequence (including the tail bits) is .21 = (110100), and that the received sequence, after binary transmission over an AWGN channel, is = (-4, -1, -1, -3: +2, -3, +3, +3, -3, +3, -3, +l). Figure 77 shows the trellis diagram for this code. For i = 0, l, . . . , 6, each state has a label on top of it of the form Mf(S,'"') Mb(S{"') ' The branches are labeled with the branch metrics S M i . The soft outputs A(ui), for i = 1,2 , . . . ,6 are -34, -16, +22, -16, +24, +24, respectively. Implementation issues In a SOVA decoder, the Viterbi algorithm needs to beexecuted twice. The forward processing is just as in conventional VD, with the exception that path metrics at each decoding stage need to be stored. The backward processing uses the VA, but does not need to store the surviving paths, only the metrics at each decoding state. Note that both backward processing and soft-decision computation can be done simultaneously. The backward processing stage does not need to store path survivors. In addition, the soft outputs need to becomputed, after both the forward and backward recursions finish. Particular attention should be paid to the normalization of metrics at each decoding stage, in both directions. Other implementation issues are the same as in a Viterbi decoder, discussed in Sections 5.4.3 and 5.5.1. The SOVA decoder can also be implemented as a sliding window decoder, likethe conventional Viterbi decoder. By increasing the computation time, the decoder operates continuously, not on a block-by-block basis, without forcing the state of the encoder to return to the all-zero state periodically. The idea is the same as that used in the VD with traceback memory, as discussed in Section 5.4.3, where forward recursion, traceback and SOFT-DECISION 137 backward recursion, and soft-output computations are implemented in several memory blocks (see also [ViU]). 7.8.2 Maximum-a-posteriori (MAP) algorithm The BCJR algorithm [BCJR] is an optimal symbol-by-symbol MAP decoding method for linear block codes that minimizes the probability of a symbol error. The goal of this MAP decoder is to examine the received sequence F and to compute the a-posteriori probabilities of the input information bits, as in (7.12). The MAP algorithm is described next, following closely the arguments in [BCJR]. The state transitions (or branches) in the trellis have probabilities and for the output symbols Vi, where z = f l , and zi= m(vi)= (-l)"%, 0 i 5 N. The sequence P is transmitted over an AWGN channel and received as a sequence F , with transition probabilities N n-l N (7.19) where p(ri,jIzi,j) is given by (7.1). Let B!') be theset of branches connecting state S!:;) to state S/m)such that theassociated information bit ui = j , with j E (0, l}.Then P r {ui= j l ~ =} c P r {Si?;', S!.'),?} (rn',rn)€Bjj) e c c~i(m',m). (7.20) (m',m)&) The value of oi(m',m) in (7.20) is equal to cJz(m',m ) = ai-l(m'). # ) ( m ' , m) . /?i(rn), (7.21) A where the joint probability ai(m)= Pr {S!"', F P } is given recursively by (7.22) and is referred to as the forward metric. A The conditional probability ~ j ' ) ( m m ' , ) = Pr { si"),FlSi?;)} is given by THE ART OF CORRECTING ERROR CODING 138 { where pi(mlm’) = P r Sirn)1Si~~)}, which for the AWGN channel can be put in the form where 15ij(m, m’) = l if {m‘,m} E B:’), and &(m,m‘) = 0 otherwise. #)(m’, m)is referred to as the brunch metric. The conditional probability pi(m) 2 P r F f IS!m)} is given by { (7.25) m’ j=O and referred to as the backward metric. Combining (7.25), (7.24), (7.22) , (7.21) and (7.12), the soft output (LLR) of information bit ui is given by with the hard-decision output is given by Qi = sgn(h(ui))and the reliability of the bit ui is IA(ui)l. The above equations can be interpreted as follows. A bi-directional Viterbilike algorithm can be applied, just as in SOVA decoding (previous section). In the forward recursion, given the probability of a state transition at time i , the joint probability of the received sequence up to time i and the state at time i is evaluated. In the backward recursion, the probability of the received sequence from time i 1to time N , given the state at time i is computed. Then the soft output depends on the joint probability of the state transition and the received symbol at time i. The MAP algorithm can be summarized as follows. 0 Initialization For m = 0 , 1 , . . . ,2” - 1, + ao(0) = 1, ~ o ( m=) 0 , m # 0, Form’ = 0 , 1 , . . . , 2 m - l, P N ( 0 ) = 1, 0 0 PlV(m’) = 0,m’ # 0, Forward recursion Fori = 1 , 2 , . . . , N , 1. For j = 0 , l, compute and store the branch metrics 7:’’ (m‘,m ) as in (7.24). 2. For m = 0,1, ...,2m - l, compute and store the forward metrics q ( m ) as in (7.22). Backward recursion Fori=N-l,N-2;..,0, 1. Compute the backward metrics Pi(m) asin (7.25), usingthebranchmetric computed in the forward recursion. 2. Compute the log-likelihood ratio A(ui) as in (7.26). SOFT-DECISION DECODING 139 Implementation issues The implementation of a MAPdecoder is similar to that of a SOVA decoder, as both decoders perform forward and backward recursions. All the issues mentioned in the previous section apply to a MAP decoder. In addition, note that branch metrics depend on the noise power density NO,which should be estimated to keep optimality. To avoid numerical instabilities, the probabilities ai(m) and /3i(m)need to be scaled at every decoding stage, such that (.dm) = Pi(m) = 1. cm cm 7.8.3 Log-MAPalgorithm To reduce the computational complexity of the MAP algorithm, the logarithms of the metrics may be used. This results in the so-called log-MAP algorithm. From (7.22) and (7.25) [RVH], m’ j = O 1 m’ j = O Taking the logarithm of ~ y ) ( mm ‘ :) in (7.24), By defining &(m) = logai(m),pi(m) = logPi(m) and Ty)(m‘:m)= logyi( A (m’:m), (7.26) can be written as and an algorithm that works in the log-domain is obtained. The following expression, known as the Jacobian logarithm [RVH], is used to avoid the sum of exponential terms, + The function log (1 e-161-621) can be stored in a small look-up table (LUT), as only a few values (eight reported in [RVH]) are required to achieve practically the same performance as the MAP algorithm. Therefore, instead of several calls to slow (or hardware expensive) exp(z) functions, simple LUT accesses give practically the same result. I40 THE ART OF CORRECTING ERROR CODING 7.8.4 Max-Log-MAPalgorithm A more computationally efficient, albeit suboptimal derivative of the MAP algorithm is the Max-Log-MAP algorithm. It is obtained as before, by taking the logarithms of the MAP metrics, and using the approximation [RVH] log (e61 + 2 2 ) x max (dl 62); (7.3 1) which is equal to the first term on the right-hand side of (7.30).As a result, the log-likelihood ratio of information bit zli is given by The forward and backward computations can now be expressed as For binary codes based on rate-lln convolutional encoders, in terms of decoding complexity (measured in number of additions and multiplications), the SOVA algorithm requires the least amount, about half of that of the max-log-MAP algorithm. The log-MAP algorithm is approximately twice more complex compared to the max-log-MAP algorithm. In terms of performance, it has been shown [FBLH] that the max-log-MAP algorithm is equivalent to a modified SOVA algorithm. The log-MAP and MAP algorithms have the same best error performance. 7.8.5 Sofr-output OSD algorithm The OSD algorithm of Section 7.5 can be modified to output the symbol reliabilities [%S]. This modification is referred to as the soft-output OSD, or SO-OSD. The SO-OSD algorithm is a two-stage order-i re-processing. The first stage is the same as conventional OSD, determining the most likely codeword 2.”~up to order-i re-processing. To describe the second stage, the following definitions are required. For each most reliable position j , 1 5 j 5 K, define the codeword f i ~ ~is(obtained j) by complementing position j in B M L , f i ~ ~=(BML j ) CB E(j); where E ( j ) is the set of all length-K vectors of Hamming weight one. The vector E ( j ) is the coset representative of the partition of code C1 (equivalent to the original code after reordering, as in OSD, see Section 7.5) into two sets of codewords, having the j-th position equal to zero or one in codewords of C. These sets, after removing position j , become punctured subcodes of Cl, and are denoted C(0)and C(1).Let C(j) = C(0). The SO-OSD algorithm consists of the following steps: 1. Determine the most likely codeword BMI, covered by order-i re-processing. SOFT-DECISION DECODING 141 2. For each subcodeC ( j ) ,1 5 j 5 K , (a) Initialize all soft output values of the least reliable positions (LW) based on U M L and U M L ( j ) .That is, compute where rn(vi) = (U ( j ) E C ( j ) covered by order4 re(b) Determine themostlikelycodeword processing. (c) Evaluate Lj (7.34) based on GML and f j ( j ) . (d) Update soft output values of L W in the positionsof GML @ @ ( j with ) Lj. 3. For K 1 5 j 5 N , choose the smallest output value associated with each LRPj. + The performanceof SO-OSD is the same as max-log-MAP decodingfor many binary linear block codes of length up to 128 and high-rate codes [FL5]. In addition, scaling down the extrinsic values (7.34) improves the performance by a few tenths of a dB. The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) Iteratively decodable codes Iterative decoding may bedefined as atechnique employing a soft-output decoding algorithm that is iterated several times to improve the error performance of a coding scheme, with the aim of approaching true maximum-likelihood decoding (MLD), with less complexity. When the underlying error correcting code is well designed, increasing the number of iterations results in an improvement of the error performance. Iterative decoding techniques date back to 1954 with the work of Elias [Elil] on iterated codes. Later, in the 1960s, Gallager [Gal] and Massey [Masl] made important contributions. Iterative decoding was then referred to as probabilistic decoding.The main concept was then, as it is today, to maximize the a-posteriori probability of a symbol being sent given a noisy version of the coded sequence. In this chapter, code constructions that can beiteratively decoded are presented. Elsewhere in the literature, these coding schemes have been named turbo-like codes. In terms of the application of iterative decoding algorithms, and for the purpose of this chapter, ECC schemes can be broadly classified into two classes: Product codes Examples of codes in this class include parallel concatenated codes, or turbo codes, and serially concatenated codes. Members of this class are also block product codes, presented in Section 6.2.3. Low-density parity-check(LDPC) codes These are linear codes with the property that their parity-check matrix has a small ratio of number of nonzero elements to the total number of elements. In both of the above classes of codes, the component codes can be either convolutional or block codes, with systematic or nonsystematic encoding, or any combination thereof. To provide a motivation for the study of iterative decoding techniques, the fundamental structure of turbo codes is discussed next. Turbo codes Over the past eight years, there has been an enormous amount of research effort dedicated to the analysis of iterative decoding algorithms and the construction of iteratively decodable codes or “turbo-like codes” that approach the Shannon limit’. Turbo codes were introduced in The Shannon limit refers to the capacityof a discrete-input continuous output channel THE ART OF CORRECTING ERROR CODING 144 1993 [BGT]. The results reported there sent a shock wave throughout the researchcommunity. With binary transmission over an AWGN channel, a rate-1/2 coding scheme, based on a combination of two rate-1/2 16-state RSC codes, connected with an interleaver of size 256 x 256, achieved BER rates of at an signal-to-noise (SNR) ratio per bit, or &,/NO, of 0.7 dB, closer than ever before to the Shannon limit (0.2 dB). See Figure 78. Figure 78 Performance of a rate-1/2 parallel concatenated (turbo) code with memory-4 rate-1/2 RSC codes, 90 = 21 and 91 = 37. Block interleaver size 256 x 256 = 65536. From [BG2]. Reproduced by permission of IEEE. ( 0 1 9 9 6 IEEE) The basic elements in the turbo coding scheme proposed in [BGT] are the following: Coding structure A product code (referred to in the original paper [BGT] as parallel concatenated code) structure, with constituent recursive systematic convolutional encoders. ITERATIVELY DECODABLECODES 145 Reliability-based Employ sop-input sop-output(SZSO) maximum-a-posteriori (MAP) decoders for the component codes, in order to generate log-likelihood ratios. Iterative decoding The use of feedback of part of the symbol reliabilities, in the form of extrinsic information, from an outer (column) decoder to an inner (row) decoder, and vice versa. Random permutation A long random interleaver applied between the two encoders. Its main function is to ensure that, at each iteration, the component MAP decoders get independent estimates on the information symbols. These four elements have been extensively studied over the years. [VUC,ISTC1, ISTC2, JSAC l, JSAC2, Hew] are a good sample of the research efforts. 8.1 Iterativedecoding The purpose of this section is to introduce basic concepts behind iterative decoding. Consider the a-posteriori log-likelihood ratio (LLR)2 of an information symbol, Equation (7.12) on page 134, repeated here for convenience, Let Ci(0)and Ci(1) denote the set of modulated coded sequences 3 = m(U) E m ( C ) such that the i-th position in U E C is equal to zero and one, respectively. Recall that modulation was a one-to-one and onto mapping from v E (0,l}to z E {+l,-l}, as in Equation (7.2). Then the symbol LLR (8.1) can be written as, from which it follows that the LLR of an information symbol can be expressed as The LLR is also referred to as "L-value" by some authors. THE ART OF ERROR CORRECTING CODING 146 A(w) = + A a ( ~ i +) (8.3) where Aci is known as the channel LLRand given by ( log (5) , fora BSC channel with parameter p ; for anAWGN channel (0: = N 0 / 2 ) ; and 4 (8.4) for aflat Rayleigh fading channel, the quantity P r { q = -l}) - log (Pr{ui = l}) Pr{ui = 0} Pr{zi = +l} is the a-priori LLR of the information symbol, and Aa(ui) = log f N is the extrinsic LLR, which is specified by the constraints imposed by the code on the other information symbols. Assume that binary transmission is performed over an AWGN channel and that binaryratel / n RSC encoders are used as components. In an iterative decoding procedure, the extrinsic information provided by Ai,,(C) can be fed back to the decoder as a-priori probability for a second round of decoding. In terms of forward-backward decoding using a trellis (see Section 7.8.2), the extrinsic LLR can be written as where ai(m)and Pi(rn) are given by (7.22) and (7.25), respectively. A modified branch metricis needed to compute the extrinsic LLR, where, as before, 147 ITERATIVELY DECODABLE CODES This extrinsic LLR, for an information position i, does not contain any variabledirectly related to ui,for i = 1;2; . . . N . Because it is assumed that encoding is systematic, notice that the branch labels in the trellis are of the form (ui; v ~ . . .; q n - l ) , and therefore the sum in the modified branch metric(8.8) starts at q = l.3 In the more general caseof a two-dimensional product coding scheme, the first (e.g., row) decoder produces h{:i(C)which is given to the second (e.g., column) decoder as a-priori probability Ri2)(ui)to be used in the computation of the LLR of information symbol ui.In other words, the extrinsic information provides a soft output that involves only soft inputs (reliabilities) that are not directly related to the information symbolui.Iterative decoding of product codesis the topicof the next section. 8.2 Productcodes In this section, coding schemes based on products of codes with interleaving are presented. These schemes allow the use of simple decoders for the component codes. With iterative decoding, symbol-by-symbolMAP decoders of the component codescan exchange extrinsic LLR values, generally improving the reliability of the estimatedinformation bits,with increasing number of iterations. 8.2.1 Parallelconcatenation:turbocodes Figure 79 shows the block diagram of an encoder of a parallel concatenated code, better known as a turbo code. Two encoders are used, each delivering as output only redundant symbols. For instance, if rate-1/2 RSC codes areused, the encoders represent theterm g1 ( D ) / g o ( D )in the polynomial generator matrix G(D). The inputs to the encoders are U and I I G , where II denotes a permutation matrix4 associated with the interleaver. The output corresponding to an input symbol ui is ( ui vp1,i vp2,i), with wp1,iand vp2,i representing the redundant symbols,for i = 1;2: . . . N , where N is the block length. Figure 79 Encoder structure of a parallel concatenated (turbo) code. Compare this with the expression of the branch metric y p ) ( m ’ ,m) in (7.24). A permutation matrix is a binary matrix with only one nonzero entry per row and per column, if ~ then a symbol in position i at the input is sent to position j at the output. i = ,1 ~ THE ART OF ERROR CORRECTING CODING 148 The original turbo code scheme used RSC encoders. However, to examine this coding scheme, assume that the component codes are two binary systematic linear block (ni;k i ) codes, i = 1;2. The rate of the overall code is (see Figure 80) A where Ri = ki/ni, i = 1;2. Let Gi = ( Iil Pi ) denote the generator matrix of code Ci, i = 1;2. Then the generator matrix of the parallel concatenated code Cpc can be put in the following form, GPC = ( ( Ik1kz p1 p1 .*. p l ) II ( p2 p2 .,. p 2 ) ) where II is the permutation matrix associated with the interleaver, and Pi is theparity submatrix of code Ci, i = 1;2. The number of times that P1 appears in the middle part Pi of Gpc is k z , while the number of times that P2 appears in theleftmost portion Pi of GPC is kl . All other entries in Pi and Pi are zero. It follows that codewords of Cpc are of the form ( a I GP; I GP; ). Example 89 Let C1 be a binary repetition ( 2 , l ;2) code and C2 be an binary SPC (3; 2; 2) code. Then klk2 = 2. There is only one possible permutation matrix II in this case. The parity submatrices and permutation matrix are respectively.From (8.10) it follows that the generator matrix of the parallel concatenated (5; 2) code Cpc is Note that the minimum distance is dpc = 3, which is less than that of the product of C1 and (72. One way to interpret a parallel concatenated (turbo) code, from the point of view of linear block codes, is as a punctured product code, in which the redundant symbols corresponding with the checks-on-checks are deleted. This interpretation is shown in Figure 80. The only essential difference between a turbo code and a block product code, is that the interleaver is not simply a row-by-row column-by-column interleaver, but one that introduces sufficient ITERATIVELY DECODABLECODES 149 "1 - symbols n2 -k, d 3 - Figure 80 A parallel concatenated code interpreted as a punctured product code. disorder in the information sequence, so that iterative decoding works. When interpreted as a linear block code, a turbo code can be further interpreted as a generalized concatenated code5, after replacing the puncturing operation with an equivalent multiplication by a binary matrix and a direct-sum. Let MI be a 1 x N matrix with nlkz successive ones followed by N - nlkz zeros. Let M2 be another 1 x N matrix with N - (nz - k2)kl zeros followed by (nz - kz)kl ones. Then a turbo encoder can be visualized as an encoder of a GC code, as shown in Figure 8 1. Figure 81 A turbo encoder as an encoder of a generalized concatenated code. Iterative decodingof parallel concatenated codes The basic structure of an iterative decoder for a parallel concatenated coding scheme with two component codes is shown in Figure 82. Each iteration consists of two phases, one decoding phase per component decoder, as follows. First phase In the first decoding iteration, first phase, the soft-in soft-out (SISO) decoder for the first component code computes the a-posteriori LLR (8.7) assuming equally likely symbols, i.e., A,(u) = 0. This decoder computes the extrinsic information for each information symbol, Ael (G), based on the part of the received sequence that corresponds to the parity symbols, P p l , and sends the result to the second SISO decoder. See Section 6.2.5 THE ART OF ERROR CORRECTING CODING 150 Figure 82 Block diagram of an iterative decoder for a parallel concatenated code. Second phase In the second phase offirst decoding iteration, the permuted (or interleaved) extrinsic information from the first decoder is used as a-priori LLR, A,(C) = HA,1 (U).Extrinsic information he2 (G) is computed, based on the part of the received sequence that corresponds to the parity symbols of the second component code, F p 2 , thus terminating the first decoding iteration. At this point, a decision can be made on an information symbol, based on its aposteriori LLR h ( ~ ) . In subsequent iterations, the firstdecoder uses the deinterleaved extrinsic information from the second decoder, DI-lhe2(C), as a-priori LLR for the computation of the soft output (the a-posteriori LLR), A(U). This procedure can be repeated until either a stopping criterion is met [HOP, SLF], or a maximum number of iterations is performed. Note that making decisions on the information symbols after the first (row) decoder saves one deinterleaver. Asin Section 7.8.4, the iterative MAP decoding algorithm described above can be simplified by using the approximation of the “log” by the “max” as in (7.3 1).Also, there is an iterative version of the SOVA algorithm presented in Section 7.8.1, that essentially computes the a-posteriori LLR A(C) directly and then, subtracting the channel LLR A , and theextrinsic LLR Aej(C) from the other decoder, produces the extrinsic LLR [FV]. Example 90 Consider the turbo ( 5 : 2: 3) code of Example 89. A codeword is arranged in a 3 x 2 array with the lower right comer deleted. Suppose that the following codeword V m U1 U2 W21 =0 =l =1 W11 2112 =0 =1 (where “ X ’ denotes that the position is deleted or ignored), is (BPSK) modulatedand transmitted over an AWGN channel and received as (this is the channel LLR A c ): For the computations below, we use the fact that, for two binary variables x1 and x2, the LLR of their sum x1 x2 is [HOP] + ITERATIVELY DECODABLECODES 151 First iteration In the first decoding phase, the row decoder for the repetition code, computes Ael(u1) = A(rl1) = f 2 . 5 , Ae1(u2) A(r12) = +1.0. In the second decoding phase, the extrinsic information from the row decoder is used as apriori LLR for the column decoder for the parity-check code. The updated LLRs are E l In the second phase of the first iteration, the column decoder computes the extrinsic LLRs, At the end of the first decoding iteration, the a-posteriori LLR are A(u1) h(u2) = Ac1 = Acz + Ael(u1) + Aea(u1) = +1.58 + Ael(u2) + A e z ( ~ 2 )= -2.13 The iterative decoder could make a decision at this point, based on the sign of the a-posteriori LLR, and deliver the (correct) estimates Q1 = 0 and Q 2 = 1. Increasing the number of iterations improves the soft outputs associated with the information bits. Second iteration Row decoder: Ael(u1) Ael(u2) = A(r11) = A(rl2) + + A e 2 ( ~ 1 )= A e 2 ( ~ 2 )= +2.5 - 0.42 = +1.08, +1.0 0.38 = +1.38. + Column decoder: A-posteriori LLR: = Acl A(u~) = ACZ A(u1) + Ael(u1) + AeZ(u1) = +0.41, + Ael(u2) + Ae2(~2)= -1.96. After the second iteration, based on the signs of the LLRs, the decoder delivers the same (correct) hard decisions. Moreover, note that the soft outputs changed because of the influence of the extrinsic LLR computed in the first iteration. In particular, the reliability of the first bit (the amplitude of the LLR) decreases from lA(ul)l = +1.58 to IA(u1)l = +0.41. 152 THE ART OF ERROR CORRECTING CODING Error performance With reference to Figure 78, page 144, it is clear that, as the number of iterations grows, the error performance improves. Typical of turbo coding schemes is the fact that increasing the number of iterations results in a monotonically decreasing improvement in coding gain. From Figure 78, increasing the number of iterations from 2 to 6 gives an improvement in SNR of 1.7 dB, whereas going from 6 to 18 iterations yields only a 0.3 dB improvement in coding gain. Since the appearance of turbo codes, advances have taken place in understanding the BER behavior of turbo codes. At the time of this writing, there appears to be a consensus among researchers on the reasons why turbo codes offer such an excellent error performance: 0 0 0 Turbo codes have a weight distribution that approaches, for long interleavers, that of random codes. Recursive convolutional encoders and proper interleaving map most of the lowweight information sequences into higher weight coded sequences. Systematic encoders allow the effective use of iterative techniques utilizing MAP decoders. Information symbol estimates are available directly from the channel. In addition, most recently (end of 2001), a number of interesting semi-analytical tools have appeared, based on density evolution [RU], Gaussian approximation [EH] and mutual information [tBr2] or SNR transfer [DDP] characteristics, to study the convergence properties of iterative decoding algorithms. The art of interleaving A critical component in achieving good performance with iterative decoding of a turbo code is the interleaver. In turbo codes, the interleaver serves three main purposes: (1) Build very long codes with weight distributions that approach those of random codes; and (2) help in the iterative decoding process by decorrelating the input LLRs to the SISO decoders as much as possible; and (3) proper termination of the trellis in a known state, after the transmission of short to medium length frames, to avoid edge effects that increase the multiplicity of low weight paths in the trellises of the component codes. To emphasize, the specific type of interleaver becomes an important factor to consider as the frame lengths (or interleaver lengths) become relatively small, say, up to one thousand symbols. There is already a wealth of publications devoted to interleaver design for turbo codes. In this section, a brief description of the basic interleaver types and pointers to the literature are given. In 1970, several types of optimum interleavers were introduced [Ram]. In particular, an ( n 1 , n 2 ) interleaver wasdefined there as a device that “reorders a sequence so that no contiguous sequenceof 722 symbols in the reordered sequence contains any symbols that were separated by fewer than n 1 symbols in the original ordering.” Let . . ,a t l , at,, at3,. . . denote the output sequence from an ( n 1 , n 2 ) interleaver, where l,, e,, . . . are the positions of these symbols in the input sequence. Then the definition in the previous paragraph translates into the following condition: whenever li - j l < 712. ITERATIVELY DECODABLE CODES 153 It can be shown thatthe deinterleaver of an (nl ,n 2 ) interleaver is itself an (nl:122) interleaver. The delay of the interleaver and deinterleaver are both equal to the delay introduced by the overall interleaving-deinterleaving operation. Another important parameter of an interleaver is the amount of storage or memory required. Ramsey showedfour types of (nl n2) interleavers that are optimum in the sense of minimizing the delay and memory required to implement them. These interleavers are known as Ramsey interleavers. At about the same time, Forney [For51 proposed an interleaver with the same basic structure of an (n1:n2) Ramsey interleaver, known as a convolutional interleaver. Convolutional interleavers were discussed in Section 6.2.3 and have been applied to the design of good turbo coding schemes [Haw]. There are several novel approaches to the analysis and design of interleavers: One is based on a random interleaver with a spreading property. Such a structure was first proposed in [DP], together with a simple algorithm to construct S-random interleavers: Generate random integers in the range [l:NI, and use a constraint on the interleaving distance. This constraint is seen to be equivalent to the definition of a Ramsey (S2:S 1 1)interleaver (this is noted in [VUC],pp. 211-213). Then additional constraints (e.g., based on the empirical correlation between successive extrinsic LLR values) are imposed to direct the selection of the positions of the permuted symbols at the output of the interleaver [HEM, SSSN]. A second approach to the design of interleavers is to consider the overall turbo encoder structure and to compute its minimum distance and error coefficient (the number of coded sequences at minimum distance) [GPB, BH]. This gives an accurate estimation of the error floor in the medium to high SNR region. Other important contributions to the design of short interleavers for turbo codes are [TC, BP]. + 8.2.2 Serial concatenation Serial concatenation of codes was introduced in [BDMPl]. Ablock diagram of an encoder of a serial concatenation of two linear codes is shown in Figure 83. Based on the results from Section 6.2.3, and in particular comparing Figure 83 with Figure 55, the serial concatenation of two linear codes is easily recognized as a product code. Note that, as opposed to a turbo code, in a serially concatenated coding system there is no puncturing of redundant symbols. - - U D Encoder C , - nil v1 * r I * Encoder C v2 Figure 83 Block diagram of the encoder of a serially concatenated code. The encoder of a serial concatenation of codes has the same structure as that of a product code. Following closely the notation in the original paper [BDMPl], the outer (p: k , d l ) code Cl has a rate R1 = k / p and the inner (n,p,d 2 ) code C2 has a rate R2 = p / n . The codes are connected in the same manner as a block product code, with a block interleaver of length L = m p . This is achieved, as before, by writing m codewords of length p into the interleaver, and reading in a different order according to the permutation matrix II. The sequence of L bits at the output of the interleaver is sent in blocks of p bits to the outer encoder. The rate of the overall ( N :K ; d l d 2 ) code CSCis Rsc = k / n , where N = nm and K = k m . The generator matrix of CSCcan be expressed as the product of the generator matrix of Cl, the kz X nl permutation matrix II of the interleaver, and the generator matrix of C,, as 154 THE ART OF CORRECTING ERROR CODING follows where Gi is the generator matrix of code Ci, i = l;2. The number of times that G1 appears in the first factor G: of Gsc in (8.12) is IC?, and the number of times that G2 appears in the third factor G; of Gsc is n1. All other entries in G: and G; are zero. Example 91 Let C1 and C2 be the same codes as in Example 89, that is, binary repetition (2: l;2) and SPC (3; 2; 2) codes, respectively. Then the serial concatenation, or product, of C1 and C2, CSCis a binary linear block (6; 2 , 4 ) code. Note that the minimum distance of CSCis larger than that of Cpc in Example 89. The generator matrices are G1 = ( 1 1) and G2 = :) ( . Assumingthataconventionalproduct code is employed, thepermutation matrix, associated with a row-by-row and column-by-column interleaver, is Therefore, the generator matrix of CSCis (:: ; :)( 1 0 0 0 Gsc = ( (h121) (1/)121)) 0 0 0 1 (1 0 (0 l = ( 1) ( l 0 1) 1) ( 0 1 1 ) ( 1) O ol, 023 (1 0 l ) ) 0 1 1 1 where O i j denotes the i x j all-zero matrix. The result can be verified by noticing that the last equality contains the generator matrix of the the SPC (3; 2; 2) code mo times, because of the repetition (2; 1;2 ) code. It is also interesting to note that this is the smallest member of the family of repeat-and-accumulate codes [DJM]. ITERATIVELY DECODABLE CODES 155 Note that the main difference between the serial concatenated coding scheme and product coding discussed in Section 6.2.3, is that there the interleaver was either a row-by-row columnby-column interleaver or acyclic interleaver (if the component codes were cyclic). In contrast, as with turbo codes, the good performance of serial concatenation schemes generally depends on an interleaver that is chosen as “random” as possible. Contrary to turbo codes, serially concatenated codes do not exhibit “interleaver gain saturation” (i.e., there is no error floor). Using a random argument for interleavers of length N , it can be shown that the error probability for a product code contains a factor N-L(dof+1)/21, where dof denotes the minimum distance of the outer code, as opposed to a factor N - l for parallel concatenated codes [BDMPl]. As a result, product codes outperform turbo codes in the SNR region where the error floor appears. At low SNRvalues, however, the better weight distribution properties of turbo codes [PSC] leads to better performance than product codes. The following design rules were derived for the selection of component codes in a serially concatenated coding scheme6 for component convolutional codes: 0 0 0 The inner code must be an RSC code. The outer code should have large and, if possible, odd value of minimum distance. The outer code may be a nonrecursive (FIR) nonsystematic convolutional encoder. The last design criterion is needed in order to minimize the number of codewords of minimum weight (also known as the errorexponent) and the weight input sequences resulting in minimum weight codewords. Iterative decodingof serially Concatenated codes With reference to Figure 84, note that if the outer code is a nonsystematic convolutional, then it is not possible to obtain the extrinsic information from the SISO decoder [BDMPI]. Therefore, different from the iterative decoding algorithm for turbo codes, in which only the LLR of the information symbols are updated, here the LLR of both information and code symbols are updated. The operation of the SISO decoder for the inner code remains unchanged. However, for the outer SISO decoder, the a-priori LLR is always set to zero, and the LLR of both information and parity symbols is computed and delivered, after interleaving, to the SISO decoder for the inner code as a-priori LLR for the next iteration. As withiterative decoding of turbo codes, there is a max-log-MAP based iterative decoding algorithm, as well as a version of SOVA that can be modified to become an approximated MAP decoding algorithm for iterative decoding of product codes [FV]. 8.2.3 Block productcodes Although turbo codes and serial concatenations of RSC codes seem to have dominated the landscape of coding schemes where iterative decoding algorithms are applied, block product codes may also be used, as is evident from the foregoing discussion. In 1993, at the same conference where Berrou and colleagues introduced turbo codes, a paper was presented on iterative decoding of product and concatenated codes [LYHH]. In particular, a threeIt should be noted that these criteria were obtained based on union bounds on the probability of a bit error. THE ART OF ERROR CORRECTING CODING 156 n .... n d... .. . .. I * SISO 2 (ROW) (COLUMN) 4 '('I n-1F h -U Figure 84 Block diagram of an iterative decoder for a serially concatenated code. dimensional product (4096,1331,64) code, based on the extended Hamming (16,11,4) code, with iterative MAP decoding was considered and shown toachieve impressive performance. One year later, near-optimum turbo-like decoding of product codes was introduced in [PGPJ] (see also [Pyn]). There the product of linear block codes of relatively high rate, single- and double-error correcting extended BCH codes, was considered. An iterative decoding scheme was proposed where the component decoders use the Chasetype-I1 algorithm7. After a list of candidate codewords is found, LLR values are computed. This iterative decoding algorithm and its improvements are described in the next section. Iterative decoding usingChase algorithm In [PGPJ, Pyn], the Chase type-I1 decoding algorithm is employed to generate a list of candidate codewords which are close to the received word. Extrinsic LLRvalues are computed based on the best two candidate codewords. If only one codeword is found, then an approximated LLR value is output by the decoder. Let C be a binary linear ( N ,K , d ) block code, capable of correcting any combination o f t = [ ( d - l)/2] or less random bit errors. Let F = ( T I , r2, . . . ,T N ) be the received word from the output of the channel, ~i = ( -l)wi wi, v E C, where wi is a zero-mean Gaussian random variable with variance N 0 / 2 . Chase type-I1 algorithm is executed based on the received word F , as described on page 130. Three possible events can happen at the end of the Chase type-I1 algorithm: + 1. Two or more codewords, (61,. . . , e l } , l 2 2, are found; 2. One codeword 61 is found; or 3. No codeword is found. In the last event, the decoder may raise an uncorrectable error flag and output the received sequence as is. Alternatively, the number of error patterns to be tested can beincreased until a codeword is found, as suggested in [Pyn]. Let X j ( l ) denote the set of modulated codewords of C, found by Chase algorithm, for which the j-th component xj = l , l E { -1, +l},for 1 5 j 5 N . Denote by Zj(l),yj(l)E X j ( l ) the closest andnext closest modulated codewords tothereceivedword F , in the Euclidean distance sense, respectively. By using the log-max approximation log(e" e b ) x max(a, b ) , the symbol LLR value (8.2) can be expressed as [Pyn, FL5] + (8.13) Chase algorithms are discussed in Section 7.4. ODABLEITERATIVELY CODES 157 from which, after normalization and redefining x, = x m ( + l ) and ,y output is A’(uj) = rj + c N c = y,(-l), the soft N rmxm = x j rmx,. (8.14) The term (8.15) is interpreted as a correction term to the soft-input r j , which depends on the two modulated codewords closest to F , and plays the same role as the extrinsic LLR.For 1 5 j 5 N , and each position j , the value w j is sent to the next decoder as extrinsic LLR, with a scaling factor Q,, so that ri = rj a C w j , (8.16) + is computed as the soft input at the next decoder. The factor ac is used to compensate for the difference in the variances of the Gaussian random variables ri and r i . A block diagram of the procedure for the generation of soft input values and extrinsic LLR values is shown in Figure 85. If for a given j-th position, 1 5 j 5 N , no pair of sequences Z j (+l)and yj (-1) can be found by Chase algorithm, in [Pyn] it has been suggested the use of the following symbol LLR, A’&) = P&, (8.17) where Pc is a correctionfacror to compensate for the approximation in the extrinsic information, and was estimated by simulations as Pc = 11% (Pr{wj Pr{wj correct} incorrect} (8.18) that is, the magnitude of the LLR of the simulated symbol error rate. In [PP, MT], it is shown how the correction factors a! and P can be computed aduptively based on the statistics of the processed codewords. It should also be notedthat the soft-output algorithm proposed in [FL5], and described in Section 7.5, can also be applied. Adaptive weights are also needed in this case, to scale down the extrinsic LLR values. Extrinsic values Channel LLR values W [I1 Soft-output Chase decoder i Delay Figure 85 Block diagram of a soft-output Chase decoder, W [I+1] 158 THE ART OF ERROR CORRECTING CODING Summarizing, the iterative decoding method with soft-outputs based on a set of codewords produced by a Chase type-I1 algorithm is as follows: Step 0: Initialization Set iteration counter I = 0. Let f[0] = f (the received channel values). Step 1: Soft inputs Forj = 1,2,...,N, 7-j [I + l] = rj [O] + Q, [I]wj[I]. Step 2: Chase algorithm Execute Chase type-I1 algorithm, using F[I + l].Let n, denote the number of codewords found. If possible, save the closest two modulated codewords, 52 and jj to the received sequence. Step 3: Extrinsic information Forj = 1,2,...,N, wj[I + 11 = xj c N T[I + lImxm, =m#ym m=l,m#j Step 4: Soft output + Let I = I 1. If I < Imax (the maximum number of iterations) or a stopping criterion is not satisfied then go to Step 1. Else compute the soft output, Forj = 1,2,...,N, and stop. For BPSK modulation, the values of Q, and pc were computed for up to four iterations (eight values in total, two values for the first and second decoders), as [Pyn] CL, = (0.0 0.2 0.3 0.5 p, = (0.2 0.4 0.7 0.9 1.0 1.0), 0.6 0.8 1.0 1.0 1.0 1.0). ITERATIVELY DECODABLE CODES 159 8.3 Low-densityparity-checkcodes In 1962, Gallager in [Gal] introduced a class of linear codes, known as low-density paritycheck (LDPC) codes, and presented two iterative probabilistic decoding algorithms. Later, Tanner [Tan] extended Gallager’s probabilistic decoding algorithm to the more general case where the parity-checks are defined by subcodes, instead of simple single parity-check equations. Earlier, it was shown that LDPC codes have a minimum distance that grows linearly with the code length and that errors up to the minimum distance could be corrected with a decoding algorithm with almost linear complexity [ZP]. In [MN, Mac] it is shown that LDPC codes can get as close to the Shannon limit as turbo codes. Later in [RSU], irregular LDPC codes were shown to outperform turbo codes of approximately the same length and rate, when the block length is large. At the time of writing, the best rate-1/2 binary code, with a block length of 10,000,000,is an LDPC code that achieved a record 0.0045 dB away from the Shannon limit for binary transmission over an AWGN channel [CFRU]. A regular LDPC code is a linear ( N ;K ) code with parity-check matrix H having the Hamming weight of the columns and rows of H is equal to J and K , respectively, with both J and K much smaller than the code length N . As a result, an LDPC code has a very sparse parity-check matrix. If the Hamming weights of the columns and rows of H are chosen in accordance to some nonuniform distribution, then irregular LDPC codes are obtained [RSU]. MacKay has proposed several methods to construct LDPC matrices by computer search [Mac]. 8.3.l Tanner graphs For every linear ( N ;K ) code C, there exists a bipartite graph with incidence matrix H . This graph is known as a Tanner graph [Tan, SS], named after its inventor. By introducing state nodes, Tanner graphs have been generalized to factor graphs [For7, KFL]. The nodes of the Tanner graph of a code are associated with two kinds of variables, and their LLR values. The Tanner graph of a linear ( N ;K ) code C has N variable nodes or code nodes, x!, associated with code symbols, and at least N - K parity nodes, z,, associated with the parity-check equations. For a regular LDPC code, the degrees of the code nodes are all equal to J and the degrees of the parity nodes equal to K . Figure 86 Tanner graph of a Hamming (7,4,3) code. Example 92 To illustrate the Tanner graph of a code, consider the Hamming (7: 4; 3) code. 1 60 THE ART OF ERROR CORRECTING CODING Its parity-check matrix iss H= ( l 1 1 0 1 0 0 0 1 1 1 0 1 0 l 1 0 1 0 0 1 1 . The corresponding Tanner graph is shown in Figure 86. The way code nodes connect to check nodes is dictated by the rows of the parity-check matrix. The firstrowgives the parity-check equation, u1 212 213 215 = 0. As indicated before, variables xe and z , are assigned to each code symbol andeach panty-check equation, respectively. Therefore, the following parity-check equations are obtained, + 21 22 23 + + = x1 + x 2 + x 3 + x 5 ; = x2 + x 3 + x 4 + x 6 ; = x1 x2 x4 27. + + + From the topmost equation, code nodes x1 xZix3 and x5 are connected to check node 21. Similarly, the columns of H , when interpreted as incidence vectors, indicate in which paritycheck equations code symbols appear, or participate in. The leftmost column of H above, ( 1 0 1)T, indicates that z1 is connected to check nodes 2 1 and 23. Example 93 The parity-check matrix in Gallager's paper [Gal], 11110000000000000000~ 00001111000000000000 00000000111100000000 00000000000011110000 00000000000000001111 H= 10001000100010000000 01000100010000001000 00100010000001000100 00010000001000100010 00000001000100010001 10000100000100000100 01000010001000010000 00100001000010000010 00010000100001001000 00001000010000100001. is that of an LDPC (20; 5) code with J = 3 and K = 4. Its Tanner graph is shown in Figure 87. Note that every code node is connected to exactly three parity-check nodes. In other words, the degree of the code nodes is equal to J = 3. Similarly, the degree of the parity nodes is equal to K = 4. Tanner graphs can be used to estimate codewords of an LDPC code C with iterative probabilistic decoding algorithms, based on either hard or soft decisions. In the following, the two basic iterative decoding algorithms introduced by Gallager are presented. See Example 13 on page 23. CODABLEITERATIVELY CODES 161 Figure 87 Tanner graph of Gallager's (20,5) code. 8.3.2 Iterative hard-decision decoding: The b i t f i p algorithm In his 1962 paper, Gallager gave the following algorithm [Gall9. The decoder computes allthe parity checks and then changes any digit that is contained in more than some fuced number of unsatis$ed parity-check equations. Using these new values, the parity-checks are recomputed, and the process is repeated until the parity checks are all satis$ed. The input of the algorithm is the vector Th = (sgn(rl), sgn(r-2); .. . sgn(rN)), where r 2' -- (-l)'i wi,V E C, wi denotes a zero-mean Gaussian random variable with variance No/2, i = 1;2:. . . N , and sgn(z) = 1,if z < 0, and sgn(z) = 0 otherwise. Let T denote a threshold such that, if the number of unsatisfied parity-check equations where a code symbol ~ l iparticipates exceeds T , then symbol is "flipped", wi = 2ri @ 1. Figures 88 and 89 show simulation results for Gallager's (20; 5) code CG and theHamming (7; 4; 3) code C H ,respectively, withbinary transmission over an AWGN channel. The maximum number of iterations was set to 4 for CG and 2 forC H .In both figures, the threshold was set to T = 1;2: 3. For the Hamming code, in Figure 89, also shown are single-error harddecision decoding (HD), soft-decision MLD decoding (SD) and the union bound (1.35). In Figures 90 to 92, the error performance of the Berlekamp-Massey (BM) algorithm and Gallager bit-flip (BF) algorithm is compared for the BCH (31;26,3), (31,21,5) and (31,16; 7) codes, respectively. It is evident that, as the error correcting capability of the code increases, the performance of the BF algorithm is inferior to that of the BM algorithm. On the other hand, in terms of computational complexity, the Gallager BF algorithm requires simple exclusive-or gates and comparisons, as opposed to GF(2") arithmetic processors in the case of the BM algorithm. This suggests that, for some high-rate codes, such as single- and doubleerror correcting BCH codes, the bit-flip algorithm might be considered an alternative to the BM algorithm. An additional feature of the BF algorithm, as well as the iterative probabilistic decoding algorithm presented below, is that its complexity depends only on the degrees of the nodes in the Tannergraph. In other words, for fixed values ofJ and K , the decoding complexity grows linearly with the code length. + This algorithm is also known as Gallager's algorithm A. THE ART OF ERROR CORRECTING CODING 162 4 5 6 7 9 8 10 12 11 13 14 15 Eb/No (dB) Figure 88 Performance of Gallager’s (20,5) code with bit-flip decoding. Four iterations with different threshold values 8.3.3 Iterative probabilistic decoding: belief propagation In this section, an iterative belief-propagation (IBP) decoding algorithm is presented. This algorithm is also known as Pearl’s algorithm [Prl], and sum-product algorithm [Wib, KFL, For7]. Wiberg in his Ph.D. thesis [Wib] has shown that the forward-backward, turbo and Gallager algorithm (described below) are all special cases of the sum-product algorithm. Furthermore, it was demonstrated [MMC] that iterative decoding algorithms for turbo codes and product codes are particular cases of IBP decoding. The description below follows closely [Prl, Mac] for binary LDPC codes. The following notation is convenient in describing the algorithm. Let hi,j denote the entry of W in the i-th row and j-th column. Let C(m)= {e : hm,e = l}; (8.20) denote the set of code positionsthat participate in the m-th parity-check equation, and let M(!) = { m : hm,e l}; denote the set of check positions in which code position l participates. The algorithm iteratively computes two types of conditional probabilities: e has the value z, given the information obtained via the check nodes other than check node m. qgc The probability that the l-th bit of rge The probability l o that a check node m is satisfied when bit!is fixed to a value z and the other bits are independent with probabilities qmp , l’ E C ( m ) \ ! This is an abuse of notation, because T %denotes the i-th component of the received vector. However, it helps to keep the same notation as most of the literature in the topic. l o Note: (8.21) ITERATIVELY DECODABLECODES 3 4 5 163 7 8 Eb/No (dB) 6 10 9 11 12 Figure 89 Performance of a Hamming ( 7 , 4 , 3 ) code with bit-flip decoding. Two iterations with different threshold values. As noted in [Prl, Mac], the following IBP decoding algorithm would produce the exact aposteriori probabilities after some number of iterations, if the Tanner graph of the code containedno cycles.In the following, binary transmission over an AWGN channel is assumed. As before, the modulated symbols m(vi) are transmitted over an AWGN channel and received as ri = m(vi) wi,where wi is a Gaussian distributed random variable with zero mean and variance No/2, l 5 i 5 N . + Initialization For C E { 1,2, . ’ ., N } , initialize the a-priori probabilities of the code nodes, 1 1- - I and p! = 1 - p i . For every (C, m) such that h,,! 0 %,e 0 (8.22) + exp(re*-’ = Pe , = 1, I %,e 1 (8.23) = Pe . Message passing Step I : Bottom-up (horizontal): For each e, m, compute (8.24) THE ART OF ERROR CORRECTING CODING 164 1e-01 1e-02 K W m le-03 1e-04 1e-05 0 1 2 3 4 5 6 Es/No (dB) 7 8 9 1 0 Figure 90 Performance of Berlekamp-Massey and Gallager bit-flip algorithms for the binary BCH (31,26,3)code. (8.25) Step 2: Top-down (vertical): For each e, m, compute (8.26) m'cM(e)\m m'€M(e)\m (8.27) For each e, compute the a-posteriori probabilities (8.28) and normalize, with a = l/($ + Q ; ) , (8.29) 165 ITERATIVELY DECODABLE CODES Es/No (dB) Figure 91 Performance of Berlekamp-Massey and Gallager bit-flip algorithms for the binary BCH (31,21,5) code. Decoding andsoft-outputs For i = 1 , 2 , . . . N , compute 6i = sgn(q9) (8.30) If 6HT = 0, then 6 is the estimated codeword and the soft outputs are A(Q) = lOg(4.t) - log(qp), 15i 5 N. (8.31) The algorithm stops. Otherwise, return to Step 2. If the number of iterations exceeds a predetermined threshold, a decoding failure is declared. Output received values as they are. The algorithm stops. Figure 93 shows the performance of IBP decoding for the binary cyclic PG (273,191,17) code. This code is a small member of the family of binary finite geometry LDPC codes recently introduced in [KLF]. Also shown in the plot is the performance of the BF algorithm with a threshold equal to 8. Notes As proposed in [Gal], the IBP decoding algorithm can be modified ina straightforward way to use log-likelihood ratios instead of probabilities. This has the advantage that no normalization is required in the second step of message passing. In this case, the function A ez+l 1 F ( z ) = -e" - 1 tanh(z/2)' (8.32) or its inverse, is needed in Step 1 of message passing. A program implementing this log-IBP decoder is available on the ECC web site. THE ART OF ERROR CORRECTING CODING 166 0 1 2 3 4 5 6 EslNo (dB) 7 8 9 1 0 Figure 92 Performance of Berlekamp-Massey and Gallager bit-flip algorithms for the binary BCH (31,16,7) code. Note that F ( z ) can be implemented as a look-up table. Numerical results suggest that quantization of F ( z )to eight levels results in practically no performance loss compared with a floating point version. In [FMH], several reduced-complexity versions of the IBP algorithm are presented that favorably trade off decoding complexity and error performance. Other interesting applications of iterative decoding algorithms include fast-correlation attacks in cryptanalysis [MG, Glc]. It is notable that LDPC codes can always detect whenever decoding errors occur, whereas turbo codes and product codes based on convolutional codes cannot detect many errors [Mac]. As it is evident from the foregoing presentation, LDPC codes have very low complexity compared with iterative decoding using component MAP decoders. This is truewhen complexity is measured in terms of the number of real additions and multiplications per block per iteration. However, it should be pointed out that IBP decoding gives the MLD codeword only if the Tanner graph contains IZO cycles. Because for most practical codes the Tanner graph has relative short cycles (lengths 4 and 6), convergence of the IBP decoding algorithm is either not guaranteed or slow [RSU, KFL, For71. As a result, in general, IBP decoding algorithms for LDPC codes may require much more iterations than iterative decoding algorithms for product codes with MAP component decoders. In terms of implementation of IBP decoding, two architectures are the following. A fast parallel architecture can be realized with N X-processors for code nodes and M Z-processors for check nodes, with connections between them specified by multiple address computation units (ACU). This architecture is shown in Figure 94. In the figure, X and 7r are LLR values associated with the conditional probabilities used in the IBP algorithm. Alternatively, only one X-processor and one Z-processor can be used and shared between nodes, whose metrics are stored in two memories, a 7r-memory and a X-memory, as shown in Figure 95. These architectures represent the extremes in the spacehime tradeoff to find the best architecture. 167 ITERATIVELY DECODABLECODES 1e-01 1e-02 K W m 1e-03 1e-04 1e-05 0 1 2 3 4 5 6 7 8 9 10 Eb/No (dB) Figure 93 Performance of iterative decoding of a binary (273,191) cyclic code. The number of computations in the IBP decoding algorithm can be reduced by making preliminary hard decisions based upon the amplitudes of the log-likelihood ratio values (reliabilities) of the channel symbols. This idea appears in [FK] and in [Prl] as a method of dealing with short cycles in Bayesian networks (Tanner graphs). A set of highly reliable positions (HRP) can be selected by comparing the channel LLR values to a threshold T . If the reliability of a channel symbol exceeds T , then that code node is fixed to a hard decision (HD) value. As a result, the X-processor is not utilized in the decoding process. Instead, this processor always gives as an output either the maximum probability (LLR) or a flag indicating that this position is highly reliable. Consequently, the 2-processor to where the HD value is sent performs a smaller number of computations, because this input is highly reliable and not taken into account in the probability or LLR computations. THE ART OF ERROR CORRECTING CODING 168 REG 1 1'1 ! Figure 94 Parallel architecture of an IBP decoder. BOTTOM-UP MESSAGE PASSING 7K- 1RAM Figure 95 Serial architecture of an IBP decoder. The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) Combining codes and digital modulation In discussing soft-decision decoding, attention was focused on binary transmission, i.e., using two possible transmitted symbols { -1, +l}. Let the Nyquist bandwidth of a transmission (or storage) signal be the rate R, at which symbols are transmitted (or stored). For binary transmission, the two values of a bit, 0 and 1, are assigned to two values of the transmitted symbol, +l and -1, respectively. Therefore, R, bitdsecond require a Nyquist bandwidth of R, Hz. The spectral eficiency p of a binary transmission system equals 1 bit/sec/Hz (or 1 bit/symbol). Coded modulation is the joint design of error correcting codes and digital modulation formatsto increase the bandwidth efficiency of a digital communication system. 9.1 Motivation Suppose that an error correcting coding scheme is required to increase the reliability of the binary transmission (orstorage) system. Let R, = k / n denote the rate of the code. Then the spectral efficiency is p = R, bps/Hz. Thus for a codedbinary transmission system a spectral efficiency p 5 1 bps/Hz is achieved. This translates into a faster signalling rate or larger bandwidth for the same bit rate. Equivalently, the bit rate has to be reduced, by a factor of l/R,, so as to keep the transmitted symbolrate or bandwidthconstant. Example 94 Assume that 56 kbps are desired to be transmitted over a channel. The required Nyquist bandwidth of an uncoded binary transmission system is 56 kHz. Suppose a rate1/2 convolutional codeis used. Then, with binary transmission, the spectral efficiency of the coded system is p = 0.5 bps/Hz. The effect of p in data rate and signalling rate is illustrated in Figure 96. Increasing the bandwidth of the signal, as in Figure 96 (a) is not a practical solution, since typically the channel bandwidth is expensive (or limited, as in a telephone line). Decreasing data rate (Figure 96 (b)) is a possible solution, but places a limit on the number of services or applications offered. In addition, the increased transmission delay may not be acceptable (e.g., voice or video). The fundamental question then is the following: How to increase the data rate without increasing the bandwidth (or symbol rate)? The answer, as given by Ungerboeck [Ungl] and Imai and Hirakawa [IH], is to use an expanded signal set (e.g., 2”-ary PSK or QAM digital modulation) and then to apply error correcting coding to increase the Euclidean distance between coded sequences. THE ART OF ERROR CORRECTING CODING 170 - kbps 112 S6 kbps Rate- 1/2 encoder I Binary modulator f--+ Channel i kbps 112 56 kbps Decoder Demodulator lo (a) Keeping bit rate constant kbps 28 __ I 0I modulator kbps 4 S6 kbps Rate-IR encoder 1 4 Binary 56 kbps 28 (b) Keeping symbolrate constant Figure 96 Effect of coding in binary transmission of information. 9.1. l Examples of signal sets Several signal sets used in digital communication systemsare shown in Figure 97. The I and Q axis represent orthogonal signals that are used in transmission.' In digital communication systems, generally I = cos(2.rrfct), Q = sin(2.rrfct), (9.1) where I stands forin-phase and Q for quadrature. If a point in the IQ-plane has coordinates (x,y), then the transmitted signal is ~ ( t=)R c o s ( ~ T + ~4), ,~ ( k - l) T 5 t < kT, (9.2) d m , where R = (p = tan-l(y/z), and T is the symbol duration. In other systems (e.g., storage), orthogonal pulsesmay be used, such as those illustrated in Figure 98. The set of signal symbols in the two-dimensional IQ-planeis called a constellation, and symbols are called signal points. From the viewpoint of digital signal processing, modulation is a mapping. That is, the process of assigning an v-dimensional binary vector 6 to a signal point (x($),y(6)) in the A A For simplicity of exposition, let I = I ( t ) and Q = Q ( t ) . COMBINING CODES ANDDIGITAL MODULATION 1 bit/symbol 2 bitslsymbol 171 3 bitslsymbol l BPSK r 4 bitdsymbol QPSK 8PSK 5 bits/symbol 6 bitslsymbol t" .... 16-QAM 32-QAM 64-QAM Figure 97 Examples of M-PSK and M-QAM signal constellations. constellation. In previous chapters, only BPSK was considered, in which case U = 1. For U > 1, there are many possible assignments of bits to signal points. That is, many ways to label the signal points. Figure 99 shows an example of a QPSK constellation (U = 2) with two different (non-equivalent) mappings. Moving from binary modulation to 2v-ary modulation has the advantage that the number of bits per symbol is increased by a factor of U , thus increasing the spectral eficiency of the system. On the other hand, the required average energy of the signal increases (QAM), or the distance between modulation symbols decreases (PSK). In practice, transmitted power is limited to a maximum value. This impliesthat the signal points become closerto each other. Recall that the probability of error in an AWGNchannel betweentwo signal points separated by an Euclidean distance equalto D,is [Hay, Pro] where &(x) is given by (1.2). As a result, a higher probability of error is experienced at the receiver end. In this sense, the function of error correcting coding is to reduce the probability of error P T ( € ) and to improve the quality of the system. THE ART OF ERROR CORRECTING CODING 172 Figure 98 An example of two orthogonal pulses. 11 10 01 (a) GRAY MAP 01 (b) NATURAL MAP Figure 99 Two different mappings of a QPSK constellation. 9.1.2 Coded modulation In 1974,Massey introduced the key concept of treating coding and modulation as a joint signal processing entity [Mas3], see Figure 100. That is, the coordinated design of error correcting coding and modulation schemes. ModulatorEncoder Source I \ (Massey, 1974) Combine coding and modulation Figure 100 The idea of joint coding and modulation Two fundamental questionson combining coding andmodulation arise: 1. How to construct the bits-to-symbols mapping? 2. How to assign coded bit sequences to coded symbol sequences? COMBINING CODES ANDDIGITAL MODULATION 173 Two basic approaches were proposedin the 1970s to design coded modulation systems: 1. Trellis Coded Modulation (TCM) [ U n g l ] Apply a natural mapping of bits to signals, through set partitioning. Givenan underlying finite-state machine, assign symbol sequences to trellis paths. Perform Viterbi decoding at the receiver. 2. Multilevel Coded Modulation (MCM) [IH] Apply a mappingof codewords to bit positions, through a binary partition. For 2”ary modulation, use v error correcting codes, one perlabel bit. Perform multistage decoding at the receiver. In both TCM and MCM, the basic idea is to expand the constellation in order to obtain the redundancy needed for error correcting coding, and then to use coding to increase the minimum Euclidean distance between sequencesof modulated signals. 9.1.3 Distanceconsiderations To illustrate how error correcting codes and digital modulation can be combined, and the consequent increase in the minimum distance between signal sequences from an expanded signal constellation, consider the following. Example 95 A block coded QPSK modulation scheme is shown in Figure 101. Codewords of the extended Hamming (8;4,4) code are divided in four pairs of symbols and mapped to QPSK signal points with Graymapping.A nice feature of Gray labeling of QPSK points is that the squared Euclidean distance between points is equal to twice the Hamming distance between their labels. As a result, a block coded modulation scheme with p = 1 bits/symbol and aminimum squared Euclidean distance, or MSED, DLi,, = 8 is obtained. An uncoded system with the same spectral efficiency is BPSK, which has an MSED DE,, = 4. Consequently, the asymptotic coding gain of this scheme is G = 1Ologl0 (2) = 3 dB. (9.4) Over an AWGN channel, this coded QPSK modulation requires half the power of uncoded BPSK modulation,to achieve the same probability of error P ( € )Figure . 102 shows simulation results of this scheme. I 4 bits I Extended Hamming Figure 101 Block coded QPSK modulation using an extended Hamming (8,4,4) code. THE ART OF ERROR CORRECTING CODING 174 1e-01 1e-02 U W m le-03 1e-04 1e-05 0 1 2 3 4 5 6 Eb/No (dB) 7 a 9 10 Figure 102 Simulation results of a block coded QPSK modulation using an extended Hamming (8,4,4) code. AWGN channel. Therefore, while the use of an expanded 2”-ary modulation causes the distance between signal point to decrease, a properly selected error correcting code can make sequences of signal points at a minimum distance larger than that of an uncoded system, with the same spectral efficiency. 9.2 Trellis-codedmodulation (TCM) Proposed by Ungerboeck in 1976, the main idea in TCM isto perform mapping by set partitioning. A basic trellis structure, associated with the state transitions of a Jinite-state machine, is selected and signal subsets mapped to trellis branches. For systems that require high spectral efficiency, uncoded bits may be assigned to parallel branches in the trellis. 9.2.1 Set partitioning and trellis mapping Bit labels assigned to the signal points are determined from partition a of the constellation. A 2”-ary modulation signal set S is partitioned in v levels. For 1 5 i 5 v, at the i-th partition level, the signal set is dividedinto two subsets Si(0) and Si (l),such that the intra-set distance, S:, is maximized. A label bit bi E (0, l} is associated with the subset choice, S i ( b i ) , at the i-th partition level. This partitioning process results in a labeling of the signal points. Each signal point in the set has a unique v-bit label blbz . . . b,, and is denoted by s ( b l ~ b2: . . . b y ) . With this standard (Ungerboeck) partitioningof a 2”-ary modulationsignal constellation, the intra-set distances are in nondecreasing order6: 5 62 5 . . . 5 6:. This strategy corresponds to a natural labeling for M-PSK modulations, i.e., binary representations of integers, whose AND COMBINING CODES DIGITAL MODULATION 175 value increases clockwise (or counter-wise). Figure 103 shows a natural mapping of bits to signals for the case of 8-PSK modulation,with S: = 0.586, 6; = 2, and Si = 4. Ungerboeck regardedthe encoder “simply as a$nite-state machine with a given number of states and specijied statetransitions”. He gave a set of pragmatic rules to map signal subsets and points to branches in a trellis. These rules can be summarized as follows: Rule 1: All subsets should occurin the trellis with equal frequency. Rule 2: State transitions that begin and end in the same state should be assignedsubsets separated by the largest Euclidean distance. Rule 3: Parallel transitions are assigned signal points separated by the largest Euclidean distance (the highest partition levels). The general structure of a TCM encoder is shown in Figure 104. In the general case of a rate (v - l ) / v TCM system, the trellis structure is inherited from a k / ( k 1) convolutional encoder. The uncodedbits introduce parallel branches in the trellis. + Example 96 In this example, a 4-state rate-2/3 TCM system is considered. A constellation for 8-PSK modulation is shown in Figure 103. The spectral efficiency is p = 2 bits/symbol. A block diagram of the encoder is shown in Figure 105. The binary convolutional code is the same memory-2 rate-1/2 code that was used in Chapter 5. Note from Figure 106 that the trellis structure is the same as that of the binary convolutional code,with the exception that every branchin the original diagram is replaced by two parallel branches, associatedwith the uncoded bit u1. 9.2.2 Muximum-likelihood decoding The Viterbi algorithm2 can be applied to decode the most likely TCM sequences, provided that the branch metric generatoris modified to include parallel branches. Also, the selection of the winning branch and surviving uncodedbits should be changed. The survivor path(or trace-back) memory should include the (v - 1 - IC) uncoded bits, as opposed to just one bit for rate-l/n binary convolutional codes. It is also important to note that in 2”-ary PSK or QAM modulation, the correlation metria for two-dimensional symbols are of the form xPxT ypy,., where (xp,yp) is a referencesignal point in the constellation and ( x T ,y,.) is the received signal point. All other implementationissues discussed in Sections 5.4 and 7.2 apply to TCM decoders. + 9.2.3 Distance considerations and error pelfomance The error performance of TCM can be analysed in the same way as for convolutional codes. That is, a weight enumerating sequence canbe obtained from the state diagram of the TCM ~~ ~~ The Viterbi algorithm is discussed in Sections5.4 and 7.2. THE ART OF ERROR CORRECTING CODING 176 b3 b2b, = 100 olo/o 101 b2 = 0 0 b3= 1 b3= 0 b3 = 0 b3=0 b3= 1 Figure 103 Natural mapping of an 8PSK constellation. encoder, as in Section 5.3. The only difference is that the powers are not integers (Hamming distances) but real numbers (Euclideandistances). Care needs to be taken of the fact that the state transitions contain parallel branches. This means that the labels of the modified state diagram containtwo terms. See [BDMS]. Example 97 Figure 107 shows the modified state diagram for the 4-state TC8-PSM modulation of Example 96. Thebranches in the trellis havebeen labeled with integers corresponding to the eight phases of the modulation signals. To compute theweight enumerating sequenceT ( z ) ,the same procedureas in Section 5.3 is applied. Alternatively, by directly analysing the trellis structure in Figure 108, it can be deduced that theMSED between coded sequencesis D; = min{Di,,, D&} = 3.172, which, when compared to an uncoded QPSK modulation system with the same spectral efficiency of 2 bits/symbol, gives an asymptotic codinggain of 2 dB. 9.2.4 Pragmatic TCM and two-stage decoding For practical considerations, it was suggested in [Vit4, ZW] that the 2U-ary modulationsignal constellations be partitioned in such a way that the cosets at the top two partition levels are COMBINING CODES ANDDIGITAL MODULATION 177 2’-ary modulation ........................ Rate-W(k+l) k bits (coded) I Selection of ’ convolutional 7signal subset encoder v -1 -k bits (b =- (uncoded) .. .... b,,,) b2, j j I Selection of point . in thesubset (b k+2, .... b v 1 . (x,y) > ....................... Figure 104 General encoder of rate-(v - l ) / v trellis-coded modulation. Figure 105 Encoder of a 4-state rate-2/3 trelliscoded 8-PSK modulation. associated with the outputs of the standard memory-6 rate-l/:! convolutional encoder. This mapping leads to a pragmatic TCM system. With respect to the general encoder structure in Figure 104, the value of k = 1is fixed, as shown in Figure 109.As a result, the trellis structure of pragmatic TCM remainsthe same, as opposed to Ungerboeck-type TCM,for all values of v > 2. The difference is that the number of parallel branches v - 2 increases with the number of bits per symbol. This suggests a two-stage decoding method in which, at the first stage, the parallel branches in the trellis “collapse” into a single branch, and a conventionaloff-the-shelf Viterbi decoder usedto estimate the coded bits associated with the two toppartition levels. In a second decodingstage, based on the estimated coded bits and the positions of the received symbols, the uncoded bits are estimated. Figure 110is a block diagram of a two-stage decoder of pragmatic TCM. In [MM], a symbol transformation is applied to the incoming symbols that enables use of a Viterbi decoder without changesin the branch metric computationstage. The decoding procedure is similar to that presented in [CRKO, PS], with the exception that, with symbol transformation, the Viterbi algorithm can be applied as if the signals were BPSK (or QPSK) modulated. Thismethod is describedbelow for M-PSK modulation. Specifically, let (x,y) denote the I and Q coordinates of a received M-PSK symbol with amplitude T = and phase 4 = tan-’ (y/z). Based on 4,a transformation is applied suchthat the M-PSK points are mapped into “coset” points labeled by the outputs of a rate- 1/2 64-state convolutional encoder. d m THE ART OF ERROR CORRECTING CODING 178 Figure 106 Trellis structure of a rate-2/3 trellis coded 8-PSK modulation based on the encoder of Figure 105. For TCM with M-ary PSK modulation, M = 2’, v > 2, let 5 denote the number of coded bits per symbol3, where 5 = 1:2. Then the following rotational transformation is applied to each received symbol, (x,y), to obtain an input symbol (2’; y’) to the VD, where is a constant phase rotation of the constellation that affects all points equally. Under the transformation (9.5), a 2”-C-PSK coset in the original 2”-PSK constellation “collapses” into a coset pointin a 2c-PSK coset constellation in the 2’ - y’ plane. Example 98 A rate-2/3 trellis-coded 8-PSK modulation with 2 coded bits per symbol is considered. Two information bits (u1; u2) are encoded to produce threecodedbits (u2; v2;wl), which are mapped onto an 8-PSK signal point, where (v2;VI) are the outputs of the standard rate-l/2 64-state convolutional encoder4. The signal points are labeled by bits (u2; v2;V I ) , and the pair ( v ~V; I ) is the index of a coset of a BPSK subset in the 8-PSK constellation, as shown at the top of Figure 1 l 1. In this case$‘ = 2 4 and, underthe rotational transformation, a BPSK subset in the original 8-PSK constellation collapses to a coset point of the QPSK coset constellation in the 5’ - y’ plane, as shown in Figure 1 1 1. Note that both points of a given BPSK coset have the same value of 4’. This is because their phases are given by $ and 4 + 7r. < The case = 2 corresponds to conventionalTCMwith8-PSKmodulation. The case = 1 isusedin TCM with coded bits distributed over two 8-PSK signals, such as the rate-5/6 8-PSK modulation scheme proposed in the DVB-DSNG specification [DVBl. v1 and v2 are the outputs from generators 171 and 133, in octal, respectively. COMBINING CODESAND DIGITAL MODULATION 179 48 a = d(0,4) = 2 b = d(0.1) = 0.765 c = d(0,2) = 1.414 d = d(0.3) = 1.848 Figure 107 The modified state diagram of a 4-state TC 8-PSK modulation scheme. The output of the VD is an estimate of the coded information bit, ul. In order to estimate the uncoded information bit, u2, it is necessary to re-encode u1 to determine the most likely coset index. This index and a sector in which the received 8-PSK symbol lies can be used to decode u2. For a given coset, each sector S gives the closest point (indexed by u2) in the BPSK pair to the received 8-PSK symbol. For example, if the decoded coset is (1,1) and the received symbollies within sector 3, then u2 = 0, as can be verified from Figure 1 1. 1 Figure 1 12 shows simulation results of MLD (denotedwith the legend “TC8PSK-23-SSD”) and two-stage decoding of pragmatic TC-8PSK (legend “TC8PSK23-TSD”). With two-stage decoding, aloss in performance of only 0.2 dB is observed comparedwith MLD. A similar transformation can be applied in the case of M-QAM, the difference is that the transformation is based solely on the I-channel and Q-channel symbols. That is, there is no need to compute the phase. An example is shown in Figure 113 for TC 16-QAMwith 6 = 2 coded bits per symbol. The coded bits are now the indexes of cosets of QPSK subsets. The transformation of 16-QAM modulation (5 = 2) is given by a kind of “modulo 4” operation: Finally, it is interesting to note that a pragmatic TCM systemwith component codewas recently proposed in [WMI. aturbocode as THE ART OF CORRECTING ERROR CODING I80 ,586 = 3. I72 V Figure 108 Two paths at minimum squared Euclidean distance in the trellis of Example 96. Memory-6 Rate-112 UI bits-to-signal mapping for go=(11110O1) g,=(1011011) 1 2' -ary modulation v - 2 bits t U2 Figure 109 Block diagram of an encoder of pragmatic TCM. 9.3 Multilevelcodedmodulation(MCM) In the Imai-Hirakawa multilevel coding scheme [IH], the 2V-ary modulation signal set is binary partitioned in v levels. The components of codewords of v binary component codes Ci, 1 5 i 5 v, are used to index the cosets at each partition level. One of the advantages of MCM is the flexibility of designing coded modulation schemes by coordinating the intra-set Euclidean distances, d:, i = 1;2;. . . v, at each level of set partitioning, and the minimum Hamming distances of the component codes. Wachsmann et al. [WFH] have proposed several design rules that are based on capacity (by applying the chain rule of mutual information) arguments. Moreover, multilevel codes with long binary component codes, such as turbo codes orLDPC codes, were shown to achieve capacity [WFH, For81. It also worthwhile noting that, while generally binary codes are chosen as component codes, i.e., the partitionis binary, ingeneral the component codes can be chosen from any finite field G F ( q ) matching the partition of the signal set. Another important advantage of multilevel coding is that (binary) decoding can be performed separately at each level. This multistage decoding results in greatly reduced complexity, compared with MLD for the overall code. ~ COMBINING CODESAND DIGITAL MODULATION 181 . . . . . . . . . . . . . . . . LUT I X X' c COSET MAPPING Y I : Y' * D RATE- 112 VITERBl . DECODER )"I RATE-112 ENCODER - SECTOR MAPPING I I SECTOR D SECTOR DELAY -----c COSET INDEX INTRA-COSET SELECT (LW t u 2 ...... ....... Figure 110 Block diagram of a two-stage decoder of pragmatic TCM. 9.3.1 Constructions and multi-stagedecoding For 1 5 i _< v, let Ci denote a binarylinear block ( n ,ki,di) code. LetVi = (vil, vi2,.... uin) be a codewordin Ci,1 5 i 5 U. Consider apermuted time-sharing code .rr(lC1IC2 I ... IC,[), with codewords v = (ZIllW21 . . . w,1 21122122 ... v,2 ... 211n212,. .. U v n ) . Each v-bit component in V is the label of a signal in a 2"-ary modulation signal set S. Then 421) = ( S ( ~ 1 1 w 2 1 ~ ' ~ 0 u 1 ) , S ( 2 1 1 2 w 2 2 ~ ~ ~ 2 1 u 2 ) , ~ ~ ~ , S ( w l n w 2 n . ~ ~ ~ w v n ) is a sequenceof signal points in S. The followingcollection of signal sequences over S , Ab { ~ ( v ): V E .rr(lCllC~l...lC,I)}, forms a u-level modulation code over the signal set S , or a v-level coded 2"-ary modulation. The samedefinition can be applied to convolutional component codes. The rate, or spectral efficiency, of this codedmodulationsystem, in bitshymbol,is R = ( k l k2 . . . k,)/n. The MSED of this system, denotedby D ; ( A ) , is given by [IH] + + + A > min {did:}. - l<?,<, (9.6) Example 99 In this example,a three-level blockcoded8-PSKmodulationsystem is considered. The encoderstructure is depicted in Figure 114. Assuminga unit-energy 8-PSK signal set, and with reference to Figure 103, note that the MSED at each partition level are 6f = 0.586,s; = 2 and 6: = 4. The MSEDof this coded 8-PSK modulationsystem is: D&(A)= min{dld?, d&, dsd;} = min{8 x 0.586,2 x 2, l x 4) = 4, and the coding gain is 3 dB with respect to uncoded QPSK The trellises of the component codes are shown in Figure 115. The overall trellis is shown in Figure 116. THE ART OF ERROR CORRECTING CODING 182 COSET 00 COSET 01 COSET l 1 COSET 10 t y I ’ + y t’ Figure 111 Partitioning of an 8-PSK constellation (E = 2) and coset points. As mentioned before, one of the advantages ofmultilevel coding is that multistage decoding can be applied. Figures 117 (a) and (h) show the basic structures used in encoding and decoding of multilevel codes. Multistage decoding results in reduced complexity (e.g., measured as number of branches in trellis decoding), comparedto MLD decoding (e.g., using the Viterbi algorithm and the overall trellis.) However, in multistage decoding, the decoders at early levels regard the later levels as uncoded. Thisresults in more codewordsat minimum distance, i.e., an increase in error multiplicity or number of nearest neighbors. The value of this loss depends on the choice of the component codes and the bits-to-signal mapping, and for BER lo-’ 10W5 can be in the order of several dB. Example 100 In this example, multistage decodingof three-level coded 8-PSK modulation is considered. The decoder in the first stage uses the trellis of the first component code Cl. Branch metrics are the distances (correlations) from the subsets selected at thefirst partitioning level to the received signal sequence, as illustrated in Figure 1 18. Once a decision is made inthe first stage, it is passed on to the second stage. The decoderin the second stage uses the trellis of the second component code with information from the first stage. For 8-PSK modulation, if the decoded bit in the first stage is bl = 0, then the received signal sequence is unchanged. If the decoded bit is bl = l, then the received signal is rotated by 45”. Again, branch metrics are distances (correlations) from the subsets selected at the COMBINING CODESAND DIGITAL MODULATION 183 a: W m le-05 . - .. I 6.5 6 I I I 7 8 7.5 I I 8.5 I I I 9 9.5 10 EslNo (dB) I I I I I' 10.5 11 11.5 12 12.5 Figure 112 Simulation results of MLD versus two-stage decoding for pragmatic 8-PSK modulation. second partitioning stage - given the decision at the first decoding stage - to the receive signal sequence. Finally, based on the decisions in the first two decoding stages, the decoder of the third component is used. The branch metrics are the same as for BPSK modulation. There are four rotated versions of the BPSK constellation, in accordance with the decisions in the first two decoding stages. Therefore one approach is to rotate the received signal according to the decisions on blba and use the same reference BPSK constellation. This is illustrated in Figure 119. For medium to large code lengths, hybrid approaches may be the way to go for ultimate MCM performance, with powerful turbo codes used in the top partition levels and binary codes with hard-decision decoding assigned to lower partition levels. These combinationscan achieve excellent performance [WFH]. 9.3.2 Unequal-error-protectionwith MCM Because ofits flexibility in designing the minimum Euclidean distances between coded sequences at each partition level, MCM is an attractive scheme to achieve unequal error protection (UEP). However, great care has to be exercised in choosing the bits-to-signal mapping, so that the desired UEP capabilities are not destroyed. Thisissue was investigated in [MFLI, IFMLI], where several partitioning approaches were introduced that constitute generalizations of the block [WFH] and Ungerboeck [Ungl] partitioning rules. In these hybrid partitioning approaches, some partition levels are nonstandard while at other levels partitioning is performed using Ungerboeck's rules [Ungl]. In this manner, a THE ART OF ERROR CORRECTING CODING 184 Y A 0110 c2 , 0011 ob1 3 d l 0 8 -s=2 j S::l j, s=o ...... ............. -3 : s=5 -1 S =4 il s=3 ...... 3 l 1110 ......@) .......... b X 1011 ..e.. S=6 0 : -3 Figure 113 16-QAM constellation for pragmatic TCM with two-stage decoding. 7 bits 8 bits 4F-I encoder (8,8,1) encoder b, Natural mapping 8 signal-point sequence b3 Figure 114 Example of MCM with 8-PSK modulation. good tradeoff is obtained between error coefficients and intra-level Euclidean distances. To achieve UEPcapabilities, the Euclidean distances at each partition level are chosen suchthat dlS; 2 d2S; 2 . . . 2 d,S;. (9.7) For 1 5 i 5 v, let i&(iii) be the codeword of Ci in correspondence to a Ici-bit message vector iii, and let S = S ( G ) and S’ = S(G‘) denotecoded2”-arymodulation signal sequences correspondingto message vectors U = ( G I , U 2 , . . . , G v )and i i‘ = (G;, G;, ... , G:), respectively. The Euclidean separations [YI] between coded sequences at the i-th partition level, for i = 1,. . . ,v, are defined as A si = min{d(S,S’) : ~i # G ! , , U ~= G S , ~< i ] , (9.8) with SI = dl&:, s2 = d262, .. ., S , = d,SE. For transmissionover an AWGN channel, the set of inequalities (9.7) results in message vectors with decreasing error protection levels. It is known from [WFH]that Ungerboeck’s partitioning rules [Ungl] are inappropriate for multistage decodingof multilevel coded modulations,at low to medium signal-to-noise ratios, because of the large number of nearest neighbor sequences(NN) in the first decoding stages. Example 101 Figure 120 shows simulationresults of the performance of a three-level coded 8-PSK modulation with the (64,18,22), (64,57,4)and (64,63,2) extended BCH codes (exBCH codes) as component codes Ci, i = 1,2,3, respectively. The Euclidean separations are COMBINING CODES ANDDIGITAL MODULATION 185 .- - . . ... - 0 ..... 1 Figure 115 Trellises of component codes of an example MCM with 8-PSK modulation. = 12.9, s2 = s3 = 8, for 18 and 120 information bits, respectively (asymptotic coding gains of 8.1 dB and 6 dB, respectively). The adverse effects of the number of NN (or error coefficient) in the first decoding stage are such that the coding gainsare greatly reduced. SI In the following, a UEP scheme based on nonstandard partitioning is presented. The reader is referred to [WFH, MFLI, IFMLI] for details on multilevel coding design for both conventional (equalerror protection) and UEP schemes. Nonstandard partitioning The block partitioning [ W H ] shown in Figure 121 (a)is used to construct three-level coded 8-PSK modulation schemes with UEP. In the figure, the color blackis used torepresent signal points whose label is of the form Obzb3, with b z , b3 E (0, l}. Similarly, the color white is used for points with labels l b z b 3 . A circle indicates that the label is of the form blobs, bl ,b3 E (0, l},while a squareis used to represent signal points with labels bl l b 3 . It can be seen from Figure 121 (b) that in order to determine the value of the first label bit, b l , only the X-coordinate is sufficient. If a signal point is on the left-hand half plane ( X < 0) then it corresponds to bl = 0, otherwise it corresponds to bl = 1.In the same way, the Y-coordinate suffices to determine the value of the second label bit bz. If a signal point lies in the upper half plane (Y > 0), then b2 = 0, otherwise bz = 1. This property of block partitioning allows the first and second levels to be decoded independently or in parallel. A similar observation led to the development of parallel decoding (PD) for multilevel codes with Gray mappingin [Schr]. Multistage decoding In the first and second decoding stages, the decision variable is just the projection of the received signal sequence onto the X or Y axis, respectively. Figure 121 (c) shows a block THE ART OF ERROR CORRECTING CODING 186 Figure 116 Overall trellis of an example MCM with 8-PSK modulation. diagram of amultistagedecoder for a three-level coded 8-PSK modulation with block partitioning. The decoders for the first and second stages operate independently on the inphase and quadrature component of the received signal sequences, F, and F y , respectively. Once decisionsare made as to the estimates of the corresponding codewords,61 and 62,they are passed on to the third decoding stage. Let 6i = (6i1, 6i2,. . . ,6in) E Ci be the decoded codeword at the i-th stage, i = l,2 . Before the third-stage decoding, each two-dimensional coordinate( r z j ,r v j ) of the received signal F = (Fz, Fy) is projected onto a one dimensional coordinate r&, 1 5 j 5 n. The values rkj are the decision variables used by the decoder of C,. The projection depends on the decoded quadrant, which is indexed by the pair (Cl,, S,,), 1 5 j 5 n, as shown in the table below. This is a scaled rotation of F by 7r/4,so that the rotated sequence F' = (rL1,rkz, . . . , rkn) can be decoded using a soft-decision procedure for component code C,. Note that, unlike Ungerboeck partitioning, the independence betweenthefirstand second levels in block partitioning results in no error propagationfrom the first decoding stage to the second. For i = 1 , 2 , . . . , v, let A;) denote the number of codewords in Ci of weight W . Assuming systematic encoding, union a bound onthe bit error probability of the first decoding stage can be written as [MFLI, IFMLI] where d$(i) = $ [iA,+ (W - i)A,I2. The probability of a bit error in the second decoding COMBINING CODESAND DIGITAL MODULATION 187 2' -my modulation Bits-to-signal mapping (a) Multilevel Encoder - r . Decoder 1 - Delay T h t U1 Delay D, Delay T 2 qp-d Delay Dv-l Decoder v 'v (b) Multistage Decoder Figure 117 Basic structures of an encoder and a decoder of multi-level coded modulation systems. stage upper is also bounded by (9.9) using the same arguments above. The bound(9.9) can be compared with a similar one for the Ungerboeck's partitioning (UG) strategy: (9.10) From (9.9) and (9.10), it is observed that,while Ungerboeck's partitioning increases exponentially the effect of nearest neighbor sequences,by a factor of 2 w , the block partitioning has for d2,(w) = wA: an error coefficient term, 2 - w , that decreases exponentially with the distances of the first-level component code. As a result, for practical values of & / N o , the block partitioning may yield, at the first stage, a real coding gain even greater than the asymptotic coding gain. This is a very desirable feature of a coded modulationwith UEP. For nonstandard partitioning (NS), the second level is generally designedto have a larger coding gain thanthe third level. Under this assumption, a good approximation is obtained by assuming that decoding decisionsin the first and the second decoding stages are correct, (9.11) THE ART OF ERROR CORRECTING CODING 188 Figure 118 Trellis and symbols used in metric computations in the first decoding stage. ,..- 1 OR ROTATETHE RECEIVED SIGNAL POINT .... ..,, x b, b2= 00 (0 deg) *x t ....-. X . b,b2= 01 (-45 deg) x b, b,= l : (-90 deg) b,b2= 11 (-135 deg) Figure 119 Trellis and symbols used in metric computations in the third decoding stage. Example 102 Consider a three-level 8-PSK modulation forUEP with extended BCH (64,18,22), (64,45,8) and (64,63,2) codes as the first-, second- and third-level codes, respectively. This coding scheme has rate equal to1.97 bits per symbol and can be compared with uncoded QPSK modulation, which has approximately the same rate (a difference of only 0.06 dB). Simulation results are shown in Figure 122. S l ( n , k ) and UB(n, k ) denote simulations and upper bounds. An large coding gain of 8.5 dB isachieved at the BER of lop5 for 18 most important bits (14.3%) encoded in thefirst level. In the second and third stages, the corresponding values of coding gain are 2.5 dBand -4.0 dB, respectively5. Note that at this BER, thesimulatedcodinggainat the firstdecoding stage isevengreaterthan asymptotic coding gain (8.1 dB), because of the reducederror coefficients. the COMBINING CODES ANDDIGITAL MODULATION 189 0 "Sirn(64,18)" "Sirn(64,57)" "Sirn(64,63)" "QPSK c [I- -1 >: + ----0 x -2 h U: W m v zI -3 rs) 0 - -4 -5 L X -6 0 2 4 6 12 8 Eb/No (dB) 10 Figure 120 Simulation results of a three-level coded 8-PSK modulation with Ungerboeck mapping. 9.4 Bit-interleaved coded modulation (BICPVI) In [CTBl, CTB21, the ultimate approach to pragmatic coded modulation is presented. The system consists of binary encoding followed by a pseudo-random bit interleaver. The output of the interleaver is grouped in blocks of U bits which are assigned, via a Gray mapping, to points in a 2"-ary modulation constellation. The capacity of this bit-interleaved coded modulation (BICM) scheme has been shown to be surprisingly close to the capacityof TCM, when Gray mapping is employed. Moreover, over flat Rayleigh fading channels, BICM outperforms a CM with symbol interleaving [CTB2]. A block diagram of a BICM system is shown in Figure 123. Let X denote a 2"-ary signal constellation with minimum distance &in. For i = 1;2: . . . U , let li(s)be the i-th bit of the label of a signal point x , and let Xi C X the subset of signal points with labels such that the i-thbit has a value b E (0: l}. 9.4.I Graymapping A one-to-one and onto binary map m from { 0; l}" to X is a Gray mapping if, for all i = l;2 ; . . . ;v,and b E (0: l}, each 5 E X: has at most one nearest neighbor y E Xi,at distance Dmin,where b' = b 1. Gray mapping is the key component of a BICM system. Its main function is - ideally - to produce an equivalent channel that has v parallel, independent and memoryless, binary channels. Each channel corresponds to a position in the label of a signal x E X . For each codeword at the output of the binary encoder, the interleaver assigns at random a position in the label of the signals t o transmit the coded bits. THE ART OF ERROR CORRECTlNG CODING 190 Y OOO. I -x -A2-A, O1O0 A, A2 (b) - r X - Decoder 1 - r - Y c2 + c3 Figure 121 An 8-PSK constellation with block partitioning: (a) labeling; (b) X coordinate projections; (c) decoder structure.(Ci denote estimated codewords in C;,i = l, 2 , 3 . ) 9.4.2 Metricgeneration:De-mapping Before the description of how metrics for an MLD decoder are generated, some notation is needed. Let T denote the channel output after transmission of z. Assuming uniform input distribution, the conditional probability of r given ti(z)= b is p ( ~ l @ ( z=) b ) = C p ( ~ / ~ ) p ( ~ l=t b~)(=~ 2-(”-1) ) C ~(TIz). XEX (9.12) XEX; Let i denote the position of the coded bit ‘ujin the label of zcT(j). At each time j , let uj be a codesymboland the interleaved signal point, received as r c T ( jafter ) transmissionover a noisy channel. The receiver then produces bit metrics (9.13) for b = 0; l and i = 1;2,. . . v. An MLD algorithm, such as theViterbi algorithm, uses the above metrics andmakes decisions basedon the rule ~ 191 "UB(64,I 8)" "SI(64,18)" "UB/64.45Y t, i 0. 0.. 'W l o+ X i, -l0 i, lt X ? i i -5 0 15 10 5 20 Eb/No (dB) Figure 122 Simulation resultsof a three-level coded8-PSK modulation with UEP capability. BCH component codes and block partitioning. - - - Encoder Modulator K Mapping j. Channel ____ * Decoder Demodulator * Figure 123 A bit-interleaved coded modulation system. (9.14) As before, a max-log approximation of (9.13) is possible, resulting in the approximated bit metric, % h ( j ) ; b) = logP(5T(j) max ).I (9.15) Z€Xb' 9.4.3 Interleaving With transmission over an AWGN channel, a short interleaver will suffice.The main purpose is tobreak the correlationintroduced by the 2"-ary modulation signal set, which carries U bits per signal. Therefore, an interleaver of length equal to a few times Y is enough to approach best performance [CTB2]. Note thatthis interleaver has nothing to do with the interleaverthat a turbo code ora block product code would use. THE ART OF ERROR CORRECTING CODING 192 9.5 Turbo trellis-coded modulation (TTCM) Conceptually, there are various approaches to the combination of turbo codes, or product codes with interleaving, and digital modulation: Pragmatic coded modulation [LGB], turbo TCM with symbol interleaving [RWl, RW2] and turbo TCM with bit interleaving [BDMP2]. 9.5.1 Pragmaticturbo TCM Motivated by the extraordinary performance of turbo coding schemes, in 1994 [LGB], another pragmatic coded modulation scheme was introduced. Its block diagram is shown in Figure 124. The main feature is, as in pragmatic TCM, the use of the turbo encoder and decoder operating as in binary transmission mode. This requires careful computation of the bit metrics. as in the case of BICM. Store and n’- i Turbo - X Bits-to-signal mapping Y + V-k k Figure 124 Combination of a turbo encoder and digital modulation [LGB] 9.5.2 Turbo TCM with symbol interleaving In 1995, Robertson and Worz proposed the useofrecursive systematic convolutional encoders, such as those proposed by Ungerboeck [Ungl], as components in an overall coding system similar to that of turbo codes. A block diagram of this scheme is shown in Figure 125. As can be seen from the diagram, interleaving operates on symbols of v bits, instead of on bits for binary turbo codes. There isa need to puncture redundant symbols, due to the two paths of modulated signal points. A careful component code (noparallel transitions) and interleaver design (even positions to even positions, odd-to-odd; or even-odd and odd-even) are required. In terms of iterative decoding, note that the systematic component cannot be separated from the extrinsic one since they are transmitted together in one symbol. However, the LLR can be separated into an a-priori and a systematic-and-extrinsic part. Care must be taken so that the information is not used more than once in the component decoders. This is the reason why redundant symbol puncturing, in the form of a selector, is needed at the output of the encoder [RW2]. Figure 126 shows a block diagram of an iterative decoder for turbo TCM. 9.5.3 Turbo TCM with bit interleaving In 1996, Benedetto et al. [BDMP2] proposed symbol puncturing rules such that the outputs of the encoder contain the information bitsonly once. Moreover, as opposedto symbol interleaving and puncturing of redundant symbols, multiple bit interleavers were proposed. A COMBINING CODES ANDDIGITAL MODULATION v-l-k 193 * Mapping k I ”+ ns RSC 1 M even-to-even odd-to-odd * Mapping T RSC 2 Figure 125 The encoder structure of a turbo TCM scheme with symbol interleaving. Decoder 1 I Figure 126 An iterative decoder forturbo TCM with symboi interleaving. block diagramof the encoder structure is shown inFigure 127, for the case of two component codes. MAP decoding and bit metrics The decoder structure for turbo TCM with bit interleaving is similar to that of binary turbo codes. Themain difference is that conversion of LLRs frombits to symbols, and from symbols to bits, needs to be performed between decoders [BDMP2, VUC]. For decoding of turbo TCM with bit interleaving, the LLRs that are computed per bit need to be converted to a symbol level a-priori probability. Also, the a priori probabilities per symbolneed to beconverted to a bit level extrinsic LLR’s. This is done in the following way. Let X denote the 2”-ary modulation signal set. For a symbol z(6) E X with label 6 = ( b l , b2, ’ ., b”), the extrinsic information ofbit bi, i = 1 , 2 , . . . ,v, is computed as (9.16) THE ART OF ERROR CORRECTING CODING 194 (v - 1 - k )/2 bits (v - 1 - k )/2 bits i Mapping Figure 127 Encoder for turbo TCM with bit interleaving. Similarly, the a-priori symbol probability can be computed from the extrinsic LLR at the bit level through the expression, (9.17) The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) References [BCJR] L. R. Bahl, J. Cocke, F. Jelinek and J. Raviv, “Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate,” IEEE Trans. Infix Theory, vol. IT-20, pp. 284-287, Mar. 1974. [BP] A. S. Barbulescu and S. S. Pietrobon, “Interleaver Design for Turbo Codes,” Elect. Letters, vol. 30, no. 25, pp. 2107-2108, Dec. 1994. [Bat] G. Battail, “A Conceptual Framework for Understanding Turbo Codes,” IEEE J. Sel. Areas in Comm., vol. 16, no. 2, pp. 245-254, Feb. 1998. [BB] S. Benedetto and E. Biglieri, Principles of DigitalTransmission, Kluwer AcademiclPlenum Publishers, 1999. [BM] S. Benedetto and G. Montorsi, “Unveiling Turbo Codes: Some Results on Parallel Concatenated Coding Schenes,” IEEE Trans. Info. Theory, ~01.42,no. 2, pp. 409-428, March 1996. [BDMPI] S. Benedetto, D. Divsalar, G. Montorsi and F. Pollara, “Serial Concatenation of Interleaved Codes: Performance Analysis, Design, and Iterative Decoding,” IEEE Trans. Info. Theory, vol. 44, no. 3, pp. 909-926, May 1998. [BDMP2] S. Benedetto, D. Divsalar, G. Montorsi and F. Pollara, “Parallel Concatenated Trellis Coded Modulation,” Proc. 1996 IEEEInt. Con$ Comm. (ICC’96),pp. 974-978, 1996. [Berl] E. R. Berlekamp, Algebraic Coding Theory,rev. ed., Aegean Park Press, 1984. [Bed] E. R. Berlekamp, “Bounded Distance + 1 Soft-Decision Reed-Solomon Decoding,” IEEE Trans. Info. Theory, vol. 42, no. 3, pp. 704-720, May 1996. [BGI] C. Berrou and A. Glavieux, “Reflections on the Prize Paper: ‘Near Optimum ErrorCorrecting Coding and Decoding: Turbo Codes’,’’ IEEE Info. Theory Soc. News., vol. 48, no. 2, June 1998. [BG2] C. Berrou and A. Glavieux, “Near Optimum Error Correcting Coding and Decoding,” IEEE Trans. Comm., vol. 4 4 , no. 10, pp. 1261-1271, Oct. 1996. [BGT] C. Berrou, A. Glavieux and P. Thitimajshima, “Near Shannon Limit Error-Correcting Coding and Decoding: Turbo-Codes,” Proc. 1993 IEEEInt. Con$ Comm. (ICC’93),pp. 10641070, Geneve, Switzerland, May 1993. [BDMS] E. Biglieri, D. Divsalar, P. J. McLane and M. K. Simon, Introduction to Trellis-Coded Modulation with Applications,Macmillan Publishing, 1991. [Blah] R. E. Blahut, Theory and Practice of Error Control Codes,Addison-Wesley, 1984. [BL] M. Blaum, “A Family of Efficient Burst-Correcting Array Codes,” IEEE Trans. Info. Theory, vol. 36, no. 3, pp. 671-675, May 1990. [BZ] E.L.Blokh and V.V. Zyablov, “Coding of Generalized Concatenated Codes,” Probl. Pered. Informatsii, vol.lO, no.1, pp. 45-50, 1974. [BABSZ] M. Boo, F. Arguello, J. D. Bruguera, R. Doallo and E. L. Zapata, “High-Performance VLSI Architecture for the Viterbi Algorihtm,” IEEE Trans. Comm., vol. 45, no. 2, pp. 168-176, Feb. 1997. 196 REFERENCES [Bos] M. Bossert, Channel Coding for Telecommunications, John Wiley & Sons, 1999. [BH] M. Breiling and J. B. Huber, “Combinatorial Analysis of the Minimum Distance of Turbo Codes,” IEEE Trans. Info. Theory, vol. 47, no. 7, pp. 2737-2750, Nov. 2001. [Bro] A. E. Brouwer and T. Verhoeff, “An Updated Table of Minimum-Distance Bounds for Binary Linear Codes,” IEEE Trans. Info. Theory, vol. 39, no. 2, pp. 662-677, Mar. 1993. [BW] H. 0. Burton and E. J. Weldon, Jr., “Cyclic Product Codes,” IEEE Trans. Info. Theory, vol. IT-l 1, no. 3, pp. 433-439, July 1965. [CCG] J. B. Cain, G. C Clark and J. M. Geist, “Punctured Convolutional Codes of Rate ( n - l ) / n and Simplified Maximum Likelihood Decoding,” IEEE Trans. Info. Theory, vol. IT-25, pp. 97100, Jan. 1979. [CTBI] G. Caire, G. Taricco and E. Biglieri, “Capacity of Bit-Interleaved Channels,” Elect. Letters, vol. 32, pp. 1060-1061, June 1996. [CTB2] G. Caire, G. Taricco and E. Biglieri, “Bit-Interleaver Coded Modulation,” IEEE Trans. Info. Theory, vol. 44, no. 3, pp. 927-946, May 1998. [CRKO] F. Carden, M.D. Ross, B.T.Kopp and W.P. Osborn, “Fast TCM Decoding: Phase Quantization and Integer Weighting,” IEEE Trans. Comm., vol. 42, no. 4, pp. 808-812, Apr. 1994. [CY] C.-C. Chao andY.-L. Yao,“Hidden Markov Models for the Burst Error Statistics of Viterbi Decoding,” IEEE Trans. Comm., vol. 44, no. 12, pp. 1620-1622, Dec. 1996. [Cha] D. Chase, “A Class of Algorithms for Decoding Block Codes with Channel Measurement Information,” IEEE Trans. Info. Theory, vol. IT-18, pp. 170-182, 1972. [CK] P. Chaudhari and A. K. Khandani, “Using the Fourier Transform to Compute the Weight Distribution of a Binary Linear Block Code,” IEEE Comm. Let., vol. 5 , no. I , pp. 22-24, Jan. 2001. [CFRU] S.-Y. Chung, G. D. Forney, Jr., T. J. Richardson and R. Urbanke, “On the Design of Low-Density Parity-Check Codes within 0.0045 dB of the Shannon Limit,” IEEE Comm. Let., vol. 5, no. 2, pp. 58-60, Feb. 2001. [Cla] G. Clark and J. Cain, Error-Correction Codingfor Digital Communications, Plenum Press, 1981. [Col] 0. M. Collins, “The Subtleties and Intricacies of Building a Constraint Length 15 Convolutional Decoder,” IEEE Trans. Comm., vol. 40, no. 12, pp. 1810-1819, Nov. 1992. [COT]T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley & Sons, 1991. [DFKI] Y. Desaki, T. Fujiwara and T. Kasami, “A Method for Computing the Weight Distribution of a Block Code by Using Its Trellis Diagram,” IEICE Tran. Fundamentals, vol. E77-A, pp. 1230-1237, Aug. 1994. [DFK2] Y. Desaki, T. Fujiwara and T. Kasami, “The Weight Distribution of Extended Binary Primitive BCH Codes of Length 128,” IEEE Trans. Info. Theory, vol. 43, no. 4, pp. 13641371, July 1997. [DYS] U. Dettmar, G. Yan and U. K. Sorger, “Modified Generalized Concatenated Codes and Their Application to the Construction and Decoding of LUEP Codes,” IEEE Trans. Info. Theory, ~01.41,no. 5 , pp. 1499-1503, Sept. 1995. [Dho] A. Dholakia, Introduction to Convolutional Codes With Applications, Kluwer, 1994. [DDP] D. Divsalar, S. Dolinar and F. Pollara, “Iterative Turbo Decoder Analysis Based on Density Evolution,” IEEE J. Sel. Areas in Comm., vol. 19, no. 5, pp. 891-907, May 2001. [DJM] D. Divsalar, H. Jin and R.J. McEliece, “Coding Theorems for Turbo-Like Codes,” Proc. 1998 Allerton Con$ on Commurz., Control, and Computing, pp. 210-219, Univ. Illinois; Urbana-Champaign, 1998. [Div] D. Divsalar and F. Pollara, “Turbo Codes for Deep-Space Communications,” JPL TDA REFERENCES 197 Progress Report 42-120, pp. 29-39, Feb. 1995. [DP] D. Divsalar and F. Pollara, “Turbo Codes for PCS Applications,” Proc. 1993 IEEE Int. Con$ Comm. (ICC’93),pp. 1064-1070, Geneva, Switzerland,May 1993. [DVB] EN 310 210, ETSI, “Digital Video Broadcasting (DVB): Framing Structure, Channel Coding and Modulation for Digital Satellite News Gathering (DSNG) and Other Contribution Applications by Satellite,” Mar. 1997. [EH] H. El Gama1 and A. R. Hammons, Jr., “Analyzing the Turbo Decoder Usingthe Gaussian Approximation,” IEEE Trans. Info. Theory, vol. 47, no. 2, pp. 671-686, Feb. 2001. [Elil] P. Elias, “Error-FreeCoding,” IRE Trans., vol. PGIT-4, pp. 29-37, 1954. [Eli21 P. Elias, “Coding for NoisyChannels,” IRE Conv. Rec., vol. 3, pt. 4, pp. 37-46, 1955. Also in E.R. Berlekamp, ed., Key Papers in the Development of Coding Theory, pp. 48-55, IEEE Press, 1974. [Eli31 P. Elias, “Error-Correcting Codes forList Decoding,” IEEE Trans. Info. Theory, vol. 37, no. 1, pp. 5-12, Jan. 1991. [ W ] W. Feng and B. Vucetic, “A List Bidirectional Soft Output Decoderof Turbo Codes,” Proc. Int. Symp. Turbo Codes and Rel. Top.,pp. 288-292, Brest, France, Sept. 3-5, 1997. [Forl] G. D. Forney, Jr., Concatenated Codes, MIT Press Reseach Monograph 37,1966. [For21 G,D, Forney, Jr., “On Decoding BCH Codes,” IEEE Trans. Info. Theory, vol. IT-l 1 , pp. 393-403, Oct. 1965. Also in E.R. Berlekamp, ed., Key Papers in the Development of Coding Theory, pp. 149-155, IEEE Press,1974. [For31 G. D. Forney, “Generalized Minimum Distance Decoding,” IEEE Trans. Info. Theory, vol. IT-12, pp. 125-131, April 1966. [For41 G. D. Forney, “Convolutional Codes I: Algebraic Structure,” IEEE Trans. Info. Theory, vol. IT-16, no. 6, pp. 720-738, Nov. 1970. [Fors] G. D. Forney, Jr., “Burst Correcting Codes forthe Classic Bursty Channel,” IEEE Trans. Comm. Tech., vol. COM-19, pp. 772-281, 1971. [For61 G. D.Forney, Jr., “Coset Codes 11: Binary Lattices and Related Codes,” IEEE Trans. Info. Theory, vol. 24, no. 5, pp. 1152-1187, Sept. 1988. [For71 G. D. Forney, “Codes on Graphs: Normal Realizations,” IEEE Trans. Info. Theory, vol. 47, no. 2, pp. 520-548, Feb. 2001. [Fors] G. D. Forney andG. Ungerboeck,“Modulation and Coding for Linear Gaussian Channels,” IEEE Trans. Info. Theory, Special Commemorative Issue, vol. 44, no. 6, pp. 23842415, Oct. 1998. [FBLH] M. P. C. Fossorier, F. Burkert, S. Lin and J. Hagenauer, “On the Equivalence Between SOVA and Max-Log-MAP Decodings,” IEEE Comm. Letters, vol. 2, no. 5, pp. 137-139, May 1998. [FLC] M. P. C. Fossorier, S. Lin and D. J.Costello, Jr., “On the Weight Distribution of Terminated Convolutional Codes,” IEEE Trans. Injo. Theory, vol. 45, no. 5, pp. 1646-1648, July 1999. [FLI] M. P. C. Fossorier and S. Lin, “Soft-Decision Decoding on Linear Block Codes Basedon Ordered Statistics,”IEEE Trans. Info. Theory, vol. 41, no. 5, pp. 1379-1396, Sept. 1995. [ E 2 1 M. P. C. Fossorier and S. Lin, “Differential Trellis Decoding of Convolutional Codes,” IEEE Trans. Info. Theory, vol. 46, no. 3, pp. 10461053, May 2000. [FL3] M. P. C. Fossorier andS. Lin, “Some DecomposableCodes: The la zlb zla b z1 Construction,” IEEETrans. Info. Theory, vol. 43, no. 5, pp. 1663-1667, Sept. 1997. [FL4]M. P. C. Fossorierand S. Lin, “Complementary Reliability-Based Soft-Decision Decoding of Linear Block CodesBased on Ordered Statistics,” IEEE Trans. Info. Theory, vol. 42, no. 5, pp. 1667-1672, Sep. 1997. [FL5] M. P. C. Fossorier and S. Lin, “Soft-Input Soft-Output Decodingof Linear Block Codes + + + + 198 REFERENCES Based on Ordered Statistics,” Proc. 1998 IEEE Global Telecomm. Con$ (GLOBECOM’YS)), pp. 2828-2833, Sydney, Australia, Nov. 1998. [FLR] M. P. C. Fossorier, S. Lin and D. Rhee, “Bit-Error Probability for Maximum-Likelihood Decoding of Linear Block Codes and Related Soft-Decision Decoding Methods,” IEEE Trans. Info. Theory, vol. 44, no. 7, pp. 3083-3090, Nov. 1998. [FMH] M. P. C. Fossorier, M. MihaljeviC and H. Imai, “Reduced Complexity Iterative Decoding of Low-Density Parity Check Codes Based on Belief Propagation,” IEEE Trans. Comm., vol. 47, no. 5, pp. 673-680, May 1999. [FK] B. J. Frey and F. R. Kschischang, “Early Detection and Trellis Splicing: ReducedComplexity Iterative Decoding,” IEEE J. Sel. Areas in Comm., vol. 16, no. 2, pp. 153-159, Feb. 1998. [FKKL] T. Fujiwara, T. Kasami, A. Kitai and S. Lin, “On the Undetected Error Probability of Shortened Hamming Codes,” IEEE Trans. Comm., vol. COM-33, pp. 570-574, June 1985. [Gal] R. G. Gallager, “Low-Density Parity-Check Codes,” IRE Trans. Info. Theory, vol. 8, no. I , pp. 21-28, Jan. 1962. [GPB] R. Garello, P. Perleoni and S. Benedetto, “Computing the Free Distance of Turbo Codes and Serially Concatenated Codes with Interleavers: Algorithms and Applications,” IEEE J. Sel. Areas in Comm., vol. 19, no. 5, pp. 800-812, May 2001. [GC]J. vonzur Gathen and J. Gerhard, Modern Computer Algebra,Cambridge University Press, 1999. [Col] M. J. E. Golay, “Notes on Digital Coding” Proc. IRE., vol. 37, p. 657, June 1949. Also in Key Papers in the Development of Coding Theory, E. R. Berlekamp, ed., p. 13, IEEE Press, 1974. [Glc] J. D.GoliC, “Iterative Optimum Symbol-by-Symbol Decoding and Fast Correlation Attacks,” IEEE Trans. Info. Theory, vol. 47, no. 7, pp. 3040-3049, Nov. 2001. [GZ] D. Gorenstein and N. Zierler, “A Class of Error Correcting Codes in p m Symbols,” J. SIAM, vol. 9, pp. 207-214, June 1961. [Cur] V. Guruswami and M. Sudan, “Improved Decoding of Reed-Solomon and AlgebraicGeometry Codes,” IEEE Trans. Info. Theory, vol. 45, no. 6, pp. 1757-1767, Sep. 1999. [Hag] J.Hagenauer, “Rate-Compatible Punctured Convolutional Codes (RCPC Codes) and their Applications,” IEEE Trans. Comm., vol. 36, no. 4, pp. 389-400, April 1988. [HH] J. Hagenauer and P. Hoher, “A Viterbi Algorithm with Soft-Decision Outputs and Its Applications,” Proc. 1989 IEEE Global Telecomm. Con$ (GLOBECOM’89),pp. 47.1.147.1.7, Dallas, Texas, 1989. [HOP] J. Hagenauer, E. Offer and L. Papke, “Iterative Decoding of Binary Block and Convolutional Codes,” IEEE Truns. Info. Theory, vol. 42, no. 2, pp. 429-445, Mar. 1996. [Haw] E. K. Hall and S. G. Wilson, “Stream-Oriented Turbo Codes,” IEEE Trans. Info. Theory, vol. 47, no. 5, pp. 1813-1831, July 2001. [Ham] R.W. Hamming, “Error Detecting and Error Correcting Codes,” Bell Syst. Tech. J., vol. 29, pp. 147-160, 1950. Also in Key Papers in the Development of Coding Theory, E. R. Berlekamp, ed., pp. 9-12, IEEE Press, 1974. [Hay] S. Haykin, Digital Communications, John Wiley and Sons, 1988. [Hew] C.Heegard and S. B. Wicker, Turbo Coding, Kluwer Academic Press, 1999. [HEK] A. P. Hekstra, “An Alternative to Metric Rescaling in Viterbi Decoder,” IEEE Trans. Comm., vol. 37, no. 11, pp. 1220-1222, Nov. 1989. [Her] I. N. Herstein, Topics in Algebra, 2nd ed., John Wiley and Sons, 1975. [HEM] J. Hokfelt, 0. Edfors and T. Maseng, “A Turbo Code Interleaver Design Criterion Based on the Performance of Iterative Decoding,” IEEE Comm. Let., vol. 5, no. 2, pp. 52-54, Feb. 200 1. REFERENCES [HMI B. Honay and G.S. Markarian, Trellis Decoding of Block Codes: A Practical Approach, Kluwer, 1996. [HMF] B. Honary, G.S. Markarian and P. G. Farell, “Generalised Array Codes and Their Trellis Structure,” Elect. Letters, vol. 28, no. 6, pp. 541-542, Mar. 1993. [Hsia] M.Y. Hsiao, “A Class of Optimal Minimum Odd-Weight-Column SEC-DED Codes,” IBM J. Res. and Dev., vol. 14, July 1970. [IH] H. Imai and S. Hirakawa, “A New Multilevel Coding Method Using Error-Correcting Codes,” IEEE Trans. Info. Theory, vol. IT-23, no. 3, pp. 371-377, May 1977. [IFMLI] M. Isaka, M. P. C. Fossorier, R. H. Morelos-Zaragoza, S. Lin and H. Imai, “Multilevel Coded Modulation for Unequal Error Protection and Multistage Decoding. Part 11: Asymmetric Constellations,” IEEE Trans. Comm., vol. 48, no. 5, pp. 774-784, May 2000. [ISTCl] Proceedings of the International Symposium o n Turbo Codes and Related Topics,Brest, France, September 3-5, 1997. [ISTC2] Proceedings of the Second International Symposium on Turbo Codes and Related Topics, Brest, France, September 4-7, 2000. [ITCG] Special Issue on Codes and Graphs and Iterative Decoding Algorithms, IEEE Trans. Info. Theory, vol. 47, no. 2, Feb. 2001. [Jer] M. C. Jeruchim, P. Balaban and K. S. Shanmugan, Simulation of Communication Systems, Plenum Press, 1992. [Joh] R. Johannesson and K. S. Zigangirov, Fundamentals of Convolutional Coding, IEEE Press, 1999. [JSACl] Special Issue onthe turbo principle: from theory to practice. Part I, IEEE J. Sel. Areas in Comm., vol. 19, no. 5, May 2001. [JSAC2] Special Issue on the turbo principle: from theory to practice. Part 11, IEEE J. Sel. Areas in Comm., vol. 19, no. 9, Sept. 2001. [Kam] N. Kamiya, “On Algebraic Soft-Decision Decoding Algorithms for BCH Codes,” IEEE Trans. Info. Theory, vol. 47, no. l , pp. 45-58, Jan. 2001. [KNIH] T. Kaneko, T. Nishijima, H. Inazumi and S. Hirasawa, “An Efficient MaximumLikelihood Decoding Algorithm for Linear Block Codes with Algebraic Decoder,” IEEE Trans. Info. Theory, vol. 40, no. 2, pp. 320-327, Mar. 1994. [Kasl] M. Kasahara, Y. Sugiyama, S. Hirasawa and T. Namekawa, “A New Class of Binary Codes Constructed on the Basis of BCH Codes,” IEEE Trans. Info. Theory, vol. IT-21, pp. 582-585, 1975. [Kas2] M. Kasahara, Y. Sugiyama, S. Hirasawa and T. Namekawa, “New Classes of Binary Codes Constructed on the Basis of Concatenated and Product Codes,” IEEE Trans. Info. Theory, vol. IT-22, no. 4, pp. 462-468, July 1976. [Kasm] T. Kasami, “A Decoding Procedure for Multiple-Error-Correcting Cyclic Codes,” IEEE Trans. Info. Theory, vol. IT-10, pp. 134-138, 1964. [ U P ] T. Kasami, S. Lin and W.W. Peterson, “Polynomial Codes,” IEEE Trans. Info. Theory, vol. IT-14, no. 6, pp. 807-814, Nov. 1968. [KKTFL] T. Kasami, T. Koumoto, T. Takata, T. Fujiwara and S. Lin, “The Least Stringent Sufficient Condition on the Optimality of Suboptimally Decoded Codewords,” Proc. 1995 IEEE Int. Symp. Info. Theory (ISIT’95),p. 470, Whistler, Canada, 1995. [Kaz] P. Kazakov, “Fast Calculation of the Number of Minimum-Weight Words of CRC Codes,” IEEE Trans. Info. Theory, vol. 47, no. 3, pp. 1190-1195, March 2001. [KLF] Y. Kuo, S. Lin and M. P. C. Fossorier, “Low-Density Parity-Check Codes Based on Finite Geometries: A Rediscovery and New Results,” IEEE Trans. Info. Theory, vol. 47, no. 7, pp. 271 1-2736, NOV.2001. [KFL] R. R. Kschischang, B. J. Frey and H.-A. Loeliger, “Factor Graphs and the Sum-Product 199 REFERENCES Algorithm,” IEEE Trans. Info. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001. [KV] R. Koetter and A.Vardy, “Algebraic Soft-Decision Decoding of Reed-Somon Codes,” Proc. 2000 IEEE Int. Symp. Info. Theory (ISIT’OO), p.61, Sorrento, Italy, June 25-30, 2000. [Lee] L. H. C. Lee, Convolutional Coding: Fundamentals and Applications, Artech House, 1997. [LGB] S. Le Goff, A. Glavieux and C. Berrou, “Turbo-Codes and High-Spectral Efficiency Modulation,” Proc. 1994 IEEE Int. Con$ Comm. (ICC’94),pp. 645-649, 1994. [LC] S. Lin and D. J. Costello, Jr., Error Control Coding: Fundamentals and Applications, Prentice-Hall, 1983. [LKFF] S. Lin, T. Kasami, T. Fujiwara and M. Fossorier, Trellises and Trellis-Based Decoding Algorithms for Linear Block Codes, Kluwer Academic Press, 1998. [LYHH] J. Lodge, R. Young, P. Hoeher and J. Hagenauer, “Separable MAP ‘Filters’ for the Decoding of Product and Concatenated Codes,” Proc. 1993 IEEE Int. Con$ Comm. (ICC’93), pp. 1740-1745, May 1993. [Mac] D. J. C. MacKay, “Good Error-Correcting Codes Based on Very Sparse Matrices,” IEEE Trans. Info. Theory, vol. 45, no. 2, pp. 399-432, Mar. 1999. [MN] D. J. C. MacKay and R. M. Neal, “Near Shannon Limit Performance of Low Density Parity Check Codes,” Electronics Letters, vol. 32, pp. 1645-1646, 1996. [MS] F. J. MacWilliams and N. J. A. Sloane, The Theory of Error Correcting Codes, NorthHolland, 1977. [Man] D. Mandelbaum, “Decoding Beyond the Designed Distance of Certain Algebraic Codes,” Info. and Control, vol. 35, pp. 209-228, 1977. [MT] P. A. Martin and D. P. Taylor, “On Adaptive Reduced-Complexity Iterative Decoding,” Proc. 2000 IEEE Global Telecomm. Con$ (GLOBECOM’OO),pp. 772 -776,2000. [Masl] J. L. Massey, Threshold Decoding, MIT Press, 1963. [Mas21 J.L. Massey, “Shift Register Synthesis and BCH Decoding,” IEEE Trans. Info. Theory, vol. IT-15, no. 1, pp. 122-127, Jan. 1969. [Mas31 J. L. Massey, “Coding and Modulation in Digital Communications,” Proc. Int. Zurich Seminar on Dig. Comm., pp. E2( I)-E2(4), Zurich, Switzerland, 1974. [Mas41 J. L. Massey, “The How and Why of Channel Coding,” Proc. Int. Zurich Seminar on Dig. Comm., pp. 67-73, Zurich, Switzerland, 1984. [MW] B. Masnick and J. Wolf, “On Linear Unequal Error Protection Codes,” IEEE Trans. Info. Theory, vol. IT-13, no. 4, pp. 600-607, July 1967. [McE] R. J. McEliece, The Theory oflnformation and Coding, Addison-Wesley, 1977. [MMC] R. J. McEliece, D. J. C. MacKay and J.-F. Cheng, “Turbo Decoding as an Instance of Pearls’ ‘Belief Propagation’ Algorithm,” IEEE J. Sel. Areas in Comm., vol. 16, no. 2, pp. 140-152, Feb. 1998. [Meg] J.E. Meggit, “Error Correcting Codes for Correcting Bursts of Errors,” IBM J. Research Develop., no. 4, pp. 329-334, 1960. [ML] A.M. Michelson and A.H. Levesque, Error-Control Techniquesfor Digital Communications, John Wiley and Sons, 1985. [MG] M. J. MihaljeviC and J. D. GoliC, “A Comparison of Cryptoanalitic Principles Based on Iterative Error Correction,” Advances in Cryptology - EUROCRYPT’91 (Lecture Note in Computer Science), vol. 547, pp. 527-531, Springer-Verlag, 1991. [MFKL]R. H. Morelos-Zaragoza, T. Fujiwara, T. Kasami and S. Lin, “Constructions of Generalized Concatenated Codes and TheirTrellis-Based Decoding Complexity,” IEEE Trans. Info. Theory, vol. 45, no. 2, pp. 125-731, Mar. 1999. [MI]R. H. Morelos-Zaragoza and H. Imai, “Binary Multilevel Convolutional Codes with Unequal Error Protection Capabilities,” IEEE Trans. Comm., vol. 46, no. 7, pp. 850-853, July REFERENCES 1998. [MM] R. H. Morelos-Zaragoza and A. Mogre, “A Two-Stage Decoder for Pragmatic TrellisCoded M-PSK Modulation Using a Symbol Transformation,” IEEE Trans. Comm., vol. 49, no. 9, pp. 1501-1505, Sept. 2001. [MFLI] R. H. Morelos-Zaragoza, M. P. C. Fossorier, S. Lin and H. Imai, “Multilevel Coded Modulation for Unequal Error Protection and Multistage Decoding. Part I: Symmetric Constellations,” IEEE Trans. Comm.,vol. 48, no. 2, pp. 204-213, Feb. 2000. [Mor] J. M. Morris, “Burst Error Statistics of Simulated Viterbi Decoded BPSK on Fading and Scintillating Channels,” IEEE Trans. Comm., vol. 40, no. 1, pp. 34-41, Jan. 1992. [OCC] I. M. Onyszchuk, K.-M. Cheung and 0. Collins, “Quantization Loss in Convolutional Decoding,” IEEE Trans. Comm.,vol. 41, no. 2, pp. 261-265, Feb. 1993. [Prl] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, 1988. [PSC] L. C. Perez, J. Seghers and D. J. Costello, Jr., “A Distance Spectrum Interpretation of Turbo Codes,” IEEE Trans. Info. Theory, vol. 42, no. 6, pp. 1698-1709, Nov. 1996. [PW] W. W. Peterson and E. J. Weldon, Jr., Error Correcting Codes,2nd ed., MIT Press, 1972. [Pet] W.W. Peterson, “Encoding and Error-Correction Procedures for the Bode-Chaudhuri Codes,” IRE Trans. Info. Theory, vol. IT-6, pp. 459-470, Sept. 1960. Also in E.R. Berlekamp, ed., Key Papers in the Developmentof Coding Theory,pp. 109-120, IEEE Press, 1974. [PP] A. Picart and R. M. Pyndiah, “Adapted Iterative Decoding of Product Codes,” Proc. 1995 IEEE Global Telecomm. Con$ (GLOBECOM’YS),pp. 2357-2362, 1995. [Plo] M. Plotkin, “Binary Codes with Specified Minimum Distances,” IEEE Trans.Info. Theory, vol. IT-6, pp. 445-450, 1960. [Pre] 0. Pretzel, Codes and Algebraic Curves, Oxford Lecture Series in Mathematics and Its Applications, 8, 1998. [Pro] J. G. Proakis, Digital Communications, 3rd ed., McGraw-Hill, 1995. [PS] M.B. Pursley and J.M. Shea, “Bit-by-Bit Soft-Decision Decoding of Trellis-Coded M DPSK Modulation,” IEEE Comm. Letters, vol. I , no. 5, pp. 133-135, Sept. 1997. [Pyn] R. M. Pyndiah, “Near-Optimum Decoding of Product Codes: Block Turbo Codes,” IEEE Trans. Comm., vol. 46, no. 8, pp. 1003-1010, Aug. 1998. [PGPJ] R.M. Pyndiah, A. Glavieux, A. Picart and S. Jacq, “Near Optimum Decoding of Product Codes,” Proc. 1994 IEEE GlobalTelecornm. Con$ (GLOBECOM’94),vol. 1, pp. 339-343, San Francisco, CA, Dec. 1994. [Ram] J. L. Ramsey, “Realization of Optimum Interleavers,” IEEE Trans. Info. Theory, vol. IT16, no. 3, pp. 338-345, May 1970. [RR] S. M. Reddy and J. P. Robinson, “Random Error and Burst Correction by Iterated Codes,” IEEE Trans. Info. Theory, vol. IT-18, no. 1, pp. 182-185, Jan. 1972. [RC] I. S. Reed and X. Chen, Error-Control Coding for Data Networks,Kluwer Academic Press, 1999. [RS] I. Reed and G. Solomon, “Polynomial Codes over Certain Finite Fields,” SIAM J. Appl. Math., Vol. 8, pp. 300-304, 1960. Also in E.R. Berlekamp, ed., Key Papers in the Development of Coding Theory,pp. 70-71, IEEE Press, 1974. [RSU] T. J. Richardson, M. A. Shokrollahi and R. L. Urbanke, “Design of CapacityApproaching Irregular Low-Density Parity-Check Codes,” IEEE Trans.Info. Theory, vol. 47, no. 2, pp. 619-637, Fe. 2001. [RU] T. J. Richardson and R. L. Urbanke, “The Capacity of Low-Density Parity-Check Codes Under Message-Passing Decoding,” IEEE Trans. Info. Theory, vol. 47, no. 2, pp. 599-618, Feb. 2001. [Rob] P. Robertson, “Improving Decoder and Code Structure of Parallel Concatenated Recursive 20 1 202 REFERENCES Systematic (Turbo) Codes,” Proc. IEEE Int. Con$ Univ. Per: Comm. (ICUPC’94),pp. 183-187, 1994. [RVH] P. Robertson, E. Villebmn and P. Hoeher, “A Comparison of Optimal and Sub-optimal MAP Decoding Algorithms Operating in the Log Domain,” Proc. 1995 IEEE Int. Con$ Comm. (ICC’95),pp. 1009-1013, 1995. [RWl] P. Robertson and T. Worz, “Coded Modulation Scheme Employing Turbo Codes,” Electronics Letters, vol. 31, pp. 1546-1547, Aug. 1995. [RW2] P. Robertson and T. Worz, “Bandwidth-Efficient Turbo Trellis-Coded Modulation Using Punctured Component Codes,” IEEE J. Sel. Areas in Comm., vol. 16, no. 2, pp. 206-218, Feb. 1998. [SSSN] H. R. Sdajadpour, N. J. A. Sloane, M. Salehi and G. Nebe, “Interleaver Design for Turbo Codes,” IEEE J. Sel. Areas in Comm., vol. 19, no. 5, pp. 831-837, May 2001. [SB] G. Schnabl and M. Bossert, “Soft-Decision Decoding of Reed-Muller Codes as Generalized Multiple Concatenated Codes,” IEEE Trans. Info. Theory, vol. 41, no. 1 , pp. 304308, Jan. 1995. [Schr] P. Schramm, “Multilevel Coding with Independent Decoding on Levels for Efficient Communication on Static and Interleaved Fading Channels,” Proc. IEEE 1997 Int. Symp. Per:, Ind. and Mob. Radio Comm. (PIMRC’97),pp. 1196-1200, 1997. [Sha] C. Shannon, “A Mathematical Theory of Communication,” Bell Syst. Tech. J., vol. 27, pp. 379-423 and 623-656, 1948. Also in Key Papers in the Development of Information Theory, D. Slepian, ed., pp. 5-29, IEEE Press, 1974. [SLF]R.Shao, S. Lin and M. P. C. Fossorier, “Two Simple Stopping Criteria for Turbo Decoding,” IEEE Trans. Comm., vol. 47, no. 8, pp. 1117-1120, Aug. 1999. [Sin] R.C. Singleton, “Maximum Distance q-nary Codes,” IEEE Trans. Info. Theory, vol. IT-IO, pp. 116-118, 1964. [SS] M. Sipser and D. A. Spielman, “Expander Codes,” IEEE Trans. Info. Theory, vol. 42, no. 6, pp. 1710-1722, NOV.1996. [Sle] D. Slepian, “A Class of Binary Signaling Alphabets,” Bell. Sys. Tech. J., vol. 35, pp. 203234, Jan. 1956. [Slo] N. J. A. Sloane, S. M. Reddy and C.-L. Chen, “New Binary Codes,” IEEE Trans. Info. Theory, vol. IT-18, no. 4, pp. 503-510, July 1972. [Sud] M. Sudan, “Decoding of Reed-Solomon Codes Beyond the Error-Correction Bound,” J. Complexi@,vol. 12, pp. 180-193, Dec. 1997. [SKHN] Y. Sugiyama, Y. Kasahara, S. Hirasawa and T. Namekawa, “A Method for Solving Key Equation for Goppa Codes,” Info. and Control, vol. 27, pp. 87-99, 1975. [Sug] Y. Sugiyama, M. Kasahara, S. Hirasawa and T. Namekawa, “Further Results on Goppa Codes and Their Applications to Constructing Efficient Binary Codes,” IEEE Trans. Info. Theory, vol. IT-22, pp. 518-526, 1978. [TP] D. J. Taipale and M. B. Pursley, “An Improvement to Generalized Minimum Distance Decoding,” IEEE Trans. Info. Theory, vol. 37, no. 1, pp. 167-172, Jan. 1991. [TYFKL] T. Takata, Y. Yamashita, T. Fujiwara, T. Kasami and S. Lin, “On a suboptimum decoding of decomposable block codes,” IEEE Trans. Info. Theory, vol. 40, no. 5, pp. 13921406, Sept. 1994. [TC] 0. Y. Takeshita and D. J. Costello, Jr., “New Deterministic Interleaver Designs for Turbo Codes,” IEEE Trans. Info. Theory, vol. 46, no. 6, pp. 1988-2006, Sept. 2000. [TLFL] H. Tang, Y. Liu, M. P. C. Fossorier and S. Lin, “On Combining Chase-2 and GMD Decoding Algorithms for Nonbinary Block Codes,” IEEE Cornm. Letters, vol. 5, no. 5, pp. 209-2 1 1, May 200 1. [Tan]R. M. Tanner, “A Recursive Approach to Low Complexity Codes,” IEEE Trans. Info. REFERENCES Theory, vol. IT-27, no. 5, pp. 533-547, Sept. 1981. [tBrl] S. ten Brink, “Convergence of iterative decoding,” Electronics Letters, vol. 35, pp. 806808, May 1999. [tBr2] S. ten Brink , “Convergence Behaviour of Iteratively Decoded Parallel Concatenated Codes,” submitted to IEEE Trans. Comm., Mar. 2000. [TKK] H. Tokushige, T. Koumoto and T. Kasami, “An Improvement to GMD-like Decoding Algorithms,” Proc. 2000 IEEE Int. Symp. Info. Theory (ISIT’OO),p. 396, Sorrento, Italy, 2000. [Ungl] G. Ungerboeck, “Channel Coding with MultilevelPhase Signals,” IEEE Trans. Info. Theory, vol. IT-28, no. 1 , pp. 55-67, Jan. 1982. [Ung2] G. Ungerboeck, “Trellis-Coded Modulation with Redundant Signal Sets I: Introduction,” IEEE Cornm. Mag., vol. 25, no. 2, pp. 5-1 1, Feb. 1987. [Ung3] G. Ungerboeck, “Trellis-Coded Modulation with Redundant Signal Sets 11: State of the Art,” IEEE Comm. Mag., vol. 25, no. 2, pp. 12-21, Feb. 1987. [Van] W.J. van Gils, “Two Topics on Linear Unequal Error Protection Codes: Bounds on Their Length and Cyclic Code Classes,” IEEE Trans. Info. Theory, vol. IT-29, no. 6, pp. 866-876, Nov. 1983. [VO] S. A. Vanstone and P. C. van Oorschot, An Introduction to Error Correcting Codes with Applications, Kluwer Academic Publishers, 1989. [Vitl] A. J. Viterbi, “Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm,” IEEE Truns. Info. Theoly, vol. IT-13, pp. 260-269, April 1967. [Vit2] A. J. Viterbi, “Convolutional Codes and Their Performance in Communication Systems,” IEEE Trans. Comm., vol. COM-19, no. 4, pp. 751-772, 1971. [Vid] A. J. Viterbi, “An Intuitive Justification and a Simplified Implementation of the MAP Decoder for Convolutional Codes,” IEEE J. Sel. Areas in Cornm., vol. 16, no. 2, pp. 260-264, Feb. 1998. [Vit4] A.J. Viterbi, J.K. Wolf, E. Zehavi and R. Padovani, “A Pragmatic Approach to TrellisCoded Modulation,” IEEE Comm. Mag., pp. 11-19, July 1989. [ViOm] A. J. Viterbi and J. K. Omura, Principles of Digital Communication and Coding, McGraw-Hill, 1979. [VUC]B. Vucetic and J. Yuan, Turbo Codes: Principles and Applications, Kluwer Academic, 2000. [WFH] U. Wachsmann, R. F. H. Fischer and J. B. Huber, “Multilevel Codes: Theoretical Concepts and Practical Design Rules,” IEEE Trans. Info. Theory, vol. 45, no. 5 , pp. 13611391, July 1999. [ W ] B. E. Wahlen and C. Y. Mai, “Turbo Coding Applied to Pragamtic Trellis-Coded Modulation,” IEEE Comm. Letters, vol. 4, no. 2, pp. 65-67, Feb. 2000. [Well E. J. Weldon, Jr., “Decoding Binary Block Codes on Q-ary Output Channels,” IEEE Trans. Info. Theory, vol. IT-17, no. 6, pp. 713-718, Nov. 1971. [Wib] N. Wiberg, “Codes and Decoding on General Graphs,” Ph.D. dissertation, Dept. Elect. Eng., Linkoping Univ., 1996. [Wic] S. B. Wicker, Error Control Systems for Digital Communication and Storage, PrenticeHall, 1995. [WB] S.B. Wicker and V.K. Bhargava, Reed-Solomon Codes and TheirApplications, IEEE Press, 1994. [Will S. G. Wilson, Digital Modulation and Coding, Prentice-Hall, 1996. [Wol] J. K. Wolf, “Efficient Maximum-Likelihood Decoding of Linear Block Codes Using a Trellis,” IEEE Trans. Info. Theory, vol. IT-24, no. 1, pp. 76-80, Jan. 1978. [WV] J.K. Wolf and A. J. Viterbi, “On the Weight Distribution of Linear Block Codes formed From Convolutional Codes,” IEEE Trans. Comm., vol. 44, no. 9, pp. 1049-1051, Sep. 1996. 203 204 REFERENCES [WJ] J. M. Wozencraft and 1. M. Jacobs, Principles of Communication Engineering, John Wiley and Sons, 1965. [YI] K. Yamaguchi and H. Imai, “A New Block Coded Modulation Scheme and Its Soft Decision Decoding,” Proc. 1993 IEEE Intel: Symp. Info. Theory (ISIT’93),p. 6 4 , 1993. [YKH] Y. Yasuda, K. Kashiki and Y. Hirata, “High-Rate Punctured Convolutional Codes for Soft Decision Viterbi Decoding,” IEEE Trans. Comm., vol. COM-32, no. 3, pp. 325-328, March 1984. [ZW]E. Zehavi and J.K. Wolf, “P2 Codes: Pragmatic Trellis Codes Utilizing Punctured Convolutional Codes,” IEEE Communications Magazine, pp. 94-99, Feb. 1995. [ZP] V. V. Zyablov and M. S. Pinsker, “Estimation of the Error-Correction Complexity for Gallager Low-Density Codes,” Prob. Pered. Inform., vol. 11, no. 1, pp. 26-36, 1975. [Zin] V. A. Zinov’ev, “Generalized Cascade Codes,” Probl. Pered. Infonnatsii, vol. 12, no. 1, pp. 5-15, 1976. The Art of Error Correcting Coding Robert H. Morelos-Zaragoza Copyright © 2002 John Wiley & Sons Ltd ISBNs: 0-471-49581-6 (Hardback); 0-470-84782-4 (Electronic) Appendix A Weight distributions of extended BCH codes In this appendix, the weight distributions of all extended BCH codes of length up to 128 are presented. The first row of a table indicates the parameters of the code "n,k,d" (this is also the name of a file containing the weight distribution in the ECC web site.) Subsequent rows of a table list the weight W and the number of codewords of this weight A,. These codes are symmetric, in the sense that the relation A , = A,-,, for 0 only half of the distribution is listed. A.l Length 8 wd.8.1.8 8 1 wd.8.4.4 4 14 8 1 wd.8.7.2 2 28 4 70 A.2 Length 16 wd.16.05.08 8 30 wd.16.07.06 6 48 8 30 wd.16.11.04 4 140 5 W 5 n/2, holds. Consequently, 206 6448 8 870 wd.16.15.02 2120 41820 68008 812870 A.3 Length 32 wd.32.06.16 16 62 wd.32.11.12 12 496 16 1054 wd.32.16.08 8 620 12 13888 16 36518 wd.32.21.06 6992 810540 10 60512 12 228160 14 446400 16 603942 wd.32.26.04 41240 627776 8330460 10 2011776 12 7063784 14 14721280 16 18796230 wd.32.31.02 2 496 4 35960 6906192 810518300 10 64512240 12 225792840 14 471435600 16 601080390 THE ART OF ERROR CORRECTING CODING WEIGHT DISTRIBUTIONS OF EXTENDED BCH CODES A.4 Length 64 wd.64.07.32 32 126 wd.64.10.28 28 448 32 126 wd.64.16.24 24 5040 28 12544 32 30366 wd.64.18.22 22 4224 24 5040 26 24192 28 12544 30 69888 32 30366 wd.64.24.16 16 2604 18 10752 22 216576 24 291648 26 1645056 28 888832 30 4419072 32 1828134 wd.64.30.14 14 8064 16 30828 18 631680 20 1128960 22 14022144 24 14629440 26 105057792 28 65046016 30 282933504 32 106764966 wd.64.36.12 12 30240 14 354816 16 3583020 18 27105792 20 145061280 22 603113472 207 THE ART OF ERROR CORRECTING CODING 208 24 26 28 30 32 1853011776 4517259264 8269968448 12166253568 13547993382 wd.64.39.10 10 13888 12 172704 14 2874816 16 29210412 18 214597824 20 1168181280 22 4794749760 24 14924626752 26 35889146496 28 66620912960 30 96671788416 32 109123263270 wd.64.45.08 827288 10 501760 12 12738432 14 182458368 16 1862977116 18 13739292672 20 74852604288 22 306460084224 24 956270217000 26 2294484111360 28 4268285380352 30 6180152832000 32 6991765639110 wd.64.51.06 620160 8 1067544 10 37051840 12 801494400 14 11684617344 16 119266575708 18 879321948288 20 4789977429888 22 19616032446528 24 61193769988008 26 146864398476096 28 273137809339136 30 395577405119232 32 447418802536902 WEIGHT DISTRIBUTIONS OF EXTENDED BCH CODES wd.64.57.04 410416 61166592 869194232 10 2366570752 12 51316746768 14 747741998592 16 7633243745820 18 56276359749120 20 306558278858160 22 1255428754917120 24 3916392495228360 26 9399341113166592 28 17480786291963792 30 25316999607653376 32 28634752793916486 wd.64.63.02 2 2016 4635376 674974368 84426165368 10 151473214816 12 3284214703056 14 47855699958816 16 488526937079580 18 3601688791018080 20 19619725782651116 22 80347448443237936 24 250649105469666110 26 601557853127198720 28 1118770292985240200 30 1620288010530347000 32 1832624140942591500 A S Length 128 wd.128.008.064 64254 wd.128.015.056 56 8128 64 16510 wd.128.022.048 48 42672 56 877824 64 2353310 wd.128.029.044 209 210 44 48 52 56 60 64 THE ART OF ERROR CORRECTING CODING 373888 2546096 16044672 56408320 116750592 152623774 wd.128.036.032 32 10668 36 16256 40 2048256 44 35551872 48 353494848 52 2028114816 56 7216135936 60 14981968512 64 19484794406 wd.128.043.032 32 124460 36 8810752 40 263542272 44 4521151232 48 44899876672 52 262118734080 56 915924097536 60 1931974003456 64 2476672341286 wd.128.050.028 28 186944 32 19412204 36 113839296 40 33723852288 44 579267441920 48 5744521082944 52 33558415333632 56 117224663972352 60 247312085243776 64 31699236111910 wd.128.057.024 24 597408 28 24579072 32 2437776684 36 141621881856 40 4315318568736 44 74150180302848 48 73528925007168 52 4295496356229120 56 15004724612905792 WEIGHT DISTRIBUTIONS OF EXTENDED BCH CODES 6 03 1 6 5 5 9 9 1 6 2 1 4 4 5 6 3 2 6 44 5 7 4 9 6 5 3 1 7 2 6 7 2 3 8 wd.128.064.022 2 22 4 3 8 4 0 2 46 8 5 5 9 6 8 2 61 0 7 9 8 8 6 0 8 2 81 4 7 9 7 5 1 1 6 8 3 01 6 5 8 1 2 1 7 5 3 6 3 21 6 1 4 7 1 8 8 2 7 9 6 3 41 2 9 2 2 4 1 2 9 6 6 4 0 3 69 1 0 6 5 1 6 3 2 9 9 8 4 3 85 3 3 8 3 2 7 9 3 0 7 9 0 4 4 02 7 8 4 2 0 6 9 0 1 6 1 8 2 4 4 21 2 1 8 6 6 6 8 4 7 7 2 5 1 8 4 4 44 7 8 2 6 3 0 1 9 1 8 2 2 8 4 8 4 61 5 8 5 8 7 0 5 6 0 0 5 9 6 9 9 2 4 84 7 4 2 5 6 8 4 1 6 1 3 2 6 9 1 2 5 01 2 0 4 4 2 1 8 5 1 4 7 4 9 3 3 7 6 5 22 7 7 0 6 1 6 3 4 6 5 4 0 9 9 4 5 6 5 45 4 3 2 4 4 8 6 2 5 0 5 7 7 5 3 6 0 5 69 6 7 7 9 9 7 2 1 8 5 7 1 3 5 1 6 8 5 81 4 7 3 2 8 7 4 7 8 1 8 9 7 3 5 1 6 8 6 02 0 4 1 8 1 9 5 1 1 3 0 8 5 3 0 6 8 8 6 22 4 2 1 5 5 0 6 3 0 9 0 7 0 4 3 3 2 8 6 42 6 1 7 0 7 5 8 8 6 2 1 6 9 1 0 1 1 8 wd.128.071.020 2 02 6 7 4 1 1 2 2 23 7 4 8 6 3 3 6 2 48 3 9 6 9 9 6 1 6 2 61 3 8 2 5 0 4 5 2 4 8 2 81 8 8 0 0 1 3 4 7 1 3 6 3 02 1 4 0 0 9 5 1 8 2 3 3 6 3 22 0 5 1 0 6 9 7 9 2 7 4 6 8 3 41 6 6 6 8 9 9 8 0 4 3 8 0 1 6 3 61 1 5 6 6 5 8 6 6 1 4 7 1 0 4 0 3 86 8 8 6 4 9 7 2 0 9 9 3 5 6 1 6 4 03 5 3 6 3 7 7 6 2 2 0 1 9 5 3 6 0 4 21 5 7 2 0 7 7 9 8 7 7 3 1 2 9 9 8 4 4 46 0 7 4 6 8 1 6 3 0 6 7 9 9 4 3 0 4 4 62 0 4 5 7 7 3 6 7 9 0 6 8 6 8 6 3 3 6 4 86 0 2 3 7 9 6 9 5 4 7 7 8 0 1 2 4 8 0 5 01 5 5 3 7 0 4 0 5 1 6 5 4 8 1 2 6 7 2 0 5 23 5 1 9 1 1 2 4 1 1 4 6 3 3 0 0 6 4 6 4 5 47 0 0 7 8 5 8 9 2 6 9 1 5 6 9 6 9 9 8 4 5 61 2 2 9 2 5 5 6 6 9 5 2 0 8 8 6 6 0 2 8 8 5 81 9 0 0 5 4 0 8 2 7 5 8 9 5 6 1 0 7 2 6 4 6 02 5 9 3 4 2 7 3 7 9 0 2 8 4 0 3 5 5 4 5 6 6 23 1 2 3 8 0 0 3 2 1 9 8 0 3 5 5 7 9 9 0 4 6 43 3 2 4 0 9 2 0 7 8 6 7 7 8 6 5 4 3 9 1 0 211 212 wd.128.078.016 16 387096 18 5462016 20 213018624 22 539859840 24 107350803840 26 1766071867392 28 24074650400768 30 273932927993856 32 2625267567169884 34 2133648518951040 36 14805286631892608 38 881470039149213696 40 4526561735332554624 42 20122606565844068352 44 77755925658495682560 46 261859003134276581376 48 771046023044966543784 50 1988741249124011372544 52 4504463828911859699712 54 8970059328813665832960 56 15734472710169831412480 58 24326922690137187741696 60 3319587221944924483584 62 39984644079892337086464 64 42548378876302513514950 wd.128.085.014 14 341376 16 22121368 18 856967552 20 27230880768 22 680417833472 24 13721772977024 26 226128254847488 28 3081454360189952 30 35064826913355520 32 336014520825141340 34 2731238665152128768 36 1894961228051341184 38 112834993226032103936 40 579364846705294996864 42 2575849616631486204416 44 9952155728071153882112 46 33519982404512223401600 48 98687914666573428364840 50 254574296248800159922816 52 576536456040619165149184 54 1148237129819878789497856 56 213890548891825020657408 THE ART OF ERROR CORRECTING CODING WEIGHT DISTRIBUTIONS OF EXTENDED BCH CODES 5 83 1 1 4 0 3 4 6 8 4 7 4 2 7 1 5 3 9 3 8 1 5 5 5 2 6 04 2 4 8 8 1 4 0 8 8 0 2 0 5 3 0 7 9 0 4 2 2 5 2 8 6 25 1 1 8 3 4 4 4 0 8 7 4 9 4 9 2 8 9 8 4 1 1 5 2 6 45 4 4 5 8 6 2 7 0 3 3 7 3 4 4 4 5 1 7 8 2 5 4 7 8 wd.128.092.012 1 21 1 9 4 8 1 6 1 44 5 6 4 6 8 4 8 1 62 7 5 1 6 8 2 5 8 4 18 1 1 0 0 7 1 4 5 6 7 6 8 20 3484410778688 2 28 7 0 9 9 3 9 3 5 5 0 0 8 2 41 7 5 6 3 5 9 9 1 7 1 6 5 9 5 2 2 62 8 9 4 4 4 5 0 6 5 6 1 2 0 8 3 2 2 83 9 4 4 2 6 3 8 9 9 8 8 2 3 7 1 8 4 3 04 4 8 8 2 9 7 7 2 7 6 6 3 1 7 1 5 8 4 3 24 3 0 0 9 8 4 2 7 1 5 8 9 6 6 9 3 0 8 4 3 43 4 9 5 9 8 7 1 7 5 7 8 5 8 7 5 3 1 2 6 4 3 62 4 2 5 5 4 9 1 8 9 8 7 2 5 9 7 6 7 8 9 7 6 3 81 4 4 4 2 8 8 6 0 2 8 0 6 7 6 3 9 7 8 3 4 2 4 4 07 4 1 5 8 6 6 5 3 2 0 6 0 4 1 5 5 8 0 4 1 6 4 23 2 9 7 0 8 9 0 6 6 3 5 0 4 8 7 8 4 7 6 9 0 2 4 4 41 2 7 3 8 7 5 3 3 8 6 2 7 2 5 4 5 5 9 0 9 7 6 4 64 2 9 0 5 5 9 7 7 8 0 0 9 1 3 2 1 9 7 7 6 4 0 9 6 4 81 2 6 3 2 0 4 7 0 9 9 6 1 9 8 1 8 7 5 1 6 3 9 9 7 6 5 03 2 5 8 5 5 2 5 3 3 7 3 0 7 0 3 6 5 9 1 2 9 1 3 9 2 5 27 3 7 9 6 6 3 1 4 6 9 2 4 3 2 7 7 6 1 5 1 1 1 0 4 5 41 4 6 9 7 4 4 2 2 1 4 8 8 6 6 5 1 4 2 4 3 0 8 4 2 8 8 5 62 5 7 7 7 7 8 6 8 3 0 6 8 0 0 2 3 6 9 3 2 4 7 2 3 2 5 83 9 8 5 9 6 6 2 8 2 3 2 7 2 5 8 3 1 8 9 5 2 3 7 1 2 6 05 4 3 8 4 7 9 4 5 9 6 1 2 3 3 3 9 3 4 7 2 6 5 4 5 9 2 6 26 5 5 1 4 8 3 9 3 2 6 8 0 7 5 6 5 8 8 7 2 2 3 8 0 8 0 6 46 9 7 7 0 0 9 6 2 4 6 4 1 3 1 4 9 1 4 5 7 1 3 0 9 4 wd.128.099.010 1 07 9 6 5 4 4 1 29 0 1 8 0 1 6 0 1 46 4 6 3 8 8 9 5 3 6 1 63 4 7 7 6 4 5 3 9 9 2 8 18 1 4 1 2 7 5 5 9 5 7 3 1 2 0 2 04 4 5 7 5 4 7 5 4 6 9 2 4 8 2 21 1 1 4 9 6 8 5 2 6 5 4 6 7 7 7 6 2 42 2 4 8 1 1 6 9 0 6 2 7 7 1 2 3 8 4 2 63 7 0 4 8 9 5 3 7 7 8 2 1 9 1 1 0 4 2 85 0 4 8 6 5 5 6 1 7 3 1 2 1 6 7 3 6 0 0 3 05 7 4 5 0 2 1 7 6 7 3 0 5 7 1 2 5 5 5 5 2 3 25 5 0 5 2 5 9 7 8 6 9 4 4 6 7 9 9 9 0 6 2 0 3 44 4 7 4 8 6 3 5 7 2 0 2 7 3 3 8 3 1 4 3 1 6 8 3 63 1 0 4 7 0 2 9 6 2 7 9 9 9 4 3 9 2 9 7 5 3 6 3 81 8 4 8 6 8 9 4 1 7 3 0 1 3 4 9 2 4 7 8 9 9 9 0 4 4 09 4 9 2 3 0 9 1 2 3 7 3 1 9 1 1 8 5 1 5 6 6 9 7 6 213 THE ART OF ERROR CORRECTINGCODING 214 42 44 46 48 50 52 54 56 58 60 62 64 42202740212894624045103744 16356041742389991882232512 549191653602919908961484160 1616902022803263350264149928 4170947258582865019960480640 9445968792041391795950926784 1881272610465984668145312896 3299556702053516278222434304 5102036860227828704471599232 6961253682581943211726121216 83858994648317780352552315392 89224971989631194512677986758 wd.128.106.008 8 774192 10 105598976 12 11361676032 14 828626841600 16 44515013174520 18 1808265733435392 20 57056968214853376 22 1427159096213901312 24 28775892186952836240 26 474226642406696116224 28 6462279071735110418944 30 73536278816433772929024 32 704673252880779235687452 34 5727825370458099461038080 36 39740197928028063063904768 38 236632245414203838081949696 40 1215015567801313175697152304 42 5401950747433627456981266432 44 20871173342366872859566014720 46 70296531663247684816378728448 48 206963458912891026277198168776 50 533881249113840797115223461888 52 1209084005346591905941683436800 54 2408028941464856710061855682560 56 4223432578506218555128558121488 58 6530607181280659655017851666432 60 8910404713446325250255943109632 62 10733951315294301174491841282048 64 11420796414343588424136158689350 wd.128.113.006 6 341376 8 87288624 10 13842455424 12 1448180487936 14 106141978256640 16 5697211389035256 WEIGHT DISTRIBUTIONS OF EXTENDED BCH CODES 18 20 23 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 231462916338818304 7303265469631124224 182676478544670888576 3683313800412335283600 60701011366229993420928 827171718544587565763072 9412643693444880033139200 90198176361260636112668700 733161647428265153721100800 5086745334774298795535505920 30288927413044862466125137280 155521992678499175905941507120 691449695671693087458634462080 2671510187822394963671294035200 8997956052897506127123622265600 26491322740844548554720461440200 68336799886586511592317937195776 154762752684328935230921657151744 308227704507572032682000507510400 540599370048672363242473684364048 835917719204114526888908154932352 1140531803320872201314876100099072 1373945768357978846215074177297408 1461861941035652013200273232486470 wd.128.120.004 4 85344 6 42330624 8 11170182384 10 1772228014592 12 185359804775712 14 13586256544975872 16 729242357526446712 18 29627257927486958592 20 934817955092922629344 22 23382589365749366429184 24 471464166034059302122704 26 7769729456174562056216064 28 105877979970476869275385504 30 1204818392766796825789470720 32 11545366574237052418777217820 34 93844690870798540052434360320 36 651103402851220082586931517920 38 3876982708869397190103809681920 40 19906815062848699462140058602480 42 88505561045975275152200314606080 44 341953304041268345847846829061280 46 1151738374770880217441839661716480 48 3390889310828097487679807613566280 50 8747110385483091255323050018747392 52 19809632343594061105640384790579552 215 216 54 56 58 60 62 64 THE ART OF CORRECTING ERROR CODING 39453146176969302060067713110615552 69196719366229927819863672056678992 106997468058126854441956301420662272 145988070825071389633119654266397216 175865058349821585715411392357912576 187118328452563149209991044344449600 wd.128.127.002 2 8128 4 10668000 6 5423611200 8 1429702652400 10 226846154180800 12 23726045489546400 14 1739040916651367936 16 93343021201262198784 18 3792289018215984005120 20 119656698232656988471296 22 2992971438910354793431040 24 60347413251942500075044864 26 994525370392012056264966144 28 13552381436214964486037045248 30 154216754274170157987417554944 32 1477806921502279921677734248448 34 12012120431462387730048509542400 36 83341235564955678220181980577792 38 496253786735284008587127926292480 40 2548072328044630786408938247028736 42 11328711813884834832046711017308160 44 43770022917282358845017689455329280 46 147422511970672750843485296833593344 48 434033831785996763487994252512722944 50 1119630129341835894935101449793699840 52 2535632939980039741724790850645393408 54 5050002710652071717329822398986321920 56 8857180078877432500571052147864502272 58 13695675911440237013532474696584396800 60 18686473065609143965712255676040871936 62 22510727468777167143671172081479843840 64 23951146041928103937688710428982509568