JOURNAL OF COMPLEXITY ARTICLE NO. 13, 180–193 (1997) CM970439 Decoding of Reed Solomon Codes beyond the ErrorCorrection Bound Madhu Sudan * IBM Thomas J. Watson Research Center, P.O. Box 218, Yorktown Heights, New York 10598 Received August 31, 1996 We present a randomized algorithm which takes as input n distinct points ) from F F (where F is a field) and integer parameters t and d and returns a list of all univariate polynomials f over F in the variable x of degree at most d which agree with the given set of points in at least t places (i.e., y = f (x ) for at least t values of i), provided t = ( nd). The running time is bounded by a polynomial in n. This immediately provides a maximum likelihood decoding algorithm for Reed Solomon Codes, which works in a setting with a larger number of errors than any previously known algorithm. To the best of our knowledge, this is the first efficient (i.e., polynomial time bounded) algorithm which provides error recovery capability beyond the error-correction bound of a code for any efficient (i.e., constant or even polynomial rate) code. ©1997 Academic Press f( x ; y g 2 p 1. INTRODUCTION 1.1. Problem Statement We consider the following problem. PROBLEM 1. : A field ; and integers and . distinct pairs of elements : A list of all functions from ; satisfying The above problem and close variants have been considered before in the context of coding theory and learning. The best threshold on for which *E-mail: madhu@watson.ibm.com. 180 0885-064X/97 $25.00 Copyright ©1997 by Academic Press All rights of reproduction in any form reserved. DECODING OF REED SOLOMON CODES 181 a polynomial time bounded algorithm solved this problem was previously . Note that the ratio if we fix and let . In fact, when satisfies this property then there is a unique function satisfying (1). A particularly simple algorithm for this case is given by Berlekamp and Welch [4] (see, for instance, Gemmell and Sudan [9]). In this paper we present an algorithm which solves the problem given above . Note that for fixed as the fraction of agreement for required by our algorithm approaches 0 (and not which is what previous algorithms seemed to get). The algorithm is based on an algorithm of Ar et al. [1] for a restricted version of this problem. The task of reconstructing polynomials describing a given set of points is relevant in the context of coding theory and we describe this context next. This task may also be of some relevance to computational complexity theory. We touch on this motivation very briefly in the conclusions. 1.2. Error-Correcting Codes and a collection of symbols an -code over For integers is a collection of -letter words over the alphabet with and the property that any two strings in differ in at least places (i.e., the strings have Hamming distance ). Given a code , the largest for which is a -code is referred to as the distance of the code. If satisfies then an -code is also a -error-correcting code. The largest such value will be referred to as the error-correction bound of a code. This terminology reflects the fact that, given any string there is at most which is within a Hamming distance of from . one string The codes of relevance to this paper are the Reed Solomon Codes. of size and parameter , the Reed Solomon Code For a finite field is an -code over whose codewords are the strings , where is some fixed primitive element of and ranges over all polynomials of degree at most over (see, for instance, [18, page 86]). The algorithmic tasks of relevance to this paper are the tasks of “errorcorrection” and “maximum-likelihood decoding.” The problem of -errorcorrection for an -code is defined for as follows: “Given a string find a string which is within Hamming distance of , if one such exists.” Since is a error-correcting code, the answer, if it exists, is unique. Note that the problem is not defined for values of and (i.e., when > ) which may allow for non-unique answers. The maximum-likelihood decoding problem (see, for instance, [17, page 8]) is set in a model where a larger number of errors is considered less likely and is defined as follows: “Given a string find a (the) codeword which is nearest to (i.e., least Hamming distance away from ).” This problem is also sometimes referred to as the nearest codeword problem. 182 MADHU SUDAN Previous work has focused mainly on the task of error-correction, and algorithms are known for error-correction of many interesting codes. In particular, for Reed Solomon Codes the -error-correction problem for a error correcting code can be solved in polynomial time. In particular, the earlier mentioned solution of Berlekamp and Welch [4] works in this setting. The case of recovering from an error larger than the error-correction capacity of a code has not attracted the same amount of attention and significantly less is known about this problem. Since in this case the solution to the maximumlikelihood decoding problem may not be unique, it is not clear which solution is to be reported when the answer is not unique. Further, it is not clear why any algorithm would be able to (or should be allowed to) prefer any one solution over any other equally respectable solution. However, it is possible to define a closely related problem which does not offer the algorithm any choice in its solutions. This problem, sometimes called the -reconstruction problem, is defined as follows: “Given a string find all codewords that are within Hamming distance of .” This problem is also known as the list decoding problem. The reconstruction problem offers the solution to the maximum-likelihood decoding problem for a much larger range of than is allowed by the -error correction problem. This is the problem tackled in this paper for the case of Reed Solomon Codes. The -reconstruction problem is not a universal panacea for the maximumlikelihood problem. In fact, by making the task enumerative (rather than picking one element from a large set, we want the whole set), the complexity requirements of the task go up. In particular, the running time is lower bounded by the output size. Bounds on the output size of the reconstruction problem have been studied in the context of coding theory and a well known bound, due to Johnson (see [17, page 525]), bounds the number of solutions by for binary codes (i.e., codes over the alphabet {0, 1}), provided that the denominator is positive. For general codes, a simple bound can be shown by an inclusion–exclusion argument (see, for instance, [11]) which yields that the number of solutions to the -reconstruction problem is at most if provided . (Another such bound is also known due to Goldreich, et al. [11]. We do not describe this here.) However the inclusion–exclusion bound is not constructive, i.e., it does not provide a list of the codewords which may be the solution to the reconstruction problem. It is reasonable to ask for a solution to the reconstruction problem which runs in polynomial time, when the output size is bounded. Here we solve the reconstruction problem for Reed Solomon Codes for exactly the same values of the parameters and for which the inclusion–exclusion bound seems to work. Finding bounds which work for more general settings of and and finding solutions to the reconstruction problem which work in such settings remain open questions. 183 DECODING OF REED SOLOMON CODES 1.3. Previous Work As mentioned in the previous section, the -error-correcting problem is wellstudied and we will not describe past work in that problem here. The definition of the -reconstruction problem used here is based on definitions of Ar et al. [1] and Goldreich et al. [11]. To the best of our knowledge, there are only two instances where the -reconstruction problem has found interesting solutions for some error-correcting code. The first, due to Goldreich and Levin [10], provides a solution for certain families of Hadamard Codes. Kushilevitz and Mansour [15] provide a variant of this algorithm which applies to the same codes. The second instance involves a generalization of the codes and algorithm given by [10] and is due to [11]. Both the codes given here are extremely low-rate codes. ) for these codes is and thus In fact, the rate (i.e., the ratio a brute-force algorithm (running in time ) is not too inefficient for these problems. The feature that makes the solutions of [10, 11, 15] interesting and using random access to efficient is that they work in time polynomial in an oracle describing the input . This makes the solution interesting in some learning theoretic and cryptographic settings, but is however not very useful for coding theoretic applications (due to the low information rate). The techniques of Goldreich and Levin (which are inherited by [11, 15]) are interesting in that they manage to convert, modularly, the nonconstructive bounds on the number of outputs (discussed earlier) into constructive ones. But their technique does not appear to generalize to the setting of Reed Solomon Codes (or any other high-rate codes). Ar et al. [1] do provide some solutions to the reconstruction problem, but not in its full generality. In particular, they restrict the nature of the input word . For this restricted case they provide a solution to the reconstruction problem based on algebraic techniques (and in particular uses polynomial time solutions to the bivariate factoring problem). Our solution is a minor modification of their algorithm and analysis which manages to get around their restriction. 2. ALGORITHM We now present our algorithm for solving the problem given in Section 1.1. the DEFINITION 1 (Weighted Degree). For weights is . The -weighted weighted degree of a monomial degree of a polynomial is the maximum, over the monomials with nonzero coefficients, of the -weighted degree of the monomial. */ ALGORITHM. /* Inputs: A. /* Parameters to be set later. */ 184 MADHU SUDAN B. Find any function satisfying C. Factor the polynomial into irreducible factors. D. Output all the polynomials such that is a factor of for at least values of from . and Note. Step C above can be solved in randomized polynomial time with zerois of characteristic zero, or if the running time is allowed to sided error. If then the solution can be obtained deterministically. (See, be polynomial in for instance, [13].) 3. ANALYSIS In order to prove that the algorithm above can be run in polynomial time and works correctly, we need to prove the following set of claims. In all the . following claims, we fix the set of pairs CLAIM 2. If a function found in time poly( ). satisfying (2) exists, then one can be . Then we wish to find Proof. Let satisfying the constraints , for coefficients every . This is a linear system in the unknowns and hence if a solution exists then one can be found in polynomial time. CLAIM 3. If satisfying (2). > then there exists a function Proof. First we observe that the linear system is homogeneous. Therefore the setting satisfies all the linear constraints. However, this does not satisfy (2), since would be identically zero. In order to show that a nonzero solution exists, we observe that the number of unknowns in the linear system that we wish to solve is . Since this is more than we have a homogeneous linear system with more variables than constraints and hence a nonzero solution exists. The following lemma is a special case of a general class of theorems known as Bezout’s Theorem. Since we will be interested in tight behavior of the theorem, we use a version due to [1]. The proof is also from [1]. 185 DECODING OF REED SOLOMON CODES CLAIM 4. If isfying (1) and > is a function satisfying (2) and , then divides is a function sat. . This is a polynomial in Proof. Consider the function and we argue that it has degree at most . Consider any monomial of . Since has -weighted degree at most we have that . Thus the term is a polynomial in of degree at most . Thus has degree at most (since it is a sum polynomials of degree at most ). Now we argue that is identically zero. Since is zero we have that is zero for strictly greater than whenever points. Thus has more zeroes than its degree and hence is identically zero, . implying Now consider as a polynomial in with coefficients from the ring of polynomials in . By the polynomial remainder theorem, we then divides . Substituting have that if yields the lemma. We are now ready to optimize the parameters and . For now we will ignore the fact that and have to be integers and fix this aspect later. Note that we want Thus, given a value of we can compute the smallest condition holds, and that is must be at least for which the second . Thus we find that We can now minimize the expression above as a function of the unknown parameter to obtain the smallest value of for which this algorithm will work. The minimum occurs when This setting yields and 186 MADHU SUDAN Due to the integrality issues we will lose a little bit in the following theorem. where the THEOREM 5. Given a sequence of distinct pairs and are elements of a field and integer parameters and such that , there exists an algorithm, which runs in time of degree at most polynomial in that can find all the polynomials such that the number of points such that is at least . and Proof. We use the algorithm of Section 2 with . It may be verified that this setting satisfies the . Hence, by Claim 3, Step 2 will condition return a function satisfying property (2). Furthermore, the setting of and also satisfies the condition > . Thus Claim 4 guarantees that if is a function satisfying (1), then will divide the polynomial returned in Step 2 and hence be one of our outputs. 4. BOUND ON THE NUMBER OF POLYNOMIALS Here we give a new proof of an upper bound given in [11] on the number of . polynomials of degree agreeing with out of distinct points Their proof uses an inclusion–exclusion argument, while ours is different. Note that their bound is exactly the same and applies under exactly the , and with the important difference being that our same conditions on proof holds only for the univariate polynomial case, while theirs applies more generally. Nevertheless, we feel that this new proof may be of some interest. Furthermore, this justifies the statement, made in Section 1, that our algorithm works in exactly the same setting as the inclusion–exclusion bound. LEMMA 6. If nomials of degree then the number of polysatisfying (1) is at most Remark. The bound above shows that when is linear in then the number of polynomials is bounded by a constant (see also Table I). In particular, if (i.e., the number of errors is less than the error-correction limit), then the bound shown above is 1. Proof. If and are integers such that the algorithm of the previous section works correctly, then the algorithm of Section 2 gives at most solutions 187 DECODING OF REED SOLOMON CODES TABLE I = nk Error-correction bound Error tolerated by our algorithm # Solns reported by our algorithm .5 .25 .25 1 .33 .333 .35 2 .25 .375 .417 2 .10 .45 .602 4 Rate implying that is an upper bound on the number of polynomials of degree which can agree with given points at places. In other words, if and are integers satisfying and Then is an upper bound on the number of functions satisfying (1). Equation (4) indicates that we should pick to be as large as possible subject to the con. We therefore set to . Thus (3) reduces to straint In other words, we require Let Then and are the two roots to (5). If > 1, then 1 and 1Note that we need l to be an integer which satisfies the inequality (5) strictly and hence we are c rather than just d e. forced to use b +1 188 MADHU SUDAN satisfy conditions above and provides an upper bound on the number of functions satisfying (1). The condition > 1 is satisfied if If satisfies the condition above then we get the following bound for : This yields the lemma. (Equation [11] already shows that the final quantity is so we do not have to show that part.) upper bounded by 5. EXTENSIONS TO MULTIVARIATE POLYNOMIALS It is relatively simple to extend the algorithm and the analysis of the earlier sections directly to apply for multivariate polynomial fitting. We first extend the problem definition. Some slight care has to be taken to determine what is an appropriate extension of the problem definition, and the definition below turns out to be the one for which the extension works easily. to denote the We consider -variate polynomials. We use variables. The shorthand will be used to denote this tuple of variables in vector notation. 189 DECODING OF REED SOLOMON CODES PROBLEM 2. A field a set a function and integers and . A list of all functions satisfying The algorithm is a straightforward generalization of the algorithm of Section 2. We first extend the definition of weighted degree in the obvious way. the -weighted degree DEFINITION 7. For integers of a monomial is . The -weighted degree of a polynomial is the maximum, over the monomials with nonzero of the weighted degree of the monomial. coefficients in MULTIVARIATE ALGORITHM. /* Inputs: */ A′. /* Parameters to be set later. */ B′. Find any function satisfying C′. Factor the polynomial into irreducible factors. D′. Output all degree polynomials such that and for at least values of . is a factor of As usual we need to ensure that the number of coefficients of is more than and prove that, for sufficiently large the algorithm will output all solutions to the reconstruction problem above. CLAIM 8. If function satisfying (7). , then there exists a is strictly larger than Proof (Sketch). The number of coefficients in . (The bound stated is a gross approximation. A generic degree polynomial in variables has coefficients. is a degree polynomial in variables with the restriction that the degree of only ranges from 0 to . This restriction is taken care of by the 190 as large as MADHU SUDAN factor.) Again this number can be lower bounded (grossly) by and we wish this to be at least , which follows easily. Now it remains to mimic Lemma 4 in the multivariate setting. CLAIM 9. If isfying (6) and > is a function satisfying (7) and then divides is a function sat. Proof (Sketch). Proof similar to that of Lemma 4. The function is a polynomial of degree and identically zero. Hence is a root of the polynomial Thus with some optimization of parameters we are done. , where is a field THEOREM 10. Given a table for a function and is an arbitrary finite subset of and integers and a list of all polyof degree at most which agree with in places can be nomials found in time poly provided that Proof. Let . Set and . , Lemma 8 guarantees that Then since Step B′ of the multivariate algorithm will return a polynomial satisfying (7). Further, the condition on implies that indicating that if a will divide . Thus all polynomials will function satisfies (6), then be returned by this procedure. 6. CONCLUSIONS We first discuss the univariate reconstruction algorithm. The limit on i.e., is a significant weakness in the practical usability of this code. Many codes where , and for such applications tend to work with growth our proof does not yield anything interesting. However, low-rate codes, and even Hadamard codes, have been used in the past 2 and when it suffices to use a low-rate code the reconstruction algorithm has some advantage. In Table I, we list the error which the reconstruction algorithm can tolerate and the number of solutions produced for such rates. The -reconstruction problem remains open 2Apparently the Mariner used a Hadamard code ([18], page 26). DECODING OF REED SOLOMON CODES 191 for values of closer to the distance of the code, i.e., for where > 0, and finding such a solution may be of both theoretical and practical interest. One does not expect the running time of such an algorithm to be polynomial since Goldreich et al. [11] give a NP-hardness proof in this case. The in with being the degree of NP-hardness result there has the desired polynomial. The instance is defined over the reals/rationals. One of the main hopes in investigating this problem was that the algorithmic solution will be of some use in complexity theory. It is in this area that numerous applications for “Reed–Solomon-like” codes have occurred repeatedly. Examples include: (1) determining the hardness of the permanent on random instances [7, 16], (2) fault tolerant computing in distributed computing environments [3], and (3) many results involving “probabilistically-checkable proofs.” (See the survey by Feigenbaum [8] for a detailed look at many connections.) In particular, in the case of applications to probabilistically checkable proofs, it becomes useful to be able to characterize functions that are not close to polynomials (to be able to refute claimed proofs of incorrect statements). Note that our algorithm proves that the problem of recognizing points which are not very close to any polynomial is decidable in NP (by showing it is in P). The witness for this property is a proof that a function does not have a low-degree polynomial describing it on even a tiny fraction of the input. Recent work by Arora and Sudan [2] seems to justify our initial hope by using some of this analysis in a new analysis of a low-degree testing, which in turn leads to new constructions of proof systems. Another application has also been shown (very recently) by Impagliazzo and Nisan [12] in the area of cryptography. They use the algorithm presented here to obtain “random-efficient constructions of hardcore predicates for one-way functions.” (Defining any of these terms is beyond the scope of this paper.) Now, moving on to the multivariate reconstruction problem, several glaring open problems remain. For starters, it is not clear how to set a bound on the grows slower than Even in number of solutions when , the reconstruction problem is not the cases where is larger than no algorithm appears to fully solved. In particular, even for the case solve this problem efficiently. This is hopefully an oversight on our part and some simple modification of the methods in [1, 9, 11] may work. However, such a solution does not remain algebraic. The question of whether there is an algebraic solution to the -variate problem for general , which works whenever , seems to be another interesting question. Finally, we speculate on the complexity of the maximum-likelihood problem (or the nearest codeword problem). This problem is known to be NP-hard for general linear codes [5]. The hardness of the problem considered in [5] could be due to one of two reasons: (1) it is a maximum likelihood decoding problem rather than a -error correction problem; or (2) the code is specified as part of the input, rather than being a “well-known” one which is more 192 MADHU SUDAN standard. It would be nice to know which of the two causes is responsible for the harness, since for all well-known codes the -error correction problem seems to well solved. Bruck and Naor [6] present a code (not well known, but nevertheless easily presented) for which they show that the existence of small size maximum likelihood decoding circuits would imply the collapse of the polynomial hierarchy (using a result of Karp and Lipton’s [14]). However, the codes presented by Bruck and Naor do not have a large distance. It still remains open if the maximum likelihood decoding problem is hard for any constant distance code. The Reed Solomon codes would have formed a good candidate to show the hardness of this problem, except that it is not as hard to solve. It would be nice to find another candidate for such a hardness result. Finally, there is still the possibility that some error-correcting codes are hard to decode, even to the full extent of their error-correction capacity. Alternately, we can ask the question: Is it possible to construct a -error correction algorithm for every -error correcting linear code, specified by its generator matrix? A positive answer to this might necessitate an algorithm to determine the distance of a code, a well-known open problem. ACKNOWLEDGMENTS I am grateful to Sanjeev Arora, Oded Goldreich, Ronitt Rubinfeld, Martin Tompa, and Shmuel Winograd for many valuable comments and discussions. I am especially grateful to Greg Sorkin for spending numerous hours implementing a version of the algorithm presented here. REFERENCES 1. Ar, S., Lipton, R., Rubinfeld, R., and Sudan, M. (1992), Reconstructing algebraic functions from mixed data, in “Proceedings of the 33rd Annual IEEE Symposium on Foundations of Computer Science, 1992,” pp. 503–512. 2. Arora, S., and Sudan, M. (Aug. 1996), manuscript. 3. Ben-Or, M., Goldwasser, S., and Wigderson, A. (1988), Completeness theorems for noncryptographic fault-tolerant distributed computation, in “Proceedings of the 20th Annual ACM Symposium on Theory of Computing, 1988,” pp. 1–10. 4. Berlekamp, E., and Welch, L. Error Correction of Algebraic Block Codes, U.S. Patent Number 4,633,470, 1986. 5. Berlekamp, E., McEliece, R., and Van Tilborg, H. On the inherent intractability of certain coding problems, IEEE Trans. Inform. Theory (1978), 384–386. 6. Bruck, J., and Naor, M. The hardness of decoding linear codes with preprocessing, IEEE Trans. Inform. Theory (1990), 381–385. 7. Feige, U., and Lund, C. On the hardness of computing the permanent of random matrices, in “Proceedings of the 24th Annual ACM Symposium on Theory of Computing, 1992,” pp. 643–654. DECODING OF REED SOLOMON CODES 193 8. Feigenbaum, J. The use of coding theory in computational complexity, in “Proceedings of Symposia in Applied Mathematics,” (R. Calderbank, Ed.), pp. 203–229,American Math Society, Providence, RI, 1995. 9. Gemmell, P., and Sudan, M. Highly resilient correctors for polynomials, Inform. Process. Lett. 43 (1992), 169–174. 10. Goldreich, O., and Levin, L. A. A hard-core predicate for any one-way function, in “Proceedings of the 21st Annual ACM Symposium on Theory of Computing, 1989,” pp. 25–32. 11. Goldreich, O., Rubinfeld, R., and Sudan, M. Learning polynomials with queries: The highly noisy case (extended abstract), in “Proceedings of the 36th Annual IEEE Symposium on Foundations of Computer Science, 1995,” pp. 294–303. 12. Impagliazzo, R., and Nisan, N. personal communication, Sept. 1996. 13. Kaltofen, E. Polynomial factorization 1987–1991, in “LATIN ’92,” (I. Simon, Ed.), Lecture Notes in Computer Science, Vol. 585, pp. 294–313, Springer-Verlag, New York/Berlin, 1992. 14. Karp, R., and Lipton, R. Some connections between nonuniform and uniform complexity classes, in “Proceedings of the 12th Annual ACM Symposium on Theory of Computing, 1980,” pp. 302–309. 15. Kushilevitz, E., and Mansour, Y. Learning decision trees using the Fourier spectrum, in “Proceedings of the 23th Annual ACM Symposium on Theory of Computing, 1991,” pp. 455–464. 16. Lipton, R. New directions in testing, in “Distributed Computing and Cryptography,” DIMACS Series in Discrete Math. and Theoretical Computer Science, Vol. 2, AMS, Providence, RI, 1991. 17. MacWilliams, F., and Sloane, N. “The Theory of Error-Correcting Codes,” North-Holland, Amsterdam, 1981. 18. Van Lint, J. H. “Introduction to Coding Theory,” Springer-Verlag, New York/Berlin, 1982.