Browse Books

Reviewer: William Ian Gasarch

The string 111111111111 looks less “random” than the string 100110001001. Kolmogorov complexity imposes rigorous definition on just such an intuitive notion of randomness. Once this is done, new questions arise and some old questions can be answered. Chapters 1 through 4 carefully establish the rigorous definitions needed to study randomness. If this were the authors' only goal, they would not need four chapters in which to accomplish it. However, they also explore many issues that lead to or are consequences of the definitions. Chapter 1 contains motivation for the subject and a quick review of combinatorics, probability, and computability. It serves as a good refresher course, but is too mature for first-time learners. Chapters 2 and 3 define the complexity of a string and some notions of randomness. Informally, the complexity of a string x is the shortest description of x . Chapter 4 explores connections between Kolmogorov complexity and probability. The authors discuss a distribution that gives more complex strings a higher probability. Chapters 5 through 7 apply Kolmogorov complexity to computer science. Chapter 8 applies Kolmogorov complexity to physics. Most of the applications use only a small part of the contents of chapters 1 through 4. This is good, because readers who are only interested in applications can read these chapters after having learned a few things from the preceding chapters. Chapter 5 applies Kolmogorov complexity to inference. According to the concept of Occam's razor, the least complex explanation of a phenomenon is the best one. Since Kolmogorov complexity gives a precise measure of the complexity of a string, Occam's razor can be made rigorous. This chapter carries this out for Bayesian inference and probably approximately correct (PAC) learning. In most cases the learning procedure is not quite practical, but some interesting theorems are proven. Chapter 6 applies Kolmogorov complexity to combinatorics, algorithm analysis, formal language theory, and circuits (the application to circuits is new in the second edition). This is the most interesting chapter for computer scientists interested in applications. The chapter uses very little Kolmogorov complexity theory, but rather focuses on the existence and abundance of incompressible strings. Some of these applications are described below. There are many more in both the text and the exercises. Chapter 7 covers resource-bounded Kolmogorov complexity. Intuitively, C t,s x is the shortest program that prints out x in time t x and space s x . The authors prove interesting theorems about C t,s and then use C t,s to prove theorems in complexity theory. Chapter 8 applies Kolmogorov complexity to physics. A computation is thought to lose energy. If it is reversible, however, then (theoretically) no energy is lost. Hence, this chapter spends some pages on reversible computation. In thermodynamics, one speaks of the information within a system. Since Kolmogorov complexity gives a rigorous definition of information, connections between the concepts are made. This is a great book. The other extant literature on Kolmogorov complexity is scattered, in need of translation, or otherwise hard to access. This book puts it all in one place in a readable, enjoyable style. In this second edition, the authors have added many new results that have been proven since the first edition was published in 1993 [1] .