Browse Books

Reviewer: Robert Stewart Roos

The authors of this brief, accessible introduction to the main ideas of probably approximately correct (PAC) learnability emphasize the main theoretical results and provide several illustrative examples. The presentation is leisurely, building from intuitive concepts and simple examples (such as learning monomials) to the definition, in chapter 3, of PAC learning. Considerations of computational complexity (as a function of accuracy, confidence, sample size, and representation size) are deferred until chapters 5 and 6. Chapters 7 and 8 introduce the Vapnik-Chervonenkis (VC) dimension and contain the central theorem of the book—the proof that classes with finite VC dimension are precisely those that are potentially learnable, that is, for which the existence of a consistent learning algorithm implies PAC-learnability. (The authors do not neglect to mention the necessary measure-theoretic qualifications to this statement, but the book does not contain a detailed discussion of such matters.) The next chapter revisits and extends results from previous chapters in light of the VC dimension material. The book concludes with applications to perceptrons and neural nets. The preface states that the book is intended for graduate students and researchers unfamiliar with the basic concepts of PAC-learning. In fact, since the text makes few assumptions about the reader's background in complexity theory (notions such as O -notation, NP-completeness, and polynomial-time reductions are introduced at the appropriate places), topics from the first six or seven chapters could easily be used to enrich an undergraduate course in the theory of algorithms or complexity. The book's brevity makes it unsuitable as the sole textbook for a semester course in learning theory, but it could easily be used in conjunction with a collection of papers dealing with topics not covered in the book. In particular, the book omits detailed explorations of weak learnability, learning with queries, learning under fixed distributions, learning from noisy data, and mistake-bounded learning (although these subjects are mentioned in remarks at the ends of the chapters, with bibliographic references). Only the classic examples are used; readers should not expect a catalog of VC dimension results or learning algorithms for specific concept classes. The authors do not deal with learning in the limit, another area that is often classified under the heading of computational learning theory. Each of the ten chapters ends with a set of exercises; they range in difficulty from extremely simple calculus or plug-in-the-numbers problems to moderately difficult proofs. The bibliography contains all the basic pape rs on PAC learnability, but is by no means a comprehensive list of PAC learning papers. The index is good. The text appears to have few typographical errors. Some might quibble with choices of notation, such as the use of D n,k rather than the more familiar k -DNF. A comparison to Natarajan's recent book on computational learning theory [1] seems appropriate. The Anthony and Biggs book moves at a slower pace than the Natarajan text. Natarajan's book includes a number of additional topics, such as a chapter devoted to the learnability of non-Boolean concepts, one on learning finite automata, and sections devoted to weak learnability and cryptography-based hardness results. In short, the Anthony and Biggs work has been aptly classified by the publisher as a tract rather than a textbook; it is notable for its clean, readable, self-contained treatment of the foundations of PAC learnability theory.