Browse Books

Reviewer: Constantin S Chassapis

Rarely have I seen such a dense and straight to the point pedagogical monograph on such a modern subject. This excellent book, on the simulated cross-entropy method (CEM) pioneered by one of the authors (Rubinstein), is very well written, starting with an exposition on the notions to be used, continuing with examples and a detailed presentation of the variants of the method, and closing with a discussion of how CEM effectively attacks an impressive list of modern problems. These modern problems include combinatorial optimization problems (COP), like the traveling salesman problem (TSP); the quadratic assignment problem (QAP); the max-cut problem; the alignment of deoxyribonucleic acid (DNA) sequences, and noisy estimation problems, like the buffer allocation problem (BAP); noisy optimization; continuous multi-external optimization; machine learning; and clustering and vector quantization problems. The amazing thing is that such a diverse list of topics is attacked and solved by methodologies that all have, at their core, the same simple notions. These notions are briefly presented below. Assume that we are searching for the solution of an integral of the form A = I[f(x)g(x;u)dx] = E u[f] , where x , and u are multidimensional vectors. We also assume that no analytical solution is known. We use Monte Carlo simulation. We first try the "brute force" estimator, (1/N) Σi [f(x i)] , where the N points x i follow the g(x;u) distribution. The problem arises when f(x) is meaningful where the values x i are very rare, according to g . In this case, we might use an extremely large N to estimate the quantity A with an acceptable relative error. A clever approach is to find an importance sampling (IS) function h(x) , usually belonging to the family of the g(x;u) densities, and then attack an easier problem to solve: I[f(x) g(x)/h(x) h(x)dx] = E h[fg/h] . Now we go stochastic again, and use a first generation of N points { x i} that follow the h(x) distribution, and the estimator: (1/N) Σ i[f(x i) g(x i)/h(x i)] . The best way to use this estimator is to find the best h(x) to use, and this best, named g* , is equal to f(x)g(x;u)/A , since, when we put this in the summation above, we get (1/N)NA = A . The problem is that this best h is utopic, since A is unknown. If we then start from a nonoptimal h , the question is: "Can we use a generation of sample points x i so as to improve h , and then repeat the process with an ameliorated h __ __" This compels us to find a way to compare the h distributions with the initial g *. Here, the Kullback-Leibler distance (or cross-entropy) is used: D g (g;h) = I[ln(g(x)/h(x)) g(x)dx] = I[g lng dx] - I[g lnh dx] >= 0. So, what we are searching for is an h with the minimal D from g *; we are searching for the solution to the functional minimization problem minh{ D g *( g *; h )}, where the class of densities h is restricted to the family g(x;v) , such that minv{ Dg *( g *; g(x;v) }. Since the first term of D does not depend on v , this reduces to maximizing only I[g * lng(x;v)dx] with respect to v . Putting the expression for g * back we get: maxv{ I[(f(x)g(x;v)/A) lng(x;v)dx]} = maxv{ E u [f(x)lng(x;v)]} . But now we can again use the IS-approach on the last expectation integral with an IS-density like g(x;w) , to obtain: maxv{ E w [f(x)lng(x;v)g(x;u)/g(x;w)]} . Obviously, we perform this step because we want to create an iterative algorithm that will approximate g *, little-by-little, searching the u -space of g(x;u) . Now the last expression can be attacked stochastically, as we did for A , namely: v * = arg-maxv {(1/N) Σi[f(xi)lng( x i; v ) g (xi; u )/ g (xi; w )]} (a), where the x i follow the g(x;w) distribution. In typical applications, the part inside the brackets {} is a convex and differentiable function with respect to v , so v * can be found by solving the following system of equations: (1/ N ) Σi[ f ( x i) g ( x i; u )/ g ( x i; w )( d/dx )lng( x i; v )]} = 0. So, the main CE algorithm is: (1) Set t =1, v 0= u (2) Generate sample { x i } from the density g ( x ; v t -1) (3) Calculate f ( x i) and maybe do some tricks here (4) Solve the maximization problem (a) using v t -1 in place of w . Set v t = v (5) Set t = t +1 and go to 2, until satisfied (6) Estimate (1/ N ) Σ i[ f ( x i) g ( x i; u )/ g ( x i;v t)] using the latest x i set available. Step 3 and the "until satisfied" of step 5 need some clarification: since we stay in the family of the initial probability density function (pdf), there is strong potential that f ( x ) will still be meaningful for rare cases, and the initial problem remains, even under g ( x ; w ). The trick here is to go from f to a family of f s, specifically, to slightly redefine f at each iteration, so that, at each iteration, each v t will provide the optimal IS for the redefined f and the redefined A . Convergence criteria will be satisfied when the successive redefinitions of f drive it to the original f . COP problems are attacked by the CE method, by translating the original deterministic optimization into a related stochastic estimation problem, and then applying the rare-event machinery of the CE method. In general, however, choosing an appropriate parameterization and sampling scheme is more of an art than a science. The book contains exercises, extensive numerical examples, an extensive introduction into every subject (supported by an extensive literature survey), example programs, and an associated Web site, http://www.cemethod.org, with additional material, namely, programs, tutorials, papers, and links. Scientists involved in any of the problems mentioned above, students of the physical and engineering sciences, and any researcher involved in stochastic optimization and simulation should read this book. Readers will benefit from this book, both in inspiration, and in grasping the essence of a modern, extremely powerful, computational tool. Online Computing Reviews Service