A complexity theory of efficient parallel algorithms

Reviewer: James Lee Holloway

The class NC (problems with an algorithm that has at most polylogarithmic running time using polynomially many processors, and at most polynomial inefficiency) is frequently used to capture the notion of &#8220;problems that have good parallel algorithms.&#8221; This paper shows situations in which the class NC admits problems that do not have good parallel algorithms, and fails to admit problems that do have good parallel algorithms, for some reasonable definition of &#8220;good.&#8221; The authors propose a classification of parallel algorithms by their running time and inefficiency in comparison to the best serial algorithms known. Let T ( n ) be the running time of a parallel algorithm on a problem of size n , and let P ( n ) be the number of processors used by the parallel algorithm. Let t ( n ) be the running time of the best sequential algorithm for the same problem of size n . Algorithms are classified by running time as polynomially fast if T ( n ) t ( n ) &egr; , for some constant &egr; < 1 polylogarithmically fast if T n = log O 1 t n . Algorithms are also classified by inefficiency as having constant inefficiency if T n P n =O t n polylogarithmically bounded inefficiency if T n P n =t n log O 1 t n polynomially bounded inefficiency if T n P n =t n O 1 . This classification produces six classes. The authors study the class EP (algorithms that are polynomially fast and have constant inefficiency) in the most detail. Section 4 examines several parallel models of computation and then lists several separation theorems. By simulating one model with another the authors show that, if polylogarithmically bounded inefficiency and a polylogarithmic reduction in the number of processors are allowed, all the parallel models of computation presented are equivalent. These ideas provide a useful classification of problems that admit parallel algorithms. The authors state the utility of the results: Consider, for example, the probabilistic, efficient, logarithmic time parallel algorithm for connected and biconnected components, obtained in [1] for the CRCW [concurrent read, concurrent write PRAM] model. Our results immediately imply that parallel efficient algorithms exist for these problems on all the other PRAM [parallel random access machine] models, and on the DCM [direct connection machine] model. Such algorithms were not previously known. The authors provide an extensive bibliography on the complexity of parallel algorithms. Despite the papers length, it should be easily accessible to most computer scientists and will be particularly useful to people studying parallel algorithms.