JOURNAL OF COMPUTER AND SYSTEM SCIENCES 46, Threshold 129-154 (1993) Circuits of Bounded ANDRAS HAJNAL Department of Mathematics, Statistics, University of Illinois at Chicago Institute of the Hungarian Academy WOLFGANG Institute for Theoretical Austria, and Department Depth and Computer Science, and Mathematical of Sciences, Budapest MAASS* Computer Science, Technische Universitiit of Mathematics, Statistics, and Computer University of Illinois at Chicago Graz, Science, PAVEL PUDLAK Department University of Mathematics, Statistics, and Computer Science, of Illinois at Chicago and Mathematical Institute, Czechoslovakian Academy of Sciences, Prague MARIO Department qf Computer SZECEDY Science, University of Chicago AND GY~RGY TURIN Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago and Automata Theory Research Group of the Hungarian Academy of Sciences, Szeged Received December 8, 1988; revised January 5, 1991 We examine a powerful model of parallel computation: polynomial size threshold circuits of bounded depth (the gates compute threshold functions with polynomial weights). Lower bounds are given to separate polynomial size threshold circuits of depth 2 from polynomial size threshold circuits of depth 3 and from probabilistic polynomial size circuits of depth 2. With regard to the unreliability of bounded depth circuits, it is shown that the class of functions computed reliably with bounded depth circuits of unreliable A, v , 1 gates is narrow. On the other hand, functions computable by bounded depth, polynomial-size threshold circuits can also be computed by such circuits of unreliable threshold gates. Furthermore we examine to what extent imprecise threshold gates (which behave unpredictably near the threshold value) can compute nontrivial functions in bounded depth and a bound is given for the permissible amount of imprecision. We also discuss threshold quantifiers and prove an undefinability result for graph connectivity. 0 1993 Academic Press, Inc. * Supported in part by NSF Grant CCR-8703889. 129 0022~0000/93 $5.00 Copyright 0 1993 by Academic Press, Inc. All rights of reproduction in any form reserved. 130 HAJNAL ETAL. 1. INTRODUCTION Circuits with bounded depth and unbounded fanin were studied recently for several different reasons. These circuits are related to relativized separation of complexity classes (Furst et al. [13]) and definability problems (Ajtai [ 11); they also form a model of parallel computation (Chandra et al. [S]). The results of Ajtai [I], Furst et al. [ 131, Yao [34], Hastad [ 151, Razborov [25], and Smolensky [31] give superpolynomial lower bounds for the size of bounded depth circuits with A, v, 1 gates computing PARITY, and of bounded depth circuits with A, v, 1 and PARITY gates computing MAJORITY. Thus with respect to bounded depth, polynomial size reducibility ( Gus) [IS], 0 <bp PARITY <bp MAJORITY. The next step in extending this hierarchy would be to study bounded depth, polynomial size circuits where MAJORITY is also allowed as a single gate. We consider here these circuits and the complexity class TC” of Boolean functions computed by them. This class was first studied by Parberry and Schnitger [22]. The definition we use allows gates computing thresholdfunctions of the form T;t(.Y,,...,Y,) = 1 iff f a,y,Bk z=I (where a = (a,, .... a,)). Circuits with threshold gates are also related to several computational brain models (e.g., Boltzmann machines [27]) and to other models related to pattern recognition and learning, some of which have been studied since the 1950’s (e.g., perceptrons 120,271). We show that depth-2 threshold circuits computing INNER PRODUCT MOD 2 must have exponential size. The proof is based on the notion of a discriminator and a lemma of Lindsey [3,9, lo] about Hadamard matrices. As INNER PRODUCT MOD 2 can be computed by polynomial size depth-3 threshold circuits and by polynomial size depth-2 probabilistic threshold circuits, this gives a separation of the corresponding complexity classes. We were not able to prove lower bounds for circuits of larger depth. Probabilistic threshold circuits of depth d can be simulated by deterministic threshold circuits of depth d+ 1 (with polynomial increase in size). If the problem to be computed satisfies some symmetry conditions then there is a simulation in the other direction too. We also consider reliability in the context of bounded depth, unbounded fanin circuits. The study of reliable computation by circuits of unreliable gates (i.e., gates which output an incorrect value with a certain probability) goes back to von Neumann [21]. More recent results are due to Dobrushin and Ortyukov [lo] and Pippenger [23 J. These results show, that in the bounded fanin case every function can be computed reliably over any complete basis without a significant increase in complexity (depth and size). THRESHOLDCIRCUITSOFBOUNDEDDEPTH 131 In the bounded depth, unbounded fanin case the situation is quite different. The class of functions which are computable reliably with bounded depth A, v , 1 circuits of any size is very restricted. These are the functions for which {x: f(x) = 1 } can be obtained from a bounded number of subcubes by Boolean operations. In contrast, threshold circuits with unreliable gates can be used for reliable computation in bounded depth. For threshold gates one can consider another notion of unreliability, which may have some practical relevance if it becomes technically feasible to build threshold gates with large fanin. A threshold gate is called imprecise if it behaves unpredictably when the evaluated sum is near the threshold value. We prove a general lower bound which is useful in determining the permissible amount of imprecision. We also consider a logical characterization of TC” using the generalized quantifier #k (“there are at least k elements...“) introduced by Immerman and Lander [ 171. It is shown that CONNECTIVITY of graphs cannot be defined by sentences of bounded quantifier rank using threshold quantifiers and a successor relation. There are many open problems about threshold circuits. Some of these are mentioned at the end of the paper. 2. DEFINITIONS AND EXAMPLES A thresholdfunction is of the form if CyZ, EiyiBk otherwise, G(YI,...>Y,)= where the q’s are the weights, a = (aI, .... cx,) E 2”’ is the weight vector, and k E Z is the threshold value. We use the notation a= f Ictjl. i=l We shall also use the notations T;(y,, .... y,), T!Jk(yl; .... y,) for the functions “at least k l’s, ” “at most k 1’s.” A threshold circuit C is a Boolean circuit where every gate computes a threshold function. We assume that the gates are on levels. Level 0 contains 2n + 2 nodes labeled x1, .... x,, X, , .... ,3,, 0, and 1. Every edge connects a node on level 1 to a node on level I + 1. A circuit is a formula if every gate has outdegree Q 1. The size of C is the number of gates, its depth is the length of a longest path. The weight of C is the maximal absolute value of weights occurring in gates of C. If v is a gate of C and x is an input, we write C,(x) for the value computed at v for input x. C(x) is the output value. TC: is the class of languages L s { 0, 1) * for which there is a polynomial p and a sequence (CJnGN of depth d threshold circuits such that C, computes L n (0, 1)” and both the size and the weight of C, are bounded by p(n). 132 HAJNAL ET AL. TC”= U&N TC: is the class of languages recognizable by threshold circuits of bounded depth such that the size and the weight are bounded by a polynomial. (As for AC”, one can define uniform variants but we do not pursue this in this paper.) We give some examples of functions in TC”. PROPOSITION 2.1. Symmetric functions circuits of linear size and weight 1. can be computed by depth-2 threshold Proof Let f(xi, .... xk) be a symmetric function described by S= {s,, .... sk} c (0, ...>n) (i.e., f(x) = 1 o C:= i x, E S). Then the circuit T:“, , V:,(x), T:,s, (~1, ...>T:,(x)> T:,,(x)) computes f: (Observe that the number of l’s that are input to the final gate is always either k or k + 1). 1 Another example is given by the SUM-EQUALITY EQUALITY,(x,, .... x,, yl, .... y,) = 1 iff x7=, xi= C:=, y,.) PROPOSITION 2.2. SUM-EQUALITY function. (SUM- is in TC,“. INNER PRODUCT MOD 2, is the 2n-variable function x . y = xi y, @ . . 0 x, y,. Proposition 2.1 implies that this function is in TC,“, (It will be shown not to belong TC,” .) Other problems in TC,” are ADDITION, SUBTRACTION, and COMPARISON of two n-bit numbers.+ The reductions of Chandra et al. [8] and the results of Beame et al. [6] imply that the following problems are also in TC”: MULTIPLICATION of two n-bit numbers, SORTING of n n-bit numbers, MULTIPLICATION of n n-bit numbers, and DIVISION of two n-bit numbers (the last two, following [6], require nonuniformity). This relationship is further elaborated in Reif [26]. We mention some modifications of the circuit model for which the class of functions computable in bounded depth and polynomial size is equal to TC”: 1. Restriction to nonnegative weights: negative weights can be eliminated by replacing cliyi by a,( 1 - jj) if C(~< 0 everywhere in the circuit, and computing for every gate u not only C,(x) but c(x) too. This transformation does not increase the depth and at most doubles the size. 2. Restriction to majority gates: as the weights are polynomial, one can use multiple edges and new edges from the constants to obtain majority gates. 3. Generalization to threshold gates with arbitrary weights: as the number of bits of the weights can be assumed to be polynomial and the addition of several numbers can be done in bounded depth and polynomial size using majority gates [8], these gates can be simulated by subcircuits performing this task. + Note added in proof Recently Siu and Bruck have shown that ADDITION and COMPARISON are in TC,“; furthermore Siu and Roychowdhury have shown that MULTIPLICATION and DIVISION are in TC,“. THRESHOLD CIRCUITS OF BOUNDED 4. Gates for every symmetric function: tion 2.1 and modification 2 above. 133 DEPTH equivalence follows from Proposi- Proposition 2.1 suggests that depth d circuits with gates computing symmetric functions can be simulated by depth 2d threshold circuits. This simulation can be improved. THEOREM 2.3. Let f: (0, l}” -+ (0, 1) be computed by a depth d, size s circuit C of gates computing symmetric functions. Then f is computed by a depth d+ I threshold circuit C’ of size O((s + n)*) and weight 1. LEMMA 2.4. Let f: (0, 1 }” -+ (0, 1 } be computed by a depth-2, size-s circuit C of gates computing symmetric functions. Then f is computed by a depth-2 size-O(sn) circuit C’ such that C’ has threshold gates (with weights 0 or 1) on level 1 and a gate computing a symmetric function on level 2. Proof Let C be of the form where yV (1 < i < m, 1 < j < ri) are variables or their negations, S is given by 1s 1, ..-, Sk} c (0, ...) m} and Si is given by {si,,, .... s~,~,}E (0, .... ri}. Let the gates on the first level of C’ be T:;,( yil, .... y,,,) and T: s,,,( yil, .... y,,,) for 1~ i < m, 1 d j < ki and the final gate S’ be given by f i=l Then C’ computes ki+st, ..., f ki+sk . i=l f by the observation proving Proposition 2.1. 1 Proof of Theorem 2.3. C’ is constructed by applying the construction of Lemma 2.4 d - 1 times starting at levels 1 and 2 and, finally, applying the construction of Proposition 2.1. The size bound follows by observing that each gate u in C corresponds to 2k gates in C’, where k is the size of the set describing the symmetric function computed at v. n Other equivalent models (threshold parallel computer, TRAM, threshold Turingmachine, and a deterministic variant of the Boltzmann-machine) are given by Parberry and Schnitger [22]. The equivalence to circuits with ADDITION and MULTIPLICATION follows from the reductions of [S]. The equivalence to probabilistic threshold circuits is given in Section 4. A logical characterization of TC” is described in Section 7. Note that the classes TCS are not necessarily invariant under these simulations. The definition used here has the advantage that for every d, TCj is closed under p-projections [ 18,301 and has a natural complete problem MAJd: MAJd(x,, .... x,) is computed by a depth d, indegree n’ld tree of MAJORITY gates. 134 HAJNALETAL. 3. LOWER BOUNDSFOR DEPTH-2 CIRCUITS PARITY cannot be computed by a single threshold gate as every threshold function is monotone increasing or decreasing in every variable. Therefore Proposition 2.1 implies TC,” s TC,“. Now we show that the second and third levels of the hierarchy are also different. THEOREM 3.1. TC,” 5 TC;. This follows from: LEMMA 3.2. Fix any E > 0 and polynomial p. Assume that C is a depth-2 threshold circuit with weight <p(n) computing INNER PRODUCT MOD 2 of two n-bit strings. Then if n is sufficiently large, the size of C is at least 2(‘12-E)n. Before proving the lemma we define discriminators. DEFINITION. Let C be a circuit with n inputs and A, BG (0, 1 }” be disjoint sets. Let P, (resp. PB) denote the uniform probability distribution on A (resp. B). Then C is an e-discriminator for A and B if IPA = 1) - Ps(C(x) = 1)1> E. If f is an n-variable function, C is an E-discriminator for for A= {x: f(x)= i} and B= {x: f(x)=O]. f if it is an s-discriminator LEMMA 3.3. Let Tz(C, , .... C,) be a circuit with n inputs, a = CT=, 1~1~1 and A, BG (0, l}” be disjoint sets such that the circuit accepts A and rejects B. Then some subcircuit Ci (1 <id m) is a (l/a)-discriminator for A and B. Proof Let the random variable C:(x) (resp. C:(x)) be the output of Ci when x is distributed uniformly on A (resp. B). Then x7=“=,a,C,“(x)> k and x7! i criCf(x) < k - 1. Taking expectations and rearringing we obtain 1 < 5 qqC:‘(x)) i= - E(Cf(x)) 1 <a’,~,~m IP~(Ci(x)~l)~P~(Cj(x)~iI~ . 1 . This lemma reduces the problem of proving a lower bound for depth d circuits to showing that there are no depth (d - 1) &-discriminators for large E. The lemma holds for arbitrary probability distributions. Later (Theorem 4.6) we shall prove a partial converse of this lemma. The following lemma is a special case of a lemma due to Lindsey (see [3,9, 111). THRESHOLD LEMMA 3.4 (Lindsey). CIRCUITS OF BOUNDED DEPTH 135 For every X, YE { 0, 1 }“, I({(x,y)EXX Y:x.y=l)(-l{(x,y)&Yx y:x~y=o}(l Proof of Lemma 3.2. Let T;(C,, .... C,) be a circuit satisfying the assumption of the lemma. Thus the Cj are threshold gates. Let C, be a gate of the form f pjxj+ i j=l j= y,y,zl. 1 Let x,:= x: i pjxj=u ) i j=l 1 Y,,:= (thus (u( <<p(n)). i y: i yjyjkl-u j=l Then HP(n) {(x~Y):ci(x~Y)=l}= 0 x,x yu. u= -rip(n) Applying Lemma 3.4, I I((x, Y): Ci(X, y)= 1, ~~Y=~}l-l{~~,Y~:~i~x,y)=~,x~y=o}ll e y u= -rip(n) JWQ (2n-t 1) p(n) 2342 and as ~{(x,y):x~y=1}~=22”-1-22”-‘, I{(x,y):~.y=O)(=2~“-~+2”-~, if Ci is an &‘-discriminator for INNER PRODUCT MOD 2, then a’ < 2-(112-6’2)n if n is sufftciently large. Hence in the original circuit c(2 2(“2--E’2)n from Lemma 3.3. But 01< mp(n), thus ma--- 1 2(li*-“/2)“~2(1/2--E)n P(n) if n is sufficiently large. i Proof of Theorem 3.1. INNER PRODUCT MOD 2 is not in TC,” by Lemma 3.2. On the other hand, it is in TC,” as it was remarked after Proposition 2.2. 1 Lemma 3.2 implies further lower bounds using reduction by p-projections. 136 HAJNAL ETAL. COROLLARY 3.5. Fix any polynomial p. Let C he a depth-2 threshold circuit with weight 6 p(n). Then the following holds: computes MULTIPLICATION of two n-bit numbers then it has size > if c (b) if C computes CONNECTIVITY of n-vertex graphs then it has size 2”(“‘; (c) if C computes MAI’ of n bits then it has size 2*(@‘. 2nl,it!,. Proof: (a) Let x=x,O~~~Ox,O~~~Ox,~,O~~~Ox,, y= y,0...0G..y20...0y, where Llog, n J O’s are inserted between any two variables. Then x1 y, 0 . Ox, y, is the mth digit of the product xx y, where m is the length of x. (b) The fact that INNER PRODUCT MOD 2 is a p-projection of CONNECTIVITY follows from a result of Skyum [29]. A more eflicient construction is obtained by taking a cycle of length 4n and replacing every second pair of opposite edges by the labeled subgraph of Fig. 1. (Other edges are labeled 0.) (c) Proposition 2.1 shows that INNER PRODUCT MOD 2 is computed by a depth-3 formula of O(n) variable majority gates which leads to the desired projection. 1 The proof method of Lemma 3.2 can also be used to prove lower bounds for depth-2 threshold circuits having arbitrary weights on the first level. THEOREM 3.6. Let E, p, C be as in Lemma 3.2, except that there is no bound on the size of weights for gates on level 1. Then if n is sufficiently large, C has size at least 2(‘13- +. Prooj(Outline). Following the proof of Lemma 3.2 it suffices to show that if C, is a threshold gate of the form FIGURE 1 THRESHOLD CIRCUITS OFBOUNDED 137 DEPTH then if n is sufficiently large. Form a 2” x 2” matrix where the rows (resp. columns) are indexed by x’s in increasing order of Bx (resp. y’s in increasing order of yy), and entry (x, y) is Ci(x, y). In this matrix every entry either to the right of, or below an entry which is 1, is also equal to 1. Divide the matrix into 22ni3 square submatrices of size 22n/3x 22n/3. There are at most 2. 2”13 squares containing both a 0 and a 1. The 1 entries not in these squares can be covered by 2”13 rectangles of height 2”13 and width 6 2”. Thus using Lemma 3.4 the difference above is at most 2 .p3 .24”‘3+2fl13.~~=3.25”13, 1 4. PROBABILISTIC THRESHOLD CIRCUITS A probabilistic threshold circuit is a threshold circuit with two kinds of inputs, x = (Xl) **., x,) and unbiased random bits r = (ri, .... r,). P(C(x, r) = 1) is the probability that C accepts x. For A, B 2 (0, l}“, E > 0, we say that C gives advantage E to A over B if P(C(x,r)=l)b$+c for every x E A, P(C(x,r)=l)<$-.5 for every x E B. RTC: is the class of languages L G (0, 1) * for which a sequence of depth d threshold circuits (C,),, N such to Ln (0, l}” over Ln (0, l}“, cc1 <p(n), and the number of random bits used by C, are all bounded by RTC;= there is a polynomial p and that C, gives advantage E, size, the weight, and the p(n): u RTC;. LIEN We shall use the standard Chernoff inequality: if S, is the sum of m in&per&.nt random variables each taking value 1 (resp. 0) with probability p (resp. 1 - p) then P(S, > pm + h) < e-(2h2/m) and P(S, < pm-h) < e-(2h21m). LEMMA 4.1. Let A, Bc (0, 1 }” be disjoint sets. Let C be a depth-d probabilistic circuit with size, weight, and number of random bits all bounded by p(n) which gives advantage E to A over B. Then there is depth-(d + l), deterministic circuit c’ which accepts A, rejects B and has size and weight bounded by O(p(n) nEe2). Proof. Consider T”,,,(C,(x, rl), . . . . C,(x, r,)), where the Cjs are disjoint copies of C. For every x, Ci(x, ri) (I < i < m) are independent random variables. For every x E A, P(CJx, ri) = 1) > f + E, so P(Cy=, Ci(x, ri) <m/2) < e-2E2mfrom the Chernoff 138 HAJNAL ETAL. inequality and similarly for x E B, P(C;=, C, (x, r;) 2 m/2) < e “““. The probability that the circuit above gives an incorrect output for some x E A u B is < 2” v 2c2”‘, which is < 1 for some m = O(IZE-~). For this m, some choice of the random bits rl, .... rm works for every x. m PROPOSITION Proof 4.2. RTC: E TC;, , , hence RTC” = TC”. Follows directly from the previous lemma. 1 Note that the corresponding statement holds for AC” only if s;l= [2]. Now we show that the two hierarchies differ on the second level. THEOREM 4.3. (log n)o(‘) TC,” Y$RTC,“. This follows directly from Lemma 3.2 and the following lemma (giving another proof of Theorem 3.1). LEMMA 4.4. INNER PRODUCT MOD 2 is in RTC,“. LEMMA 4.5. Let L E (0, 1 } *, A,=Ln(O, l}“, B,,=En(O, l}“, and p be a polynomial. Suppose there is a sequence (C,), EN of depth d probabilistic circuits such that C, has size, weight, and number of random bits all bounded by p(n) and (1) for XEA,, P(C,(x,r)=l)>cr, (2) for x E B,,, P(C,,(x, r) = 1) 6 B, (3) O<(an-BXL~~(n). Then L E RTC”. Proof: Assume C, uses random bits rl, .... r, and is of the form Tz(C’, .... Cm). Assume w.1.o.g. ( OL,+ /I,)/2 > f . Let p, q be integers with q 6 2p and take p new random bits r’ = (r;, .... rb). Let sr, .... sy be different elementary conjunctions of the new random bits. Consider the circuit c’ given by T;‘+,+,Kt .... C”, ~1, .... sq), where c(= XT=, 1u.J and a’ is obtained by appending the weight (CI+ 1) q times to a. Then P(C(x, r, r’)= l)= P(C(x, r)= 1). P(3i<q: si= 1) P(C(x, r)= l).q/2p and a simple calculation shows that for some p = 0( -log(a, - pn)) and a suitable q, C’ gives advantage i(oz” - 8,) as required in the definition of RTC”. 1 THRESHOLD CIRCUITS OF BOUNDED DEPTH 139 Proof of Lemma 4.4. We describe a depth-2 probabilistic circuit for INNER PRODUCT MOD 2. Let x = (xi, .... x,,), y = (yi, .... y,), and let r = (r,, .... r,) be random bits. Consider the circuit T2”Gn-l(~l A yl A rl, .fl v Y1 v rl, .... x, A Y, A rn, -f, v Yn v r,). Observe that if xi= yi= 1 then xi A yi A ri = Xi v jji v ri = ri, otherwise xi A yi A ri=O, Xi v yi v ri= 1. If for (x, y) there are k indices i with xi= yj= 1, this fixes n-k O’s and n-k l’s for T2n-,. The remaining bits are generated randomly in pairs of O’s and 1’s. Thus if k is odd, we shall never have exactly n O’s and 1’s. Hence for such an input the probabilit of acce tance is 4. If k is even, the probability of having n O’s and n l’s is w jiP-2/nk > 2/nn, and the probability of having n - 2t O’s is equal to the probability of having n + 2t 0’s. Hence for such an input the probability of acceptance is f 4 - $$k. The proof is completed by applying Lemma 4.5. 1 In the remainder of this section we present some evidence that the simulation of probabilistic threshold circuits by deterministic ones must increase the depth in general. Let G be a group of permutations on { 1, .... n}. Then G induces a group G* of permutations on (0, 1 }“. DEFINITION. A,, B, E { 0, 1 }” are homogeneous if A,, and B, are the orbits of G* for some group of permutations G. For example, considering CONNECTIVITY of graphs, A, = {Hamilton cycles} and B, = (disjoint unions of two cycles of length n/2} are homogeneous sets of inputs, where G is the group of permutations of the edges induced by relabelings of vertices. THEOREM 4.6. Let d> 3 and let A,,, B, s (0, 1 }” be disjoint homogeneous sets. Then the following are equivalent: (a) A,, and B, can be separated by depth d+ 1 (deterministic) circuits with size and weight bounded by a polynomial, (b) there are depth d&,-discriminators bounded by a polynomial, for A,, and B, with size, weight, and 8;’ (c) there are depth-d probabilistic circuits giving advantage E, to A,, over B, such that the size, the weight, the number of random bits, and &;I are all bounded by a polynomial. Proof. (a) =S (b) is Lemma 3.3, (c) 2 (a) is Lemma 4.1, therefore we only have to prove (b) =S(c). Let C be an &,-discriminator for A, and B,, with PA,(C(x) = 1) = TV,, Psn(C(x) = 1) = /In. Assume w.1.o.g. E, = ~1,- /I,, > 0. For yeG let Cy denote the circuit computing C(y*(x)), where y* is the element 140 HAJNAL ET AL. of G* induced by y. Let P, be the uniform probability distribution on G. Then as G* is transitive on A, and B,, for every XEA, (resp. XE B,), PG(Ci’(x)= l)=cc,, (resp. P&C?(x) = 1) = fin). The Chernoff inequality implies as in Lemma 4.1, that there are permutations y,, .... yrn with m = O(nse2) such that if P& denotes the uniform probability distribution on (yi, .... y,}then for every x E A, (resp. x E B,), P~(C”(x) = 1) > ~1,- ~,,/3 (resp. Pb(CY(x) = 1) 6 pn + sJ3). Assume w.1.o.g. that m = 2’ and r,, .... Y, are random bits. Let C’ be a circuit such that c’(x I, ..-> x,, Tlr -.., r,) = C(yf(x)), where (ri, .... rl) is i in binary. Then applying Lemma 4.5 to center the probabilities CI, - &,/3 and fi, + ~,/3 around $, we obtain a circuit giving advantage E,112 to A, over B,. What remains is to describe the construction of C’. LEMMA 4.7. Let D be a threshold circuit of the form Tf(D,, .... D,) where all weights in fl are nonnegative and D* be a new circuit called the switch. Form a circuit D’ of the form Ty’(D,, .... D,, D*), where B’ is $ appended by a coefficient 1. Then if D* outputs 0, D’ behaves as D, otherwise it outputs constant 1. Proof Clear. 1 Continuing the proof of Theorem 4.6, let C be of the form Tz(C,, .... C,), where we assume w.1.o.g. that all weights are nonnegative. Let sr, .. .. s2f be all the elementary disjunctions of the random bits rl, .... rl. (Thus always exactly one of them is 0.) Let C{ be Ci equipped with switch sj (1 < i d m, 1 < j < 2’) and let C’ be of the form T::,,,-,,+,(Cf, .... CL, .... C:‘, .... C;,, where a’ is a repeated 2’ times and CI= Cy=, cli. (The Cfs have disjoint sets of gates but use the same random bits r,, .... r,). The correctness of the construction and the size bounds follow from the remarks preceding Lemma 4.7. The switch has depth 1, so the depth of C is not increased if it is at least 3. 1 We note that to prove (a) directly from (b) it suffices to take O(nep2) circuits C(y*(x)) with y chosen at random from G and apply a suitable threshold function to the outputs of these circuits. 5. CIRCUITS WITH UNRELIABLE THRESHOLD GATES A gate is called s-unreliable if with probability E it computes an incorrect output (E is called the unreliability of the gate). In a circuit of unreliable gates the gate THRESHOLDCIRCUITSOFBOUNDEDDEPTH 141 failures are assumed to be independent. The unreliability of different gates in a circuit may be different. C is a circuit of >&-unreliable gates if it consists of gates u that are s(u)-unreliable and E(U)> E for every u. C computes a function f with error probability 6 if for every x, C computes f(x) with probability B 1 - 6. We assume E<& First we consider bounded depth, unbounded fanin circuits with A , v , 1 gates. Let A(f) be the smallest number of subcubes from which {x: f(x) = 1 } can be obtained by union, intersection, and complementation. PROPOSITION 5.1. For every r and 6 > 0 there is an E > 0 such that every function with A(f) < r can be computed by a depth 6r + 4 circuit of A , v , 1 gates with error probability 6, for every assignment of unreliabilities < E to its gates. f Proof If A(f) dr, f can be written in the form g(h,, .... hz,), where hi (1 < i < 2r) is a conjunction or disjunction of some of the variables oft Let F’ be a formula of depth 6 2r computing g( yr , .... yZr) with fanin d 2 [ 193 and let F be obtained from F’ by replacing the variables yi by hi. Let G be a subfotmula of F of depth i computing a function fc. We claim that there is a circuit CG of depth 3i - 2 which computes fG with error probability 6 for every assignment of unreliabilities 6 E to the gates (where E depends on 6). For i= 1 this is obvious. For i > 1 the fanin is < 2, so we can use induction as in Pippenger [23, Theorem 2.41 (compute fG three times from its subcircuits and take majority, requiring two additional levels). (If F is levelled then the circuit constructed will also be levelled.) 1 Note that in the construction above the size of the circuit computing f is also bounded by a function of r. Now we can prove a converse of the proposition above which shows that if a function f can be computed reliably then A(f) is bounded by a function of the depth. (Here we do not need to assume that the circuit is levelled.) THEOREM 5.2. For every E >O and 6 > 0, if f is any function computable by a depth d circuit of > e-unreliable A , v , 1 gates with error probability f - 6 then A(f) = d°Cd). Proof probability Let C be a depth d circuit of > c-unreliable gates computing f with error $ - 6. Let s be a number satisfying (1 -&)s+’ .sd-*<& An edge of C is a red edge if its tail is a gate (not an input variable). We construct a circuit C’ from C as follows: starting from the output gate upwards, replace every A -gate (resp. v -gate) with red indegree > s by an A -gate (resp. v -gate) 142 HAJNAL ET AL. with a single constant 0 (resp. 1) input (the new gate has the same unreliability the original one); delete superfluous gates. as LEMMA 5.3. For every input x, IP(C(x)=l)-P(C’(x)= l)/ <6. Let u be an A -gate (the proof for v gates follows similarly). Let v, (t > S) be the inputs of o and assume that if ui is input to u, then i<j. Then VI 3 .*., Proof: P(& c”,(x)=l)=fi, P(c”,b)=l[;~ Q(1 -&)‘<(l C,l(r)=l) -&)s+‘, as for every input, every gate outputs 0 with probability 3 E. The number of new gates in C’ is 6 sdp 2, as new gates occur in depth > 2, and every gate in C’ has red indegree <s. Hence, if B denotes the event that all replaced A -gates (resp. v -gates) output a 0 (resp. a 1) in C, then P(B)>l-(l-~)“+‘s~-~>l-& But P(C(x) = 11B) = P(C’(x) = 11 B), so the iemma follows by considering and B. 1 the event C(x) = 1 under conditions B c’ contains < sdp ’ gates. The variables input to each gate u correspond to a subcube or its complement. Let the subcubes be Q,, .... Q, (t d sd- ‘). If A(f) > sd- ‘, then there are y and z such that f(y) = 1, f(z) = 0, and for every i,yEQiifandonlyifzEQ,.ThenP(C(y)=1)>$+6, P(C(z)=l)<$-6,and JP(c(y)=1-P(c(z)=1)~6lP(c(y)=1)-P(c’(y)=1)( +IP(c’(y)=l)-P(C’(z)=l)[ +IP(C’(z)=l)-P(C(z)=1)1<26, using the lemma above (the middle term is 0), a contradiction. by a computation showing s = O(d log d). 1 The bound follows Theorem 5.2 can be used to give lower bounds to the depth required for the reliable computation of several simple functions in AC”. As an example we mention EQUALITY, i.e., the function A’=, (xi= y,). COROLLARY 5.4. For every E > 0, and 6 > 0, every circuit of 2 &-unreliable A , v , 1 gates computing EQUALITY with error probability i- 6 has depth B(log n/ log log n). 143 THRESHOLD CIRCUITSOFBOUNDEDDEPTH Proof By Theorem 5.2 it is suffkient to show A(EQUALITY)Zn. Let Qi, .... Qm be subcubes of the 2n-dimensional cube generating {(x, x): x E (0, 1)“). Each Qi can be written as Qr x Qy, where Q: (resp. Qy) is a subcube in the first (resp. second) n dimensions. If for some x1 #x1 it holds that xi E Qi o x2 E QI for every i= 1, .... m, then we get a contradiction, considering (xi, xi) and (x,, x,). Hence m 2 n. 1 Now we turn to circuits of unreliable threshold gates. We show that in this case bounded depth circuits can be used for reliable computation. In Theorem 5.5 we use the assumption that each gate has the same unreliability. THEOREM 5.5. Let f be a function computed by a threshold circuit C of depth d, size s, and weight w. Then for every E < $, f is computed by a depth-d circuit C’ of E-unreliable threshold gates with error probability 2~, such that c’ has size O(w 3(d- ‘)s~(~- ‘j2), and weight w. This follows from the following lemma. LEMMA 5.6. Let F be a threshold formula of depth d, size s, and weight w computing a function f Then for E< i andfor every t, f can be computed by a depth-d threshold formula F’ of E-unreliable gates with error probability 6, where 6 = E + 1Jt such that F’ has size 0(( (ws)~ + (log(ts))3)“-‘) and weight w. Proof By induction on d, for d = 1 the statement is obvious. For d > 1 let F be of the form Tz(F,, .... F,), where CI= (c(i, .... a,) and Fi computes f, (1 < i<m). Let Fi.,j be disjoint formulas of s-unreliable gates computing fi with error probability 6’ for 1 < i < m, 1 d j 6 N, where 6’ and N will be specified later. Let F’= TY’(F;,,, .... F;,N, .... FL,,, .... FL,,), where a’ = (~1i , .... c1i , .... a,,, , .... a,), i.e., each F: is repeated N times. We claim that for some choice of Z, F’ computes F with error probability 6. For a fixed input x let the random variable yi, j be the output of Fi, j. The y,ls are independent as the Fi,js are disjoint. Let I0 := {i:f.(x)=O}, I, := {i: fi(x)= l} and cr*=Cy=i ai. Iffi(x)=O (resp.fi(x)=l) then E<P(yi,j=l)<h’ (resp. 1-6’<P(~,,~=l)< 1 -E). The final gate in F’ evaluates the sum N N f C “iYi,j= i=l j=l The Chernoff inequality f’ (see [‘i’, 321) implies that for iEZO, EN - N213< f j=l 511/46/2-2 N C @i C Yi,j+ 1 cli 1 Yi.j. i EIO .I= 1 ie II ,=l y,,, < 6’N + ~213 2 1 _ ze - 2~“” 144 HAJNAL ETAL. and for i E I,, P ( (1-6’)N-N2’3< 5 y;,i,<(1-&)N+N2’3 > j=l 2 1 - 2e+y thus P a< f i=l f >1-22me-2N”3, txiyi,i<b j=l where a= c ai(&N-N2’3)+ ie fo =(EN-N213) f c Uj((l-s’)N-N2’3) ie II q+N. 1 @,(1-E-6’) i=l it If =(EN-N2~3).C1*+(1-&-J’)N. f cxifi(X) i=l and c u,((~-E)N+N”~) b= c cri(G’N+N2’3)+ is IO =(6’N+N2’3) IE I, f a;+N. i=l =(6’N+N2’3)a*+(1-&-S’)N. c i&(1-&-X) ie II f c$fi(x). i= 1 A simple calculation shows that if E < i, 6’ <E + 1/4ws and N> (8~s)~ then there isanintegerIbetween(6’N+N2’3)~*+(1-&-~’)N(k-1)and(&N-N2’3)~*+ (1 -E-S’) Nk. With this integer 1, F’ computes f with error probability E + 2me-2N’13 which is smaller than the required E + l/t if N> (i In(2tm))3. The bound on the size of F’ follows by induction. 1 Proof of Theorem 5.5. To construct C’, first transform C into a formula of size O(sd- ‘) and then apply Lemma 5.6 with t = EC’. 1 As a corollary we obtain that functions in TC” can be computed reliably with bounded depth, polynomial size circuits of unreliable threshold gates. COROLLARY 5.7. Let (fN),EN E TC” and E < $. Then for some d and polynomial p there is a sequence of depth-d circuits (C,), EN of E-unreliable gates, such that C, computes f, with error probability 2&, and the size and the weight of C, are at most P(n ). Proof: Is clear from Theorem 5.5. fl THRESHOLD CIRCUITS OF BOUNDED DEPTH 145 Finally we mention one result for the general case of threshold circuits with E(U)unreliable gates u, where E(U)E [0, E,,,] may vary with V. We show that one can simulate a threshold circuit C by a circuit of the same depth with s(u)-unreliable gates, E(V)E [0, E,,,], provided that E,,, is very small. More precisely we require that Emax< l/%x, where a is the maximal sum of weight CT! 1 lail for gates T; in C. Although this condition requires, in general, that .smaxbecome smaller when n grows, the result is of some interest in the case of threshold circuits C with a = o(n) (e.g., if all weights in C are constants and the fanin is o(n)). THEOREM 5.8. Let f be a function computed by a threshold circuit C of depth d, size s, and weight w. Further, assume that a := CT= 1 lail <a,, for every gate Tt in C. Then for E,,, := 1/8a, there is a circuit c’ of depth d, weight w, and size 0( ((8a, + log( 2sd- ‘/E~,,))~ . sd- l)d- ’ ) s.t. C’ computes f with error probability 6 knax for every assignment of unreliabilities E(U)E [O, E,,,] to the gates v in c’. Proof. By the observation before Theorem 2.3 we may assume that all weights ai in circuit C are positive. As in the proof of Theorem 5.5 we First replace C by the corresponding threshold formula c of size O(sd- ‘). Then one proves a variation of Lemma 5.6 for an arbitrary threshold formula c of depth d, size sd- ‘, weight w, and Cy! 1 Iai( <a,, for every gate in 2;. The same construction as in Lemma 5.6 yields for E,,, < 1/8a, and N > (8a0)3 + (log 2sd- l/s,,,)’ a threshold formula C’ with the desired properties (in the proof we now use the more general version of the Chernoff inequality for a sum of random variables with different expected values, see Spencer [32]). 1 6. CIRCUITS WITH IMPRECISE THRESHOLD GATES Let T;(Y,, .... Y,,,),a = (aI, .... am), be a threshold gate. When CyZ, ajyj is ‘near to the threshold k, a small error in the evaluation may change the output. In this section we assume that the gate makes no error if lCyZ I aj yj - k( is larger than some sensitivity parameter S(a), whereas the output is unpredictable otherwise. Fix a function S: N + N, which is called the sensitivity function. DEFINITION. T>‘(y,, .... y,) S-imprecise threshold gate if (a) (b) (c) (where a = (aI, .... a,,,) and a =x7= i Iail) is an when CyE 1 ai yi > k + S(a), it outputs 1; when Cy= i aj yi < k - S(a), it outputs 0; otherwise it outputs 0 or 1. A computation of a circuit of S-imprecise gates is legal if the outputs of every gate satisfy (a) and (b). Thus the behavior of such a circuit is not completely determined as it may have several different legal computations. A circuit of S-imprecise 146 HAJNAL ET AL. gates computes a function f precisely if every legal computation gives the correct value. First we observe that under some conditions on S, S-imprecise gates can be used for precise computation. PROPOSITION6.1. Zf S(2/?a) </l S-imprecise threshold gate then the threshold gate T;( y,, .... yII,) and the give the same output for every input and for every legal computation of the imprecise gate. Proof: If CT= i ~r,y, B k (resp. Cy=, aiyi d k - 1) then XT= i 2BCriyi B p(2k - 1) + S(~~CY) (resp. CT! i fiCl,yi < b(2k - 1) - S(2@)). In the natural special case S(a) =&a fanin less than l/2.5 can be simulated by built using this simulation have large general, small depth circuits of imprecise tion. The lower bounds hold for circuits For a function f, let 1 this shows that, e.g., majority gates with imprecise gates. Circuits of imprecise gates depth. The bounds below imply that, in gates cannot be used for precise computawhich are not necessarily levelled. c(f, x) := I{i:f(x)#f(x”‘)}l (where xc’) is x changed in the ith component), 4f 1 :=max{U x and x)} be the critical complexity off: For a circuit C of S-imprecise gates let the sensitivity ratio of C be T2S is a gate of C THEOREM 6.2. f precisely, then Zf C is a depth-d circuit of S-imprecise gates computing a function 4f 16 usPad. Proof: If v is a gate of C then C,(x) denotes the output of v assuming that every gate of C works correctly. For every input x we define the threshold circuit C” as follows: (1) the graph of C” and the weights in each gate are the same as in C, (2) if v is a gate of C of the form T2S and C,(x)=0 (resp. C,(x)= 1) then the threshold value at v is k + S(U) (resp. k - S(a)). THRESHOLD CIRCUITS OF BOUNDED DEPTH 147 Thus C” is a “standard” threshold circuit. Note that a computation of C” on any input is a legal computation of C. Cz( y) is the output of u in C” for input y. If v corresponds to an input variable xi we write C,(y) = Cc(y) = yi. LEMMA 6.3. Let v be a gate of C of the form T>S and vl, .... v, be the gates input to v. Then for every inputs x and y if C,(x) # CE (y) then Proof. Assume C,(x) = 1, C:(y) =0 (the other case follows the same way). Then f ajCo,(x) 2 k j=l and the definition of C” implies that f aj’Z,(Y)<k-S(a). I j=l Let h’: := i IC,(x)i=l LEMMA 6.4. C;(x”‘)l. h: d (I,( C))depth(“). Proof. By induction on the depth of u. If depth(v) = 1 then ur, .... u, are input variables xi,, .... xi,. Then n h; .S(a) f 1 JC,(x) - C:(x”‘)( i=l . f IC,W- qx(")l lql j=l G i i=l < t i= f j-1 1 Ic,(x)-c~,(x”‘)l 1 (j: $=i) lajl lajl=a, where the first inequality follows from the previous lemma and the third inequality follows from the fact that C,(x) # Ct (xci’) only if ij = i. 148 HAJNALETAL. If depth(u) = d > 1 then where the first inequality induction. 1 follows as above and the second inequality follows by Theorem 6.2 now follows from applying Lemma 6.4 to the input x maximizing CCL x). I Using results of Simon [28] and Turan [33] this leads to the following lower bounds. COROLLARY 6.5. Let E > 0 and S(a) = Ea. If f is an arbitrary nondegenerate Boolean function (resp. a nontrivial property of r-vertex graphs where n = (;)) and C is a depth-d circuit of S-imprecise gates computing f precisely then d = sZ(log log n) (resp. d = Q(log n)). If f is a nondegenerate Boolean function (resp. a nontrivial I property) then c(f) = sZ(log n) [28] (resp. c(f) = Q(&) [33]). Proof: graph The remark after Proposition 6.1 implies that the lower bound for graph properties cannot be improved, in general. For general sensitivity functions S(a) write S in the form S(a) = a”(“) (&(a) < 1). We assume that E(a) is a nondecreasing function (as, e.g., in the case of S(a) = a& and S(a) = a/log a). 1 THEOREM 6.6. (a) Zf lim, _ o. E(a) < 1 then for every (f,),,, N E TC”, f, can be computed by bounded depth circuits of S-imprecise gates with polynomial size and weight. E(a) = 1 then there is no sequence of nondegenerate Boolean (b) ?,f lim,,, functions (f, ), EN with log c(fn) = Q(log n) which can be computed by circuits as in (a). (c) If 1 -&(a) = o(log log a/log a) then there is no sequence of nondegenerate Boolean functions (f,,),, EN which can be computed by circuits as in (a). ProoJ: (a) follows from Proposition 6.1, (b) and (c) follow from Theorem 6.2 similarly to Corollary 6.5. 1 THRESHOLD CIRCUITSOFBOIJNDED DEPTH 149 7. THRESHOLD QUANTIFIERS We fix a similarity type Y containing relations only. A, denotes a model on the ground set { 1, .... n} (all models considered are finite). A threshold quantifier is of the form #:(x1, .... x,), where r is the arity of the quantifier and k is the threshold value. Formulas are built from atomic formulas using A , v , -I and threshold quantifiers ( v and A of any number of formulas is allowed; #;(xi, .... x,) binds x1, .... x,). The interpretation of the quantifiers is the following: A, I= # ;(xl, .... x,1 $(x1, .... x,) * I ((4 9 .‘., a,): A, I= $(a~, .... a,)>1 2 k. (Thus 3 and V are special cases.) Let d be a class of structures and dfi be the class of structures in d on the ground set ( 1, .... n}. A sentence #n defines Z& if for every A, it holds that A sequence @ = (dn)neN defines d if 4, defines Z& for every n. The quantifier rank (QR), depth (D), and size (S) of formulas is defined inductively. For atomic formulas each measure is 0. For $ = 1 $, QR(4) = QR($), Wd)=WICI)+ 1, W)=S(ti)+ 1. For #=h v ... v A,,, QW)=maxl.ism QR(h), D(4)= l+maxlGiGm QR(4J, S(4)= l+Cy=i S($J (similarly for 4=#, A...A #,,,). For 4 = # Lti, QR(4) = QR($) + r, D(4) = D($) + r, S(4) = St+) + 1. @ = (4,JnsN is of bounded quantifier rank (resp. depth) and polynomial size, if for some d and polynomial p, QR(d,) < d (resp. D(4,) d d) and S($,) < p(n) for every n. Structures are encoded as usual (an r-ary relation on (1, .... n} by nr bits). A class d of structures corresponds to a sequence (f;;“), EN of Boolean functions, where f< is the characteristic function of &, (having p(n) variables for some polynomial p). The relationship between threshold circuits and threshold quantifiers is given by the following proposition. PROPOSITION 7.1. The sequence (ff ), EN is in TC” if and only if d is defined by a @ of bounded depth and polynomial size, where the language of @ contains a new relation symbol P and the interpretation of P is fixed for every n. Proof (Outline). We consider the “only if” part as the other direction is straightforward. The argument is a modification of the proof of the analogous result for AC” and first-order logic in Gurevich and Lewis [ 141 and Immerman [ 161. Let C, be a bounded depth, polynomialsize threshold circuit computing f f. We may assume that all gates are majority gates. The gates, edges, and paths of C, can be encoded by s-tuples over (1, .... n} for some constant s and thus the description of C, can be encoded into a single relation of constant arity over { 1, . ... n>. The 150 HAJNAL ET AL. desired (Pi giving the output of C, is built recursively, using this relation, from formulas describing the output of gates on previous levels. 1 Proposition 7.1 shows that undefinability results imply lower bounds for threshold circuits. Undefinability with special relations for P imply lower bounds for “structured” threshold circuits. Below we give an example. Let the similarity type 9’ contain R(x, y) (adjacency) only, and the additional relation with a fixed interpretation be s(x, y) always interpreted as the standard successor relation on { 1, .... a}). Let CONNECTIVITY := {G = (V, E, S): G’ = (V, E) is a connected undirected graph, S is as above}. THEOREM 7.2. CONNECTIVITY bounded quantgier rank. is not definable by any @ (over Y as above) of First we state a general lemma about threshold quantifiers of arity > 1. LEMMA 7.3. If s(u*,is defined by a sentence 4 then it is also defined by a sentence I++of the same quantlyier rank such that $ contains only wary threshold quantlyiers. Proof For simplicity we show how to eliminate a binary threshold quantifier # :(x,, x2). Let d(‘) = (d”’, , .... dt’) (i = 1, .... N) be all possible degree sequences of directed graphs on n vertices having k edges (without multiple edges but with loops allowed). (I.e., xi”=, dj’) .j = k, and dj” is the number of vertices with outdegree j.) Then -4 I=i= \jI ( /1 #:,,,(X,)(#f(XZ)(P(X,,X2)) > ,=l where ti, j = C;= j dj”. > 1 (We note that the above transformation may increase the size exponentially.) The proof of Theorem 7.2 uses a variant of the FrdissbEhrenfeucht games introduced by Immerman and Lander [7]. Given G, = (V,, E,, S,) and G, = ( Vz, El, S,), players I and II play m moves. In move i, (1) player I selects a structure G,, (I, = 1 or 2) and a set Vi! s V,,, (2) player II selects a set Vi-,, E V,-, in the other structure I q = I G-,,I, (3) player I chooses v!-“E Vipl,, (4) player II chooses VIE Vi<. with After m moves, II wins if the bijection v: o vf (1 6 i<m) is an isomorphism between {vi, .... vk} E V, and {UT, .... vi} c V,; otherwise I wins. G, zm Gz if II has a winning strategy in the game. THRESHOLD CIRCUITS OF BOUNDED 151 DEPTH Let G, and G2 be structures of size n and 4 be a sentence with unary threshold quantifiers such that QR($) 6 m. LEMMA 7.4. Zf G1 zm G2 then G, k 4 o G2 k 4. Proof. Omitted. It is analogous to the proofs of similar statements for other versions of the Frdisst-Ehrenfeucht game. 1 Proof of Theorem 7.2. From Lemmas 7.3 and 7.4 it suffices to show that for every m there are structures G1 and G, such that G, E CONNECTIVITY, G2 4 CONNECTIVITY and G, So G,. The structures G, = (I/, , E,, S,), G2 = (V,, E,, S,) are defined as follows: I/,=I/,={l,...,n},n~2”+5; S, = S, are the standard successor relation; (The two structures are shown in Fig. 2). The winning strategy of player II is given as follows: Let the vertices chosen in the first i moves be {u:, .. .. u!} in GI and {u:, .... uf} in Gz. Assume that (a) the mapping isomorphism between w~++w~ (l<j<lO) A; = {w;, ...) w:(), ?I;, ...) ui’} and (see Fig.2) Af= ~J’t)uf {wf, ...) w&l;, (l<j<i) is an ...) u;>, (b) if a,, b, and u2, b, are corresponding vertices in A t (resp. A f) then either bI - a, = b, -a2 (i.e., the distances in the successor relation are the same) or 2 m+l-i<min(jb,-a,l, (b2-aJ). Claim. If i< m, player II can maintain assumptions (a) and (b) after the (i + 1)th move. (Clearly the claim implies the existence of a winning strategy (both assumptions hold for i = O).) To prove the claim, consider interval I,= [a- 2m-i, a + 2”-j] for every u&4: UAf. The assumptions imply that if for al, b, E Ai, Ia, n Z,, #0, then bl -a, = b,--a2 (and thus, in particular, Z,,nZ,,#0), where u2, b2 are 152 HAJNAL ET AL. f “1 1111111111 “2 y5 y4 y5 v6 y7 ya *9 w10 FIGURE 2 the vertices corresponding to a,, b,. Therefore UaeAl Z, and u,,,;Z, form isomorphic sets of intervals (considering lengths) and ’ I ( 1, .... n) - U,, A; Z,l = I { 1, .... 4 - Laf &I. Now assume that in the (i + 1) th move I selects Vi,+ ’ in Gt . Then II selects P’:’ ’ in G2 as follows: he “imitates” the choice of Z in UoSA; Z, by choosing corresponding vertices in U,, Aiz Z, and selects the same number of vertices outside these intervals arbitrarily. The assumptions guarantee that any pair of vertices selected outside the intervals defined above will maintain the assumptions for is 1. After this choice the choice of II in the last phase of step i+ 1 is clear. 1 We note that the proof, in fact, implies a log, n - 0( 1) lower bound to the quantifier rank of any sentence with threshold quantifiers which defines CONNECTIVITY for n vertex graphs. Thus for this property threshold quantifiers are not more powerful than 3 and V. 8. SOME OPEN PROBLEMS The most important open problems about threshold circuits are to separate the classes TCS and to show TC” 5 NC’ (candidates showing this are mentioned in the Introduction).It would also be interesting to show that CONNECTIVITY is not in TC”. There are open problems concerning circuits of depth 2. There are no lower bounds known for depth-2 circuits of (a) gates computing symmetric functions (note that by Theorem 2.3 these circuits are simulated by depth-3 threshold circuits of weight 1 with quadratic increase in size), THRESHOLD CIRCUITS OF BOUNDED DEPTH 153 (b) threshold gates with arbitrary weights (these circuits are called Gambaperceptrons in [20], where the problem of proving a lower bound is also posed). There are also no lower bounds known for probabilistic threshold circuits of depth 2. Does Corollary 5.7 remain true if the circuits C, are required to compute f, reliably for every assignment of unreliabilities <E to the gates (where E does not depend on n as in Theorem 5.8)? Are there any nondegenerate Boolean functions computable in depth o(log n) by circuits of imprecise threshold gates with sensitivity function S(a) = EC(? (A candidate would be the “addressing function” [28] which has critical complexity m% n1.J It would also be interesting to extend Theorem 7.2 to graphs with a linear order on the vertices. REFERENCES 1. M. AJTAI, Zi formulae on finite structures, Ann. Pure Appl. Logic 24 (1984), l-48. 2. M. AITAI AND M. BEN-OR, A theorem on probabilistic constant depth computations, in “Proceedings, 16th Annu. ACM Sympos. Theory of Comput., 1984,” pp. 471474. 3. L. BABAI, P. FRANKL, AND J. SIMON, Complexity classes in communication complexity, in “IEEE 27th Annu. Found of Comput. Sci., 1986,” pp. 337-347. 4. D. A. BARRINGTON,Bounded-width, polynomial size branching programs recognize exactly those languages in NC’, in “18th Annu. Proceedings, ACM Sympos Theory of Comput., 1986,” pp. l-5. 5. D. A. BARRINGTONAND D. TI-&RIEN, Finite monoids and the fine structure of NC’, in “19th Annu. Proceedings, ACM Sympos. Theory of Comput., 1987,” pp. 101-109. 6. P. W. BEAME, S. A. COOK, AND H. J. HOOVER,Log depth circuits for division and related problems, SIAM J. Cornput. 15 (1986), 994-1003. 7. B. BOLLOB~, “Random Graphs,” Academic Press, New York, 1985. 8. A. K. CHANDRA, L. J. STOCKMEYER,AND U. VISHKIN, Constant depth reducibility, SIAM J. Comput. 13 (1984), 423-439. 9. B. CHOR AND 0. GOLDREICH, Unbiased bits from sources of weak randomness and probabilistic communication complexity, in “26th Annu. IEEE Found. of Comput. Sci., 1985,” pp. 429442. 10. R. L. DOBRUSHIN AND S. J. ORTYUKOV, Upper bound for the redundancy of self-correcting arrangements of unreliable functional elements, Problems Inform. Transmission 13 (1977), 59-65. 11. P. ERD~S AND J. SPENCER,“Probabilistic Methods in Combinatorics,” Akad. Kiadb, Budapest, 1974. 12. R. FAGIN, M. M. KLA~E, N. J. PIPPENGER,AND L. J. STOCKMEYER,Bounded depth, polynomial size circuits for symmetric functions, Theorel. Cornput. Sci. 36 (1985), 239-250. 13. M. FURST, J. B. SAXE, AND M. SIPSER,Parity, circuits, and the polynomial time hierarchy, in “22nd Annu. IEEE Found. of Comput. Sci., 1981,” pp. 260-270. 14. Y. GUREVICHAND H. R. LEWIS, A logic for constant depth circuits, Inform. and Control 61 (1984), 65-74. 15. J. HASTAD, Almost optimal lower bounds for small depth circuits, in “18th Annu. Proceedings, ACM Sympos. Theory of Comput., 1986,” pp. 6-20. 16. N. IMMERMAN,Languages which capture complexity classes, in “15 th Annu. Proceedings, ACM Sympos. Theory of Comput., 1983,” pp. 347-354. 17. N. IMMERMANAND E. LANDER, Telling graphs apart: A first order approach to graph isomorphism, preprint, 1984. 154 HAJNAL ET AL. 18. 0. B. LUPANOV, Implementing the algebra of logic functions in terms of bounded depth formulas in the basis +, *, -, Dokl. Acad. Nuuk. SSSR 136 (1961), 5, 1041-1042 (Engl. transl. Sov. Phys. Dokl. 6 (1961), 107-108). 19. W. F. MCCOLL AND M. S. PATERSON,The depth of all Boolean functions, SIAM J. Cornput. 6 (1977), 373-380. 20. M. MINSKY AND S. PAPERT, “Perceptrons: An Introduction to Computational Geometry,” MIT Press, Cambridge, MA, 1972. 21. J. VON NEUMANN, Probabilistic logics and the synthesis of reliable organisms from unreliable components, in “Automata Studies” (C. E. Shannon and J. McCarthy, Eds.), pp. 43-98, Princeton Univ. Press, Princeton, NJ, 1956. 22. I. PARBERRYAND G. SCHNITGER,Parallel computation with threshold functions, in “Structure in Complexity Theory” (A. L. Selman, Ed.), Lect. Notes in Comput. Sci., Vol. 223, pp. 272-290, Springer-Verlag, New York/Berlin, 1986. 23. N. PIPPENGER,On networks of noisy gates, in “26th Annu. IEEE Found. of Comput. Sci., 1985,” pp. 30-38. 24. A. A. RAZBOROV, Lower bounds for the monotone complexity of some Boolean functions, Dokl. Akad. Nuuk. 281 (1985), 798-801. 25. A. A. RAZBOROV, Lower bounds for the size of circuits of bounded depth with basis { A, @}, preprint; Mat. Zumefki 41 (1987), 598-607. English translation in: Math. Notes of the Academy of Sciences of the USSR 41, 333-338. 26. J. H. REIF, On threshold circuits and polynomial computation, in “Proceedings, 2nd Structure in Complexity Theory Conf., 1987,” pp. 118-125. 27. D. E. RUMELHARDT,J. L. MCCLELLAND, AND THE PDP RESEARCHGROUP, “Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, MIT Press, Cambridge, MA, 1986. 28. H. U. SIMON,A tight sZ(log log n) bound on the time for parallel RAM’s to compute nondegenerate Boolean functions, in “Found. in Comput. Theory, 1983,” Lecture Notes in Computer Science, Vol. 158, pp. 151-153, Springer-Verlag, New York/Berlin, 1983. 29. S. SKYUM, A measure in which Boolean negation is exponentially powerful, Inform. Process. Left. 17 (1983), 125-128. 30. S. SKYUM AND L. G. VALIANT, A complexity theory based on Boolean algebra, J. Assoc. Compuf. Much. 32 (1985), 484-502. 31. R. SMOLENSKY, Algebraic methods in the theory of lower bounds for Boolean circuit complexity, in “19th Annu. Proceedings, ACM Sympos. Theory of Comput., 1987,” pp. 77-82. 32. J. SPENCER, “Ten Lectures on the Probabilistic Method,” SIAM, Philadelphia, 1987. 33. GY. TURIN, The critical complexity of graph properties, Inform. Process. Letf. 18 (1984), 151-153. 34. A. C. YAO, Separating the polynomial-time hierarchy by oracles, in “26th Annu IEEE Found. of Comput. Sci., 1985,” pp. l-10.