Probabilistic Method 6
Probabilistic Method 6
In this lecture we will consider two further problems that can be solved using probabilistic
algorithms. In the first half, we will consider the problem Polynomial Identity Testing (PIT),
and in the second half we will consider the problem of finding a global minimum cut in a graph.
33
The problem PIT is not as trivial as it may appear at first sight. One might be tempted
to take the polynomial p, expand it, and then check if all of the coefficients are equal to 0.
However, expanding the polynomial may take exponential time, as illustrated by the following
example.
where the sum is over the permutations of [n].The n × n-Vandermonde matrix is the matrix
X12 . . . X1n−1
1 X1
1 X2 X22 . . . X2n−1
V (n) = . .. . (6.5)
.. ..
.. . . .
1 Xn Xn2 . . . Xnn−1
Q
It is known that the determinant of the Vandermonde matrix is given by 1≤i<j≤n (Xj − Xi ).
Therefore, the polynomial
n
σ(i)−1
X Y Y
sgn(σ) Xi − (Xj − Xi ) (6.6)
σ∈Sn i=1 1≤i<j≤n
is the zero-polynomial. Expanding the double product requires an exponential number of compu-
tations.
No polynomial time algorithm for solving PIT is known (and finding such an algorithm will
have important consequences in complexity theory, see [KI03]).
1.2. Schwartz-Zippel lemma. The following result, due to Schwartz [Sch80] and Zip-
pel [Zip79], forms the core of the probabilistic algorithm for solving PIT.
where p̂ is a polynomial of degree at most deg(p) − K that does not depend on Xn+1 , and is not
the zero-polynomial. Let A be the event that p̂(X1 , X2 , . . . , Xn ) = 0. Then, by the induction
hypothesis,
deg(p) − K
P (A) ≤ . (6.9)
|S|
34
Conditioning on A, we find that
P (p(R1 , R2 , . . . , Rn+1 ) = 0) = P (p(R1 , R2 , . . . , Rn+1 ) = 0 | A) P (A)
+ P p(R1 , R2 , . . . , Rn+1 ) = 0 | Ā P Ā
(6.10)
≤ P (A) + P p(R1 , R2 , . . . , Rn+1 ) = 0 | Ā
deg(p) − K K deg(p)
≤ + = ,
|S| |S| |S|
where we use (6.9) to bound P (A), while the second term can be bounded by conditioning
on R1 , R2 , Rn as well as Ā, noting that after this conditioning p is a polynomial of degree K
in the variable Xn+1 only, so that we can apply the n = 1-case.
The Schwartz-Zippel lemma immediately gives a probabilistic algorithm for PIT: draw ran-
dom elements R1 , R2 , . . . , Rn uniformly at random from a large set |S| and compute p(R1 , R2 , . . . , Rn ).
If the polynomial evaluates to 0, return YES; otherwise return NO.
Note that the algorithm always returns YES if the polynomial p is the zero-polynomial. If p
is not the zero-polynomial, then the probability that the algorithm mistakenly returns YES is
d
at most |S| . There are two ways of making the error probability smaller: making S larger, or
repeating the algorithm a number of times.
1.3. Two further complexity classes. In this section, we introduce three new complex-
ity classes: RP (for randomised polynomial time), co-RP, and ZPP (zero-error probabilistic
polynomial time).
Definition 6.2. A decision problem is said to be in RP if there exists a polynomial time
algorithm such that
• if the correct answer is NO, it always returns NO;
• if the correct answer is YES, it returns YES with probability at least 1/2.
This means that if the algorithm returns YES, we are sure that this is the right answer. On
the other hand, if the algorithm returns NO, it may be the case that the answer actually should
be YES. The class co-RP is similar to RP, but with the roles of YES and NO reversed.
Definition 6.3. A decision problem is said to be in RP if there exists a polynomial time
algorithm such that
• if the correct answer is YES, it always returns YES;
• if the correct answer is NO, it returns NO with probability at least 1/2.
Using the Schwarz-Zippel lemma, it is easy to show that PIT is in co-RP, provided that the
underlying field is large enough.
Definition 6.4. A decision problem is said to be in ZPP if there exists a polynomial time
algorithm that returns either YES, NO, or DO NOT KNOW, such that
• it outputs DO NOT KNOW or the correct answer;
• it outputs DO NOT KNOW with probability at most 1/2.
It is not hard to show that ZPP = RP ∩ co-RP.
1.4. Application: Perfect matchings in bipartite graphs. A matching in a graph G
is a subset of the edges, such that no two edges share an end point. A matching is called a
perfect matching or 1-factor if it does not leave any vertex unmatched, i.e. every vertex is
incident to exactly one edge in the matching.
In this section, we will look at perfect matchings in bipartite graphs. If G is a bipartite
graph with bipartition classes A and B, then a necessary condition for the existence of a perfect
matching is that A and B contain the same number of elements (in which case the graph is
called equibipartite). Indeed, a perfect matching can be seen as a bijection φ : A → B, such
35
that each {a, φ(a)}, a ∈ A is an edge. The decision version of bipartite perfect matching is the
following.
Problem: Min-Cut
Input: A graph G
Output: A cut of minimum size in G
We have seen min-cuts before in the context of minimum s, t-cuts. In that variant of
the problem, we distinguish two vertices s and t (often called the source and the sink), and
36
the objective is to find a cut (S, V \ S) of minimum size that separates s and t (i.e. s ∈ S
and t ∈ V \ S). By replacing each edge in G by two directed edges, the min-s, t-cut problem
can be solved easily by relating it to a max-flow problem on the same graph.
The global min-cut problem can be solved efficiently by fixing some vertex s, varying t over
the remaining vertices, and solving the corresponding min-s, t-cut problems.
Lemma 6.6. Min-cut can be solved in polynomial time.
In this lecture, we will see a randomised algorithm for solving Min-cut, known as Karger’s
min-cut algorithm, which was first published in [Kar93].
Repeat until |V | = 2:
• Pick a random edge e;
• Replace G ← G/e.
The idea of the algorithm is to contract random edges, until two vertices remain. By
Lemma 6.7, these two vertices correspond to a cut in the original graph, which is then output.
Hopefully, the output is in fact a min-cut in the original graph.
The key in the analysis is that we can find a lower bound on the probability that the output
in fact is a min-cut. By repeating the algorithm a suitable number of times, and taking the
smallest cut that is returned, we can make the probability of finding a min-cut as large as we
want.
Recall that if A1 and A2 are events, then
P (A1 ∩ A2 ) = P (A1 ) P (A2 | A1 ) . (6.15)
The following lemma generalises this equality to intersection of more than two events.
Lemma 6.8. Let A1 , A2 , . . . , An be events, then
n
!
\
P Ai = P (A1 ) P (A2 | A1 ) P (A3 | A1 , A2 ) . . . P (An | A1 , A2 , . . . , An−1 ) . (6.16)
i=1
We can now prove the main result of this section. We say that a cut survives the procedure,
if none of its edges is contracted during the execution.
Theorem 6.9. Let C = (S,
n
V \ S) be a minimum cut in G. The probability that C survives
the procedure is at least 1/ 2 .
37
Proof. Let Ai be the event that the cut C survives the first i contractions, i = 1, 2, . . . , n−2.
Note that A1 ⊇ A2 ⊇ . . . ⊇ An−2 . We are interested in the probability that C survives the
first n − 2 contractions, that is
n−2
!
\
P (An−2 ) = P Ai = P (A1 ) P (A2 | A1 ) . . . P (An−2 | A1 , A2 , . . . , An−3 ) . (6.17)
i=1
We bound each of the probabilities on the right-hand side from below. To start with note that C
survives the first contraction if the randomly selected edge is not one of the edges crossing C.
Let κ = |C| be the size of the minimum cut. As the edges adjacent to a single vertex form a
cut, it follows that the degree of every vertex is at least κ. Therefore, the number of edges in G
is at least 21 κn. It follows that
κ n−2
P (A1 ) ≥ 1 − 1 = . (6.18)
2 κn
n
Next, note that conditional on C surviving the first i contractions, C is still a minimum cut
in the multigraph. As the number of edges now is at least 12 κ(n − i), it follows that
κ n−i−2
P (Ai+1 | A1 , A2 , . . . , Ai ) ≥ 1 − 1 = . (6.19)
2 κ(n − i) n−i
Upon combining (6.18)–(6.19) with (6.17), we obtain the desired bound
n−2n−3 1 2 1
P (An−2 ) ≥ ... = = n . (6.20)
n n−1 3 n(n − 1) 2
According to the theorem, each run of the algorithm produces a min-cut with probability
at least 1/ n2 . By repeating the algorithm ln 2 n2 times, and taking the smallest cut returned,
Bibliography
[Kar93] David R. Karger. “Global Min-cuts in RNC, and Other Ramifications of a Simple
Min-Cut Algorithm”. In: SODA. Vol. 93. 1993, pp. 21–30.
[KI03] Valentine Kabanets and Russell Impagliazzo. “Derandomizing polynomial identity
tests means proving circuit lower bounds”. In: Proceedings of the thirty-fifth annual ACM symposium
ACM. 2003, pp. 355–364.
[Sch80] Jack Schwartz. “Fast probabilistic algorithms for verification of polynomial identi-
ties”. In: Journal of the ACM 27.4 (1980), pp. 701–717.
[Zip79] Richard Zippel. “Probabilistic algorithms for sparse polynomials”. In: Proceedings of the Internationa
1979, pp. 216–226.
38