CS 332: Algorithms: Dijkstra's Algorithm Disjoint-Set Union

CS 332: Algorithms
Dijkstras Algorithm
Disjoint-Set Union
Homework 5
Check web page this afternoon
Review: Bellman-Ford Algorithm

BellmanFord()
for each v V
d[v] = ;
d[s] = 0;
for i=1 to |V|-1
for each edge (u,v) E
Relax(u,v, w(u,v));
for each edge (u,v) E
if (d[v] > d[u] + w(u,v))
return no solution;
Initialize d[], which

will converge to
shortest-path value
Relaxation:
Make |V|-1 passes,
relaxing each edge
Test for solution:
have we converged yet?
Ie, negative cycle?
Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w
Review: DAG Shortest Paths

Problem: finding shortest paths in DAG
Bellman-Ford takes O(VE) time.
Do better using topological sort.
Idea: if were lucky and processes vertices on each shortest
path in order, B-F would be done in one pass

Every path in a dag is subsequence of topologically sorted
vertex order, so processing verts in that order, we will do
each path in forward order (will never relax edges out of
vert before doing all edges into vert).
Thus: just one pass. Running time: O(V+E)
Review: Dijkstras Algorithm

If no negative edge weights, we can beat BF
Similar to breadth-first search
Grow a tree gradually, advancing from vertices
taken from a queue

Also similar to Prims algorithm for MST
Use a priority queue keyed on d[v]
Dijkstras Algorithm
Dijkstra(G)
B
2
10
for each v V
4
3
A
D
d[v] = ;
d[s] = 0; S = ; Q = V;
5
1
C
while (Q )
u = ExtractMin(Q); Ex: run the algorithm
S = S U {u};
for each v u->Adj[]
if (d[v] > d[u]+w(u,v))
Relaxation
Note: this
d[v] = d[u]+w(u,v);
Step
is really a
call to Q->DecreaseKey()
Dijkstras Algorithm
Dijkstra(G)
How many times is
for each v V
ExtractMin() called?
d[v] = ;
d[s] = 0; S = ; Q = V;
How many times is
while (Q )
u = ExtractMin(Q); DecreaseKey() called?
S = S U {u};
for each v u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
What will be the total running time?
Dijkstras Algorithm
Dijkstra(G)
How many times is
for each v V
ExtractMin() called?
d[v] = ;
d[s] = 0; S = ; Q = V;
How many times is
while (Q )
u = ExtractMin(Q); DecraseKey() called?
S = S U {u};
for each v u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
A: O(E lg V) using binary heap for Q

Can acheive O(V lg V + E) with Fibonacci heaps
Dijkstras Algorithm
Dijkstra(G)
for each v V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q )
u = ExtractMin(Q);
S = S U{u};
for each v u->Adj[]
if (d[v] > d[u]+w(u,v))
d[v] = d[u]+w(u,v);
Correctness: we must show that when u is

removed from Q, it has already converged
Correctness Of Dijkstra's Algorithm

p2
u
s
p2
Note that d[v] (s,v) v

Let u be first vertex picked s.t. shorter path than d[u]
Let y be first vertex V-S on actual shortest path from su
d[u] > (s,u)

d[y] = (s,y)
Because d[x] is set correctly for y's predecessor x S on the shortest path, and
When we put x into S, we relaxed (x,y), giving d[y] the correct value
Correctness Of Dijkstra's Algorithm

p2
u
s
p2
Note that d[v] (s,v) v

Let u be first vertex picked s.t. shorter path than d[u] d[u] > (s,u)
Let y be first vertex V-S on actual shortest path from su
d[y] = (s,y)
d[u] > (s,u)
= (s,y) + (y,u) (Why?)
= d[y] + (y,u)
d[y]
But if d[u] > d[y], wouldn't have chosen u. Contradiction.
Disjoint-Set Union Problem

Want a data structure to support disjoint sets
Collection of disjoint sets S = {Si}, Si Sj =
Need to support following operations:

MakeSet(x): S = S U {{x}}
Union(Si, Sj): S = S - {Si, Sj} U {Si U Sj}
FindSet(X): return Si S such that x Si
Before discussing implementation details, we
look at example application: MSTs
Kruskals Algorithm
Kruskal()
{
T = ;
for each v V
MakeSet(v);
sort E by increasing edge weight w
for each (u,v) E (in sorted order)
if FindSet(u) FindSet(v)
T = T U {{u,v}};
Union(FindSet(u), FindSet(v));
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
9
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
1?
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2?
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
9
14
17
T = ;
8
25
5?
for each v V
MakeSet(v);
21
13
1
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8?
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
9?
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
1
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13?
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14?
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17?
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19?
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21?
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25?
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Kruskals Algorithm
Run the algorithm:
Kruskal()
2
19
{
14
17
T = ;
8
25
5
for each v V
MakeSet(v);
21
13
T = T U {{u,v}};
}
Correctness Of Kruskals Algorithm

Sketch of a proof that this algorithm produces an MST
for T:
Assume algorithm is wrong: result is not an MST
Then algorithm adds a wrong edge at some point
If it adds a wrong edge, there must be a lower weight edge
(cut and paste argument)

But algorithm chooses lowest weight edge at each step.
Contradiction
Again, important to be comfortable with cut and paste
arguments
Kruskals Algorithm
What will affect the running time?
Kruskal()
{
T = ;
for each v V
MakeSet(v);
T = T U {{u,v}};
}
Kruskals Algorithm
What will affect the running time?
Kruskal()
1 Sort
{
O(V) MakeSet() calls
T = ;
O(E) FindSet() calls
for each v V
O(V) Union() calls
(Exactly how many Union()s?)
MakeSet(v);
T = T U {{u,v}};
}
Kruskals Algorithm: Running Time

To summarize:
Sort edges: O(E lg E)
O(V) MakeSet()s
O(E) FindSet()s
O(V) Union()s
Upshot:
Best disjoint-set union algorithm makes above 3
operations take O(E(E,V)), almost constant

Overall thus O(E lg E), almost linear w/o sorting
Disjoint Set Union

So how do we implement disjoint-set union?
Nave implementation: use a linked list to
represent each set:
MakeSet(): ??? time

FindSet(): ??? time
Union(A,B): copy elements of A into B: ??? time
Disjoint Set Union

So how do we implement disjoint-set union?
Nave implementation: use a linked list to represent each
set:
MakeSet(): O(1) time

FindSet(): O(1) time
Union(A,B): copy elements of A into B: O(A) time
How long can a single Union() take?

How long will n Union()s take?
Disjoint Set Union: Analysis

Worst-case analysis: O(n2) time for n Unions
Union(S1, S2)
copy
1 element
Union(S2, S3)
copy
2 elements
Union(Sn-1, Sn) copy
n-1 elements
O(n2)
Improvement: always copy smaller into larger

Why will this make things better?
What is the worst-case time of Union()?
But now n Unions take only O(n lg n) time!
Amortized Analysis of Disjoint Sets

Amortized analysis computes average times without
using probability
With our new Union(), any individual element is copied
at most lg n times when forming the complete set from
1-element sets
Worst case: Each time copied, element in smaller set
1st time
resulting set size
2nd time
4
(lg n)th time n
Amortized Analysis of Disjoint Sets

Since we have n elements each copied at most lg n
times, n Union()s takes O(n lg n) time

We say that each Union() takes O(lg n) amortized
time
Financial term: imagine paying $(lg n) per Union
At first we are overpaying; initial Union $O(1)
But we accumulate enough $ in bank to pay for later
expensive O(n) operation.

Important: amount in bank never goes negative
The End

CS 332: Algorithms: Dijkstra's Algorithm Disjoint-Set Union

Uploaded by

Copyright:

Available Formats

CS 332: Algorithms: Dijkstra's Algorithm Disjoint-Set Union

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CS 332: Algorithms: Dijkstra's Algorithm Disjoint-Set Union

Uploaded by

Copyright:

Available Formats

CS 332: Algorithms

Review: Bellman-Ford Algorithm

Initialize d[], which

Relax(u,v,w): if (d[v] > d[u]+w) then d[v]=d[u]+w

Review: DAG Shortest Paths

path in order, B-F would be done in one pass

Review: Dijkstras Algorithm

taken from a queue

What will be the total running time?

A: O(E lg V) using binary heap for Q

Correctness: we must show that when u is

Correctness Of Dijkstra's Algorithm

Note that d[v] (s,v) v

d[u] > (s,u)

Correctness Of Dijkstra's Algorithm

Note that d[v] (s,v) v

Disjoint-Set Union Problem

Need to support following operations:

Before discussing implementation details, we

look at example application: MSTs

Correctness Of Kruskals Algorithm

(cut and paste argument)

Kruskals Algorithm: Running Time

operations take O(E(E,V)), almost constant

Disjoint Set Union

represent each set:

MakeSet(): ??? time

Disjoint Set Union

MakeSet(): O(1) time

How long can a single Union() take?

Disjoint Set Union: Analysis

Union(Sn-1, Sn) copy

Improvement: always copy smaller into larger

But now n Unions take only O(n lg n) time!

Amortized Analysis of Disjoint Sets

(lg n)th time n

Amortized Analysis of Disjoint Sets

times, n Union()s takes O(n lg n) time

expensive O(n) operation.

You might also like