Graduate Studies Program

Term: Fall 2022/2023

Computing

Lecture 4
Graphs and Greedy Algorithms (Draft)

This lecture notes have been compiled from different resources,

I’d like to thank all authors who make them available to use.

1
 Lecture Outline
◼ Graphs

◼ Graph Representation

◼ Greedy Strategy Algorithms

◼ Greedy Strategy

◼ Shortest Path Problem

◼ Single Source All Destinations

◼ Minimum Spanning Tree

2
 Graphs
◼ A graph is a pictorial representation of a set of objects where
some pairs of objects are connected by links.
◼ The interconnected objects are represented by:
◼ points termed as vertices,
◼ links that connect the vertices are called edges.
◼ Graphs provide a way to represent relationships between data.
◼ A graph is a series of nodes connected with edges.
◼ Formally, a graph is a pair of sets (V, E), where V is the set of
vertices and E is the set of edges, connecting the pairs of
vertices.
◼ In this graph,
◼ V = {a, b, c, d, e}
◼ E = {ab, ac, bd, cd, de}

3
 Terminology and Notations
◼ Graph(G = (V,E)):
◼ A set of objects called "Nodes (Vertices) (V)" and "Edges
(Arcs) (E)" that connect the vertices
◼ Node:
◼ A data element of the graph
◼ Edge
◼ A path between two nodes
◼ Adjacent nodes:
◼ If there is a direct edge that connects the nodes
◼ Neighbors of a Node:
◼ All nodes that are adjacent to it.

4
 Graph Applications
◼ Graphs can model both real-world systems and abstract
problems, so they are used in hundreds of applications.
◼ Sampling of graphs applications:
1. Modeling connectivity in computer and communications networks.
2. Representing a map as a set of locations with distances between
locations; used to compute shortest routes between locations.
3. Modeling flow capacities in transportation networks.
4. Finding a path from a starting condition to a goal condition; for
example, in artificial intelligence problem solving.
5. Modeling computer algorithms, showing transitions from one
program state to another.
6. Finding an acceptable order for finishing subtasks in a complex
activity, such as constructing large buildings.
7. Modeling relationships such as family trees, business or military
organizations, and scientific taxonomies.
5
 Graph Applications
◼ Many problems can be naturally formulated in terms of graphs
◼ Relationships between states or cities
◼ Printed circuit boards
◼ Departments in a corporation
◼ Switching stations in a power network
◼ Graph structure
◼ A simplification of hierarchical structures, which in turn, is a
simplification of a linear structure
◼ Has no notation of either a first or root node, or last or leaf
node.

6
 Graph Applications
◼ Airline flight scheduling:
◼ What is the shortest distance between two airports?
◼ Planning time- schedules for investment projects
◼ What is the shortest time for a specific job could be
done?
◼ Testing integrated circuit boards:
◼ Are two points on a board connected?

7
Types of Graphs

◼ Connected
◼ All nodes are reachable from
all other nodes

◼ Disconnected
◼ One or more nodes are not
reachable from one or more
other nodes
 Types of Graphs
◼ Directed Graph (Digraph):
◼ If an edge between two nodes has a direction.
◼ Tree is the special case of a directed graph that is connected
and has no cycles (Acyclic).
◼ Directed path:
◼ Is a sequence of directed edges between two nodes
◼ Strongly Connected Digraph:
◼ If for any two nodes I & J in the graph, there is a directed
path from I to J
◼ Weakly Connected Digraph:
◼ If for any two nodes I & J there is a directed path from I to J
or J to I (but not both)

9
 Types of Edges 1
◼ Undirected
◼ An edge goes to and

A B
from a node
◼ An edge can be taken
both directions
◼ Edges show
connections
C ◼ Directed
◼ Edges can specifically
A B go to or from nodes
◼ Edges can still be
bidirectional
◼ Edges can show a one

way trip, visibility,

C etc..
Types of Edges 2

◼ Un-weighted
◼ Edges have no values
◼ Edges just show some sort
of connection such as
possible paths, etc..

◼ Weighted
◼ Edges have a value attached
to them
◼ These values could be
distance, cost, time, etc...
 Graph Representation
◼ Adjacency Matrix
◼ Adjacency Lists
▪ Linked Adjacency Lists

▪ Array Adjacency Lists

12
 Graph Representation

◼ Adjacency Matrix
◼ 0/1 n x n matrix, where n = # of vertices
◼ A(i,j) = 1 iff (i,j) has an edge, otherwise 0

2
3
1 2 3 4 5
1 1 0 1 0 1 0
4 2 1 0 0 0 1
5 3 0 0 0 0 1
4 1 0 0 0 1
5 0 1 1 1 0
13
 Adjacency Matrix Properties

• Diagonal entries are zero.

• Adjacency matrix of an undirected graph is symmetric.
• A(i,j) = A(j,i) for all i and j.
• Diagonal entries are zero.
• Adjacency matrix of a digraph might not be symmetric.
14
 Adjacency Matrix
◼ n2 bits of space
◼ For an undirected graph, may store only lower or
upper triangle (exclude diagonal).
▪ (n-1)n/2 bits
◼ O(n) time to find vertex degree and/or vertices
adjacent to a given vertex.

15
 Linked Adjacency Lists

◼ Each adjacency list is a chain.

Array Length = n
# of chain nodes = 2e (undirected graph)
# of chain nodes = e (digraph)

16
 Array Adjacency Lists
◼ Each adjacency list is an array list.

2 aList[1] 2 4
3
[2] 1 5
1 [3] 5
[4] 5 1
4
5 aList[5] 2 4 3

Array Length = n
# of list elements = 2e (undirected graph)
# of list elements = e (digraph)

17
 Graph Storage
◼ The storage requirements for the adjacency list depend on both
the number of edges and the number of vertices in the graph.
◼ There must be an array entry for each vertex (even if the vertex
is not adjacent to any other vertex and thus has no elements on
its linked list), and each edge must appear on one of the lists.
◼ Thus, the cost is O(V + E).

18
 Sparse Matrices
• Sparse graph – relatively few edges

• Dense graph – lots of edges

• Complete graph – contains all possible edges

• sparse … many elements are zero

• dense … few elements are zero

19
 Unstructured Sparse Matrices
Airline flight matrix:
▪ airports are numbered 1 through n
▪ flight(i,j) = list of nonstop flights from airport i to airport j
▪ Let n = 1000
▪ n x n array of list references => 4 million bytes

Web page matrix:

web pages are numbered 1 through n
web(i,j) = number of links from page i to page j

Web analysis:
authority page … page that has many links to it
hub page … links to many authority pages

20
 Representation of Unstr. Sparse Matrices
Single linear list in row-major order:
• scan the nonzero elements of the sparse matrix in row-
major order.
• each nonzero element is represented by a triple
• (row, column, value)
• the list of triples may be an array list or a linked list
(chain)
• Example
list =
0 0 3 0 4 row 1 1 2 2 4 4
0 0 5 7 0
0 0 0 0 0 column 3 5 3 4 2 3
0 2 6 0 0
value 3 4 5 7 2 6

21
 Array Linear List Representation

row 1 1 2 2 4 4
list = column 3 5 3 4 2 3
value 3 4 5 7 2 6

element 0 1 2 3 4 5
row 1 1 2 2 4 4
column 3 5 3 4 2 3
value 3 4 5 7 2 6
22
 Chain Representation
- Node structure:
row col
value next

- Single Chain row 1 1 2 2 4 4

list = column 3 5 3 4 2 3
value 3 4 5 7 2 6

1 3 1 5 2 3 2 4 4 2 4 3
3 4 5 7 2 6 null

firstNode
23
 Array of Row Chains

Node structure
null
next
3 3 5 4
col value
null
3 5 4 7

0 0 3 0 4
0 0 5 7 0 null
0 0 0 0 0
0 2 6 0 0 null
2 2 3 6

row[]

24
 Storage Complexity

500 x 500 matrix with 1994 nonzero elements?

- 2D array 500 x 500 x 4 = 1million bytes

- Single Array List 3 x 1994 x 4 = 23,928 bytes
- One Chain Per Row 23928 + 500 x 4 = 25,928

25
 Greedy Strategy
◼
Optimization problems
A greedy algorithm works in phases. At each phase:
◼ The best is taken, without regard for future consequences
◼ By choosing a local optimum at each step, we hope to end up

at a global optimum.
◼ A “greedy algorithm” sometimes works well for optimization
problems.
◼ An optimization problem is one in which we want to find, not just
a solution, but the best solution
◼ Kruskal’s algorithm for finding a minimum-cost spanning tree
◼ Always tries the lowest-cost remaining edge
◼ Prim’s algorithm for finding a minimum-cost spanning tree
◼ Always takes the lowest-cost edge between nodes in the
spanning tree and nodes not yet in the spanning tree
◼ Dijkstra’s algorithm for finding the shortest path in a graph
◼ Always takes the shortest edge connecting a known node to
an unknown node.

26
 Shortest Path Problems
❑ Directed weighted graph.

❑ Path length is sum of weights of edges on path.

❑ Vertex at which the path begins is the source, ends is the

destination.

1. Single source single destination.

2. Single source all destinations.

3. All pairs (every vertex is a source and destination).

• Single source single destination.

❑ Possible greedy algorithm:
❑ Leave source vertex using cheapest/shortest edge.
❑ Continue until destination is reached.

27
 Example

8 6
1
2 3
16
3 1
7 5
6 4 10
4
2 4 7
5 3

14

A path from 1 to 7.

Path length is 14.

28
 Example

8 6
1
2 3
3 1
16
6 7 4 5
10
4
2 4 7
5 3

14

Another path from 1 to 7,

Path length is 11.

29
 Greedy Shortest 1 To 7 Path

8 6
1
2 3
3 1
16 7 5
6 4 10
4
2 4 7
5 3

14

Path length is 12.

Not shortest path. Algorithm doesn’t work!
30
 Single-Source Shortest Path Problem
◼ Single-Source Shortest Path Problem –
◼ The problem of finding shortest paths from a source vertex v to
all other vertices in the graph.

31
 Dijkstra's Algorithm
◼ Dijkstra's algorithm - is a solution to the single-source
shortest path problem in graph theory.
◼ Works on both directed and undirected graphs. However, all
edges must have nonnegative weights.

◼ Approach: Greedy

◼ Input: Weighted graph G={E,V} and source vertex v ∈V, such

that all edge weights are nonnegative

◼ Output: Lengths of shortest paths (or the shortest paths

themselves) from a given source vertex v ∈V to all other
vertices

32
 The author: Edsger Wybe Dijkstra
◼ May 11, 1930 – August 6, 2002

◼ Received the 1972 A. M. Turing Award, widely considered the most

prestigious award in computer science.

◼ The Schlumberger Centennial Chair of Computer Sciences at The

University of Texas at Austin from 1984 until 2000
◼ http://www.cs.utexas.edu/~EWD/

33
 Dijkstra's Algorithm
◼ Dijkstra’s algorithm finds the shortest paths from a given node
to all other nodes in a graph
◼ Initially,

◼ Mark the given node as known (path length is zero)

◼ For each out-edge, set the distance in each neighboring

node equal to the cost (length) of the out-edge, and set

its predecessor to the initially given node
◼ Repeatedly (until all nodes are known),

◼ Find an unknown node containing the smallest distance

◼ Mark the new node as known

◼ For each node adjacent to the new node, examine its

neighbors to see whether their estimated distance can

be reduced (distance to known node plus cost of out-
edge)
◼ If so, also reset the predecessor of the new node

34
 Dijkstra's algorithm - Pseudocode

dist[s] ←0 (distance to source vertex is zero)

for all v ∈ V–{s}
do dist[v] ←∞ (set all other distances to infinity)
S←∅ (S, the set of visited vertices is initially empty)
Q←V (Q, the queue initially contains all vertices)
while Q ≠∅ (while the queue is not empty)
do u ← mindistance(Q,dist) (select the element of Q with the min. dist.)
S←S∪{u} (add u to list of visited vertices)
for all v ∈ neighbors[u]
do if dist[v] > dist[u] + w(u, v) (if new shortest path found)
then d[v] ←d[u] + w(u, v) (set new value of shortest path)
return dist

35
Dijkstra Algorithm Example

36
Dijkstra Algorithm Example

37
Dijkstra Algorithm Example

38
Dijkstra Algorithm Example

39
Dijkstra Algorithm Example

40
Dijkstra Algorithm Example

41
Dijkstra Algorithm Example

42
Dijkstra Algorithm Example

43
Dijkstra Algorithm Example

44
Dijkstra Algorithm Example

45
 Applications of Dijkstra's Algorithm
- Traffic Information Systems are most prominent use
- Mapping (Map Quest, Google Maps)
- Routing Systems

46
 Applications of Dijkstra's Algorithm
 Prof. Lauren Meyers (Biology
Dept.) uses networks to model the
spread of infectious diseases and
design prevention and response
strategies.
 Vertices represent individuals,
and edges their possible contacts.
It is useful to calculate how a
particular individual is connected
to others.
 Knowing the shortest path
lengths to other individuals can be
a relevant indicator of the
potential of a particular individual
to infect others.
47
 Single Source All Destinations
Need to generate up to n (n is number of vertices)
paths (including path from source to itself).
Greedy method:
▪ Construct these up to n paths in order of
increasing length.
▪ Assume edge costs (lengths) are >= 0.
▪ So, no path has length < 0.
▪ First shortest path is from the source vertex to
itself. The length of this path is 0.

48
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16
6 7 4 5 10
4
2 4 7
5 3
14
Path Length 1 2 6
1 0
1 3 5 4 9
1 3 2
1 3 6 10
1 3 5 5 1 3 6 7 11 49
 Greedy Single Source All Destinations
Path Length • Each path (other
1 0 than first) is a one
edge extension of a
1 3 2 previous path.
•Next shortest path
1 3 5 5 is the shortest one
6 edge extension of an
1 2
already generated
shortest path.
1 3 5 4 9

1 3 6 10

1 3 6 7 11
50
 Greedy Single Source All Destinations
◼ Let d(i) (distanceFromSource(i)) be the length of a
shortest one edge extension of an already
generated shortest path, the one edge extension
ends at vertex i.
◼ The next shortest path is to an as yet unreached
vertex for which the d() value is least.
◼ Let p(i) (predecessor(i)) be the vertex just before
vertex i on the shortest one edge extension to i.

51
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16
6 7 4 5 10
4
2 4 7
5 3
1 14
[1] [2] [3] [4] [5] [6] [7]
d 0 6 2 16 - - 14
p - 1 1 1 - - 1

52
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16 7 5
6 4 10
4
2 4 7
5 3
1 14
1 3 [1] [2] [3] [4] [5] [6] [7]
d 0 6 2 16 55- 10- 14
p - 1 1 1 3- 3- 1

53
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16
6 7 4 5 10
4
2 4 7
5 3
1 14
1 3 [1] [2] [3] [4] [5] [6] [7]
1 3 5 d 0 66 2 169 5- 10 - 14
p - 1 1 51 3- 3- 1

54
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16
6 7 4 5
10
4
2 4 7
5 3
1 14
1 3 [1] [2] [3] [4] [5] [6] [7]
1 3 5 d 0 6 2 99 5- 10- 14
p - 1 1 5 3- 3- 1
1 2

55
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16
6 7 4 5 10
4
2 4 7
5 3
1 14
1 3 [1] [2] [3] [4] [5] [6] [7]
1 3 5 d 0 6 2 9 5- 10 - 14
12
p - 1 1 5 3- 3- 1 4
1 2
1 3 5 4
56
 Greedy Single Source All Destinations

8 6
1
2 3
3 1
16
6 7 4 5
10
4
2 4 7
5 3
14
1 3 6 [1] [2] [3] [4] [5] [6] [7]
d 0 6 2 9 5- 10- 11 12
1
p - 1 1 5 3- 3- 1 464

57
 Greedy Single Source All Destinations

Path Length
1 0
1 3 2
1 3 5 5
[1] [2] [3] [4] [5] [6] [7]
1 2 6
0 6 2 9 5- 10
- 14
12
11
1 3 5 4 9 - 1 1 5 3- -
3 41
6

1 3 6 10

1 3 6 7 11
58
 Time Complexity
◼ O(n) to select next destination vertex.
◼ O(out-degree) to update d() and p() values when
adjacency lists are used.
◼ O(n) to update d() and p() values when adjacency
matrix is used.
◼ Selection and update done once for each vertex to
which a shortest path is found.
◼ Total time is O(n2 + e) = O(n2).

59
 Time Complexity
◼ When a min heap of d() values is used in place of
the linear list L of reachable vertices:
◼ Total time is O((n+e) log n), because O(n) remove
min operations and O(e) change key (d() value)
operations are done.

◼ When a Fibonacci heap is used:

◼ The total time is O(nlog n + e).

60
A spanning tree

◼ A spanning tree is a sub-graph

of another graph that connects
all the nodes of the graph
without forming a cycle(a loop)

◼ The graph must be connected

and be undirected
◼ Cost of spanning tree is sum of
edge costs
 A minimum spanning tree?
◼ If a graph has weights for its
edges, then a minimum
spanning tree is the spanning
tree that gives the lowest total
sum of the edges.

◼ Basically, it connects all of the

nodes together for the least total
weight

◼ A minimum spanning tree can be used for several things:

◼ Identifying Clusters
◼ Determining a central point of operations
For Example:

◼ Determining the
location of roads
◼ Where they should
be placed
◼ What cities they
should pass through
For Example:

◼ Railroad Tracks
◼ Where is a good
central hub
◼ What would cost us
the least and be the
quickest
 For Example:
◼ Airports and Flight paths
◼ Where airports should be
◼ What path a flight should take
 Example

8 10 14
1 3 5 7
7 4 12 6 3
2
2 4 6 8
9

◼ Network has 8 nodes with 10 edges.

◼ Spanning tree has only n - 1 = 7 edges.
◼ Need to either select 7 edges or discard 3.

66
 Edge Selection Greedy Strategy

◼ Start with an n-vertex 0-edge forest. Consider edges

in ascending order of cost. Select edge if it does not
form a cycle together with already selected edges.
▪ Kruskal’s method.
◼ Start with a 1-vertex tree and grow it into an n-
vertex tree by repeatedly adding a vertex and an
edge. When there is a choice, add a least cost edge.
▪ Prim’s method.

67
 Kruskal’s Method

8 10 14
1 3 5 7 1 3 5 7
7 4 12 6 3
2
2 4 6 8 2 4 6 8
9

◼ Start with a forest that has no edges.

• Consider edges in ascending order of cost.

• Edge (1,2) is considered first and added to
the forest.

68
 Kruskal’s Method

8 10 14 3
1 3 5 7 1 5 7
7 3 7 4 6 3
2 4 12 6 2
2 4 6 8 2 4 6 8
9

 Edge (7,8) is considered next and added.

• Edge (3,4) is considered next and added.
• Edge (5,6) is considered next and added.
• Edge (2,3) is considered next and added.
• Edge (1,3) is considered next and rejected because it
creates a cycle.
69
 Kruskal’s Method

8 10 14 10 14
5 1 3 5 7
1 3 7
7 3 7 4 6 3
2 4 12 6 2
2 4 6 8 2 4 6 8
9

◼ Edge (2,4) is considered next and rejected because it creates

a cycle.
• Edge (3,5) is considered next and added.
• Edge (3,6) is considered next and rejected.
• Edge (5,7) is considered next and added.

70
 Kruskal’s Method

8 10 14 10 14
1 3 5 7 1 3 5 7
7 3 7 4 6 3
2 4 12 6 2
2 4 6 8 2 4 6 8
9

◼ n - 1 edges have been selected and no cycle formed.

◼ So we must have a spanning tree.
◼ Cost is 46.
◼ Min-cost spanning tree is unique when all edge costs are
different.

71
 Pseudocode For Kruskal’s Method

Start with an empty set T of edges.

while (E is not empty && |T| != n-1)
{
Let (u,v) be a least-cost edge in E.
E = E - {(u,v)}. // delete edge from E
if ((u,v) does not create a cycle in T)
Add edge (u,v) to T.
}
if (| T | == n-1) T is a min-cost spanning tree.
else Network has no spanning tree.

72
 Prim’s Method

8 10 14 10 14
1 3 5 7
1 3 5 7
7 3 7 4 6 3
2 4 12 6 2
4 6 8 2 4 6 8
2
9

◼ Start with any single vertex tree.

• Get a 2-vertex tree by adding a cheapest edge.
• Get a 3-vertex tree by adding a cheapest edge.
• Grow the tree one edge at a time until the tree has n - 1 edges
(and hence has all n vertices).

73
 Time Complexity

◼ Prim’s method.

▪ O(n2) using priority queue.

▪ O(e + n log n) using a Fibonacci heap.

◼ Kruskal’s uses union-find trees to run in O(n + e log e).

74