Trees
Trees
Trees
Tutorial -1-Study Material (Trees & Graph)
Index
Topics Page No
1) Trees
a) Binary/ N-ary Trees 2-5
b) Binary Search Tree 6-10
c) Heaps/ Priority Queues 11-31
2) Graphs
a) Graph Representation 32-41
b) Breadth First Search 42-53
c) Depth First Search 54-58
d) Minimum Spanning Tree 59-66
e) Shortest Path Algorithm 67-70
f) Flood-Fill Algorithm 71-74
g) Articulation Points and Bridges 75-83
h) Biconnected Components 84-92
i) Strongly Connected Components 93-99
j) Topological Sort 100-106
k) Hamiltonian Path 107-115
l) Maximum flow 116-121
m) Minimum Cost Maximum Flow 122-124
n) Min-Cut 125-127
1
Topic-1 - Trees
A binary tree is a structure comprising nodes, where each node has the
following 3 components:
As the name suggests, the data element stores any kind of data in the
node.
The left and right pointers point to binary trees on the left and right side of
the node respectively.
2
Commonly-used terminologies
3
Creating nodes
Simple node
Pointer to a node
In this case, you must explicitly allocate the memory of the node type to the
pointer (preferred method).
4
Time complexity
O(n)
Application of trees
1. A Manipulate hierarchical data
2. Make information easy to search (see tree traversal)
3. Manipulate sorted lists of data
4. Use as a workflow for compositing digital images for visual effects
5. Use in router algorithms
Practice
Link to the Practice Problems:
Practice Questions
5
Binary Search Tree
For a binary tree to be a binary search tree, the data of all the nodes in the
left sub-tree of the root node should be ≤ the data of the root. The data of
all the nodes in the right subtree of the root node should be > the data of
the root.
Example
Also, considering the root node with data=5, its children also satisfy the
specified ordering. Similarly, the root node with data=19 also satisfies this
ordering. When recursive, all subtrees satisfy the left and right subtree
ordering.
6
There are mainly three types of tree traversals.
Pre-order traversal
Post-order traversal
7
In-order traversal
● The 'inorder( )' procedure is called with root equal to node with
data=10
● Since the node has a left subtree, 'inorder( )' is called with root equal
to node with data=5
● Again, the node has a left subtree, so 'inorder( )' is called with root=1
8
● Node with data=1 does not have a left subtree. Hence, this node is
processed.
● Node with data=1 does not have a right subtree. Hence, nothing is
done.
● inorder(1) gets completed and this function call is popped from the
call stack.
The stack is as follows:
Again, the node with data=6 has no left subtree, Therefore, it can be
processed and it also has no right subtree. 'inorder(6)' is then completed.
9
Both the left and right subtrees of node with data=5 have been completely
processed. Hence, inorder(5) is then completed.
The order in which BST in Fig. 1 is visited is: 1, 5, 6, 10, 17, 19. The
in-order traversal of a BST gives a sorted ordering of the data elements
that are present in the BST. This is an important property of a BST.
Insertion in BST
Algorithm
10
2. If the data of the root node is greater, and if a right subtree exists,
then repeat step 2 with root = root of right subtree. Else, insert
element as right child of current root.
Implementation
Practice
Link to the Practice Problems:
Practice Questions
Heaps/Priority Queues
Heaps
A heap is a tree-based data structure in which all the nodes of the tree are
in a specific order.
11
For example, if X is the parent node of Y, then the value of X follows a
specific order with respect to the value of Y and the same order will be
followed across the tree.
The maximum number of children of a node in a heap depends on the type
of heap. However, in the more commonly-used heap type, there are at
most 2 children of a node and it's known as a Binary heap.
In binary heap, if the heap is a complete binary tree with N nodes, then it
has smallest possible height which is log2N .
In the diagram above, you can observe a particular sequence, i.e each
node has greater value than any of its children.
Suppose there are N Jobs in a queue to be done, and each job has its own
priority. The job with maximum priority will get completed first than others.
At each instant, we are completing a job with maximum priority and at the
same time we are also interested in inserting a new job in the queue with
its own priority.
So at each instant we have to check for the job with maximum priority to
complete it and also insert if there is a new job. This task can be very easily
executed using a heap by considering N jobs as N nodes of the tree.
As you can see in the diagram below, we can use an array to store the
nodes of the tree. Let’s say we have 7 elements with values {6, 4, 5, 3, 2, 0,
1}.
12
Note: An array can be used to simulate a tree in the following way. If we
are storing one element at index i in array Arr, then its parent will be stored
at index i/2 (unless its a root, as root has no parent) and can be accessed
by Arr[i/2], and its left child can be accessed by Arr[2∗i] and its right child
can be accessed by Arr[2∗i+1]. Index of root will be 1 in an array.
Max Heap: In this type of heap, the value of parent node will always be
greater than or equal to the value of child node across the tree and the
node with highest value will be the root node of the tree.
13
Implementation:
Let’s assume that we have a heap having some elements which are stored
in array Arr. The way to convert this array into a heap structure is the
following. We pick a node in the array, check if the left sub-tree and the
right sub-tree are max heaps, in themselves and the node itself is a max
heap (it’s value should be greater than all the child nodes)
Complexity: O(logN)
Example:
In the diagram below,initially 1st node (root node) is violating property of
max-heap as it has smaller value than its children, so we are performing
max_heapify function on this node having value 4.
14
As 8 is greater than 4, so 8 is swapped with 4 and max_heapify is
performed again on 4, but on different position. Now in step 2, 6 is greater
than 4, so 4 is swapped with 6 and we will get a max heap, as now 4 is a
leaf node, so further call to max_heapify will not create any effect on heap.
Now as we can see that we can maintain max- heap by using max_heapify
function.
15
So you can see that elements 3, 2, 4, 5 are indexed by N/2+1 (i.e 4), N/2+2
(i.e 5 ) and N/2+3 (i.e 6) and N/2+4 (i.e 7) respectively.
Now let’s say we have N elements stored in the array Arr indexed from 1 to
N. They are currently not following the property of max heap. So we can
use max-heapify function to make a max heap out of the array.
How?
16
From the above property we observed that elements from Arr[N/2+1] to
Arr[N] are leaf nodes, and each node is a 1 element heap. We can use
max_heapify function in a bottom up manner on remaining nodes, so that
we can cover each node of tree.
Example:
Suppose you have 7 elements stored in array Arr.
Here N=7, so starting from node having index N/2=3, (also having value 3
in the above diagram), we will call max_heapify from index N/2 to 1.
17
In the diagram below:
In step 5, we finally get a max- heap and the elements in the array Arr will
be :
18
Min Heap: In this type of heap, the value of parent node will always be less
than or equal to the value of child node across the tree and the node with
lowest value will be the root node of tree.
As you can see in the above diagram, each node has a value smaller than
the value of their children.
We can perform same operations as performed in building max_heap.
First we will make function which can maintain the min heap property, if
some element is violating it.
19
Complexity: O(logN) .
Example:
Suppose you have elements stored in array Arr {4, 5, 1, 6, 7, 3, 2}. As you
can see in the diagram below, the element at index 1 is violating the
property of min -heap, so performing min_heapify(Arr, 1) will maintain the
min-heap.
20
Now let’s use above function in building min-heap. We will run the above
function on remaining nodes other than leaves as leaf nodes are 1 element
heap.
Example:
Consider elements in array {10, 8, 9, 7, 6, 5, 4} . We will run min_heapify
on nodes indexed from N/2 to 1. Here node indexed at N/2 has value 9.
And at last, we will get a min_heap.
21
Heaps can be considered as partially ordered tree, as you can see in the
above examples that the nodes of tree do not follow any order with their
siblings(nodes on the same level). They can be mainly used when we give
more priority to smallest or the largest node in the tree as we can extract
these node very efficiently using heaps.
APPLICATIONS:
1) Heap Sort:
22
Let’s say we want to sort elements of array Arr in ascending order. We can
use max heap to perform this operation.
Idea: We build the max heap of elements stored in Arr, and the maximum
element of Arr will always be at the root of the heap.
Processing:
Implementation:
Suppose there are N elements stored in array Arr.
23
Example:
In the diagram below,initially there is an unsorted array Arr having 6
elements. We begin by building max-heap.
After building max-heap, the elements in the array Arr will be:
24
Processing:
25
26
After all the steps, we will get a sorted array.
2) Priority Queue:
Example:
27
We can think of many ways to implement the priority queue.
Naive Approach:
Suppose we have N elements and we have to insert these elements in the
priority queue. We can use list and can insert elements in O(N)time and
can sort them to maintain a priority queue in O(NlogN) time.
Efficient Approach:
We can use heaps to implement the priority queue. It will take O(logN) time
to insert and delete each element in the priority queue.
Based on heap structure, priority queue also has two types max- priority
queue and min - priority queue.
Max Priority Queue is based on the structure of max heap and can perform
following operations:
maximum(Arr) : It returns maximum element from the Arr.
extract_maximum (Arr) - It removes and return the maximum element from
the Arr.
increase_val (Arr, i , val) - It increases the key of element stored at index i
in Arr to new value val.
28
insert_val (Arr, val ) - It inserts the element with value val in Arr.
Implementation:
Maximum :
Complexity: O(1)
Complexity: O(logN).
Increase Value: In case increasing value of any node, it may violate the
property of max-heap, so we may have to swap the parent’s value with the
node’s value until we get a larger value on parent node.
29
Complexity : O(logN).
Insert Value :
Complexity: O(logN).
Example:
Initially there are 5 elements in priority queue.
Operation: Insert Value(Arr, 6)
In the diagram below, inserting another element having value 6 is violating
the property of max-priority queue, so it is swapped with its parent having
value 4, thus maintaining the max priority queue.
30
Operation: Extract Maximum:
In the diagram below, after removing 8 and placing 4 at node 1, violates the
property of max-priority queue. So max_heapify(Arr, 1) will be performed
which will maintain the property of max - priority queue.
31
Topic - 2 - Graphs
Graph Representation
Types of nodes
● Root node: The root node is the ancestor of all other nodes in a
graph. It does not have any ancestor. Each graph consists of exactly
one root node. Generally, you must start traversing a graph from the
root node.
● Leaf nodes: In a graph, leaf nodes represent the nodes that do not
have any successors. These nodes only have ancestor nodes. They
can have any number of incoming edges but they will not have any
outgoing edges.
32
Types of graphs
33
Weighted: In a weighted graph, each edge is assigned a weight or cost.
Consider a graph of 4 nodes as in the diagram below. As you can see each
edge has a weight/cost assigned to it. If you want to go from vertex 1 to
vertex 3, you can take one of the following 3 paths:
● 1 -> 2 -> 3
● 1 -> 3
● 1 -> 4 -> 3
Therefore the total cost of each path will be as follows: - The total cost of 1
-> 2 -> 3 will be (1 + 2) i.e. 3 units - The total cost of 1 -> 3 will be 1 unit -
The total cost of 1 -> 4 -> 3 will be (3 + 2) i.e. 5 units
Cyclic: A graph is cyclic if the graph comprises a path that starts from a
vertex and ends at the same vertex. That path is called a cycle. An acyclic
graph is a graph that has no cycle.
A tree cannot contain any cycles or self loops, however, the same does not
apply to graphs.
Graph representation
You can represent a graph in many ways. The two most common ways of
representing a graph is as follows:
Adjacency matrix
Note: A binary matrix is a matrix in which the cells can have only one of two
possible values - either a 0 or 1.
The adjacency matrix can also be modified for the weighted graph in which
instead of storing 0 or 1 in Ai,j, the weight or cost of the edge will be stored.
35
In an undirected graph, if Ai,j = 1, then Aj,i = 1. In a directed graph, if Ai,j =
1, then Aj,i may or may not be 1.
36
Consider the directed graph given above. Let's create this graph using an
adjacency matrix and then show all the edges that exist in the graph.
Input file
4 // nodes
5 //edges
12 //showing edge from node 1 to node 2
24 //showing edge from node 2 to node 4
31 //showing edge from node 3 to node 1
34 //showing edge from node 3 to node 4
42 //showing edge from node 4 to node 2
Code
37
Output
Adjacency list
38
For a weighted graph, the weight or cost of the edge is stored along with
the vertex in the list using pairs. In an undirected graph, if vertex j is in list
Ai then vertex i will be in list Aj.
Note: A sparse matrix is a matrix in which most of the elements are zero,
whereas a dense matrix is a matrix in which most of the elements are
non-zero.
A1 → 2 → 4
A2 → 1 → 3
A3 → 2 → 4
A4 → 1 → 3
39
Consider the same directed graph from an adjacency matrix. The
adjacency list of the graph is as follows:
A1 → 2
A2 → 4
A3 → 1 → 4
A4 → 2
Consider the directed graph given above. The code for this graph is as
follows:
Input file
4 // nodes
5 //edges
12 //showing edge from node 1 to node 2
24 //showing edge from node 2 to node 4
31 //showing edge from node 3 to node 1
34 //showing edge from node 3 to node 4
42 //showing edge from node 4 to node 2
Code
40
Output
Practice
Link to the Practice Problems: Practice Questions
41
Breadth First Search
Graph traversals
Graph traversal means visiting every vertex and edge exactly once in a
well-defined order. While using certain graph algorithms, you must ensure
that each vertex of the graph is visited exactly once. The order in which the
vertices are visited are important and may depend upon the algorithm or
question that you are solving.
During a traversal, it is important that you track which vertices have been
visited. The most common way of tracking vertices is to mark them.
There are many ways to traverse graphs. BFS is the most commonly used
approach.
BFS is a traversing algorithm where you should start traversing from a
selected node (source or starting node) and traverse the graph layerwise
thus exploring the neighbour nodes (nodes which are directly connected to
source node). You must then move towards the next-level neighbour
nodes.
As the name BFS suggests, you are required to traverse the graph
breadthwise as follows:
1. First move horizontally and visit all the nodes of the current layer
2. Move to the next layer
42
The distance between the nodes in layer 1 is comparitively lesser than the
distance between the nodes in layer 2. Therefore, in BFS, you must
traverse all the nodes in layer 1 before you move to the nodes in layer 2.
A graph can contain cycles, which may bring you to the same node again
while traversing the graph. To avoid processing of same node again, use a
boolean array which marks the node after it is processed. While visiting the
nodes in the layer of a graph, store them in a manner such that you can
traverse the corresponding child nodes in a similar order.
In the earlier diagram, start traversing from 0 and visit its child nodes 1, 2,
and 3. Store them in the order in which they are visited. This will allow you
to visit the child nodes of 1 first (i.e. 4 and 5), then of 2 (i.e. 6 and 7), and
then of 3 (i.e. 7) etc.
To make this process easy, use a queue to store the node and mark it as
'visited' until all its neighbours (vertices that are directly connected to it) are
marked. The queue follows the First In First Out (FIFO) queuing method,
and therefore, the neigbors of the node will be visited in the order in which
they were inserted in the node i.e. the node that was inserted first will be
visited first, and so on.
43
Pseudocode
Traversing process
44
45
The traversing will start from the source node and push s in queue. s will be
marked as 'visited'.
First iteration
Second iteration
Third iteration
Fourth iteration
Fifth iteration
Sixth iteration
The queue is empty and it comes out of the loop. All the nodes have been
traversed by using BFS.
If all the edges in a graph are of the same weight, then BFS can also be
used to find the minimum distance between the nodes in a graph.
Example
47
As in this diagram, start from the source node, to find the distance between
the source node and node 1. If you do not follow the BFS algorithm, you
can go from the source node to node 2 and then to node 1. This approach
will calculate the distance between the source node and node 1 as 2,
whereas, the minimum distance is actually 1. The minimum distance can
be calculated correctly by using the BFS algorithm.
Complexity
The time complexity of BFS is O(V + E), where V is the number of nodes
and E is the number of edges.
Applications
48
As you know in BFS, you traverse level wise. You can also use BFS to
determine the level of each node.
Implementation
This code is similar to the BFS code with only the following difference:
level[ v[ p ][ i ] ] = level[ p ]+1;
In this code, while you visit each node, the level of that node is set with an
increment in the level of its parent node. This is how the level of each node
is determined.
49
2. 0-1 BFS
This type of BFS is used to find the shortest distance between two nodes in
a graph provided that the edges in the graph have the weights 0 or 1. If you
apply the BFS explained earlier in this article, you will get an incorrect
result for the optimal distance between 2 nodes.
In this approach, a boolean array is not used to mark the node because the
condition of the optimal distance will be checked when you visit each node.
A double-ended queue is used to store the node. In 0-1 BFS, if the weight
of the edge = 0, then the node is pushed to the front of the dequeue. If the
weight of the edge = 1, then the node is pushed to the back of the
dequeue.
Implementation
50
Here, edges[ v ] [ i ] is an adjacency list that exists in the form of pairs i.e.
edges[ v ][ i ].first will contain the node to which v is connected and edges[
v ][ i ].second will contain the distance between v and edges[ v ][ i ].first.
51
The adjacency list of the graph will be as follows:
Here 's' is considered to be 0 or source node.
1 -> 0 -> 4
edges[ 1 ][ 0 ].first = 0 , edges[ 1 ][ 0 ].second = 1
edges[ 1 ][ 1 ].first = 4 , edges[ 1 ][ 1 ].second = 0
2 -> 0 -> 3
edges[ 2 ][ 0 ].first = 0 , edges[ 2 ][ 0 ].second = 0
edges[ 2 ][ 1 ].first = 3 , edges[ 2 ][ 1 ].second = 0
52
4 -> 1 -> 3
edges[ 4 ][ 0 ].first = 1 , edges[ 4 ][ 0 ].second = 0
edges[ 4 ][ 1 ].first = 3 , edges[ 4 ][ 1 ].second = 0
If you use the BFS algorithm, the result will be incorrect because it will
show you the optimal distance between s and node 1 and s and node 2 as
1 respectively. This is because it visits the children of s and calculates the
distance between s and its children, which is 1. The actual optimal distance
is 0 in both cases.
Processing
Starting from the source node, i.e 0, it will move towards 1, 2, and 3. Since
the edge weight between 0 and 1 and 0 and 2 is 1 respectively, 1 and 2 will
be pushed to the back of the queue. However, since the edge weight
between 0 and 3 is 0, 3 will pushed to the front of the queue. The distance
will be maintained in distance array accordingly.
3 will then be popped from the queue and the same process will be applied
to its neighbours, and so on.
Practice
53
Depth First Search
Here, the word backtrack means that when you are moving forward and
there are no more nodes along the current path, you move backwards on
the same path to find nodes to traverse. All the nodes will be visited on the
current path till all the unvisited nodes have been traversed after which the
next path will be selected.
This recursive nature of DFS can be implemented using stacks. The basic
idea is as follows:
Pick a starting node and push all its adjacent nodes into a stack.
Pop a node from stack to select the next node to visit and push all its
adjacent nodes into a stack.
Repeat this process until the stack is empty. However, ensure that the
nodes that are visited are marked. This will prevent you from visiting the
same node more than once. If you do not mark the nodes that are visited
and you visit the same node more than once, you may end up in an infinite
loop.
Pseudocode
54
The following image shows how DFS works.
55
Applications
In DFS, if we start from a start node it will mark all the nodes connected to
the start node as visited. Therefore, if we choose any node in a connected
component and run DFS on that node it will mark the whole connected
component as visited.
56
Input File
6
4
12
23
13
45
Code
57
Output
Number of connected components: 3
Time complexity O(V+E), when implemented using the adjacency list.
Practice
58
Minimum Spanning Tree
59
There are two famous algorithms for finding the Minimum Spanning Tree:
Kruskal’s Algorithm
Kruskal’s Algorithm builds the spanning tree by adding edges one by one
into a growing spanning tree. Kruskal's algorithm follows greedy approach
as in each iteration it finds an edge which has least weight and add it to the
growing spanning tree.
Algorithm Steps:
61
In Kruskal’s algorithm, at each iteration we will select the edge with the
lowest weight. So, we will start with the lowest weighted edge first i.e., the
edges with weight 1. After that we will select the second lowest weighted
edge i.e., edge with weight 2. Notice these two edges are totally disjoint.
Now, the next edge will be the third lowest weighted edge i.e., edge with
weight 3, which connects the two disjoint pieces of the graph. Now, we are
not allowed to pick the edge with weight 4, that will create a cycle and we
can’t have any cycles. So we will select the fifth lowest weighted edge i.e.,
edge with weight 5. Now the other two edges will create cycles so we will
ignore them. In the end, we end up with a minimum spanning tree with total
cost 11 ( = 1 + 2 + 3 + 5).
Implementation:
62
63
Time Complexity:
Prim’s Algorithm
Prim’s Algorithm also use Greedy approach to find the minimum spanning
tree. In Prim’s Algorithm we grow the spanning tree from a starting position.
Unlike an edge in Kruskal's, we add vertex to the growing spanning tree in
Prim's.
Algorithm Steps:
● Maintain two disjoint sets of vertices. One containing vertices that are
in the growing spanning tree and other that are not in the growing
spanning tree.
● Select the cheapest vertex that is connected to the growing spanning
tree and is not in the growing spanning tree and add it into the
growing spanning tree. This can be done using Priority Queues.
Insert the vertices, that are connected to growing spanning tree, into
the Priority Queue.
● Check for cycles. To do that, mark the nodes which have been
already selected and insert only those nodes in the Priority Queue
that are not marked.
64
In Prim’s Algorithm, we will start with an arbitrary node (it doesn’t matter
which one) and mark it. In each iteration we will mark a new vertex that is
adjacent to the one that we have already marked. As a greedy algorithm,
Prim’s algorithm will select the cheapest edge and mark the vertex. So we
will simply choose the edge with weight 1. In the next iteration we have
three options, edges with weight 2, 3 and 4. So, we will select the edge with
weight 2 and mark the vertex. Now again we have three options, edges
with weight 3, 4 and 5. But we can’t choose edge with weight 3 as it is
creating a cycle. So we will select the edge with weight 4 and we end up
with the minimum spanning tree of total cost 7 ( = 1 + 2 +4).
Implementation:
65
Time
Complexity:
The time complexity of the Prim’s Algorithm is O((V+E)logV) because each
vertex is inserted in the priority queue only once and insertion in priority
queue take logarithmic time.
Practice
Link to the Practice Problems: Practice Questions
66
Shortest Path Algorithms
Dijkstra's Algorithm
Dijkstra's algorithm has many variants but the most common one is to find
the shortest paths from the source vertex to all other vertices in the graph.
Algorithm Steps:
● Set all vertices distances = infinity except for the source vertex, set
the source distance = 0.
● Push the source vertex in a min-priority queue in the form (distance ,
vertex), as the comparison in the min-priority queue will be according
to vertices distances.
● Pop the vertex with the minimum distance from the priority queue (at
first the popped vertex = source).
68
● Update the distances of the connected vertices to the popped vertex
in case of "current vertex distance + edge weight < next vertex
distance", then push the vertex
● with the new distance to the priority queue.
● If the popped vertex is visited before, just continue without using it.
● Apply the same algorithm again until the priority queue is empty.
Implementation:
69
Floyd–Warshall's Algorithm
Floyd–Warshall's Algorithm is used to find the shortest paths between
between all pairs of vertices in a graph, where each edge in the graph has
a weight which is positive or negative. The biggest advantage of using this
algorithm is that all the shortest distances between any 2vertices could be
calculated in O(V3), where V is the number of vertices in a graph.
The Algorithm Steps:
For a graph with N vertices:
● Initialize the shortest paths between any 2 vertices with Infinity.
● Find all pair shortest paths that use 0 intermediate vertices, then find
the shortest paths that use 1 intermediate vertex and so on.. until
using all N vertices as intermediate nodes.
● Minimize the shortest paths between any 2 pairs in the previous
operation.
● For any 2 vertices (i,j) , one should actually minimize the distances
between this pair using the first K nodes, so the shortest path will be:
min(dist[i][k]+dist[k][j],dist[i][j]).
dist[i][k] represents the shortest path that only uses the first K vertices,
dist[k][j] represents the shortest path between the pair k,j. As the shortest
path will be a concatenation of the shortest path from i to k, then from k to j.
70
Flood-fill Algorithm
Flood fill algorithm helps in visiting each and every point in a given area. It
determines the area connected to a given cell in a multi-dimensional array.
Following are some famous implementations of flood fill algorithm:
Clicking in an area with this tool selected fills that area with the selected
color.
Solving a Maze:
Given a matrix with some starting point, and some destination with some
obstacles in between, this algorithm helps to find out the path from source
to destination
Minesweeper:
71
It clearly shows how the cell in the middle is connected to cells around it.
For instance, there are 8-connections like there are in Minesweeper
(clicking on any cell that turns out to be blank reveals 8 cells around it
which contains a number or are blank). The cell (1,1) is connected to (0,0),
(0,1), (0,2), (1,0), (1,2), (2,0), (2,1), (2,2).
Now that the given area has been modeled as a graph, a DFS or BFS can
be applied to traverse that graph. The pseudo code is given below.
The above code visits each and every cell of a matrix of size n×m starting
with some source cell. Time Complexity of above algorithm is O(n×m).
72
One another use of flood algorithm is found in solving a maze. Given a
matrix, a source cell, a destination cell, some cells which cannot be visited,
and some valid moves, check if the destination cell can be reached from
the source cell. Matrix given in the image below shows one such problem.
The source is cell (0,0) and the destination is cell (3,4). Cells containing X
cannot be visited. Let's assume there are 4 valid moves - move up, move
down, move left and move right.
The code given above is same as that given previously with slight changes.
It takes three more parameters including the given matrix to check if the
current cell is marked X or not and coordinates of destination cell
(destx,desty). If the current cell is equal to destination cell it returns True,
and consequently, all the previous calls in the stack returns True, because
73
there is no use of visiting any cells further when it has been discovered that
there is a path between source and destination cell.
So for the matrix given in image above the code returns True.
If, in the given matrix the cell (1,2) was also marked X, then the code would
have returned False, as there would have been no path to reach from S to
D in that case.
Practice:
74
Articulation Points and Bridges
Articulation Point
In a graph, a vertex is called an articulation point if removing it and all the
edges associated with it results in the increase of the number of connected
components in the graph. For example consider the graph given in
following figure.
If in the above graph, vertex 1 and all the edges associated with it, i.e. the
edges 1-0, 1-2 and 1-3 are removed, there will be no path to reach any of
the vertices 2, 3 or 4 from the vertices 0 and 5, that means the graph will
split into two separate components. One consisting of the vertices 0 and 5
and another one consisting of the vertices 2, 3 and 4 as shown in the
following figure.
75
Likewise removing the vertex 0 will disconnect the vertex 5 from all other
vertices. Hence the given graph has two articulation points: 0 and 1.
Here's the pseudo code of the brute force approach, it returns the total
number of articulation points in the given graph.
76
The above algorithm iterates over all the vertices and in one iteration
applies a Depth First Search to find connected components, so time
complexity of above algorithm is O(V×(V+E)), where V is the number of
vertices and E is the number of edges in the graph.
Clearly the brute force approach will fail for bigger graphs.
There is an algorithm that can help find all the articulation points in a given
graph by a single Depth First Search, that means with complexity O(V+E),
but it involves a new term called "Back Edge" which is explained below:
77
In the above case, the edge 4 - 2 connects 4 to an ancestor of its parent
i.e. 3, so it is a Back Edge. And similarly 3 - 1 is also a Back edge. But why
bother about Back Edge? Presence of a back edge means presence of an
alternative path in case the parent of the vertex is removed. Suppose a
vertex u is having a child v such that none of the vertices in the subtree
rooted at v have a back edge to any vertex discovered before u, that
means if vertex u is removed then there will be no path left for vertex v or
any of the vertices present in the subtree rooted at vertex v to reach any
vertex discovered before u, that implies, the subtree rooted at vertex v will
get disconnected from the entire graph, and thus the number of
components will increase and u will be counted as an articulation point. On
the other hand, if the subtree rooted at vertex v has a vertex x that has
back edge that connects it to a vertex discovered before u, say y, then
there will be a path for any vertex in subtree rooted at v to reach y even
78
after removal of u, and if that is the case with all the children of u, then u
will not count as an articulation point.
So ultimately it all converges down to finding a back edge for every vertex.
So, for that apply a DFS and record the discovery time of every vertex and
maintain for every vertex v the earliest discovered vertex that can be
reached from any of the vertices in the subtree rooted at v. If a vertex u is
having a child v such that the earliest discovered vertex that can be
reached from the vertices in the subtree rooted at v has a discovery time
greater than or equal to u, then v does not have a back edge, and thus u
will be an articulation point.
So, till now the algorithm says that if all children of a vertex u are having a
back edge, then u is not an articulation point. But what will happen when u
is root of the tree, as root does not have any ancestors. Well, it is very easy
to check if the root is an articulation point or not. If root has more than one
child than it is an articulation point otherwise it is not. Now how does that
help?? Suppose root has two children, v1 and v2. If there had been an
edge between vertices in the subtree rooted at v1 and those of the subtree
rooted at v2, then they would have been a part of the same subtree.
79
Here's what everything means:
80
vertex: The vertex under consideration.
V : Number of vertices.
The following image shows the value of array disc[] and low[] for DFS tree
given in Fig. 3.
81
Clearly only for vertices 0 and 1, low[5]≥disc[0] and low[2]≥disc[1], so these
are the only two articulation points in the given graph.
Bridges
The Brute force approach to find all the bridges in a given graph is to check
for every edge if it is a bridge or not, by first removing it and then checking
if the vertices that it was connecting are still connected or not. Following is
pseudo code of this approach:
82
The above code uses BFS to check if the vertices that were connected by
the removed edge are still connected or not. It does so for every edge and
thus it's complexity is O(E×(V+E)). Clearly it will fail for big values of V and
E.
83
For graph given in Fig.1, the low[] and disc[] value obtained for its DFS tree
shown in Fig.3, by the above pseudo code, will be the same as those
obtained in case of articulation points. The values of array low[] and disc[]
are shown in Fig.4. Clearly for only two edges i.e 0-1 and 0-5, low[1] >
disc[0] and low[5] > disc[0], hence those are the only two bridges in the
given graph.
Practice:
84
Biconnected Components
The given graph is clearly connected. Now try removing the vertices one by
one and observe. Removing any of the vertices does not increase the
number of connected components. So the given graph is Biconnected.
85
Now consider the following graph which is a slight modification in the
previous graph.
In the above graph if the vertex 2 is removed, then here's how it will look:
86
Clearly the number of connected components have increased. Similarly, if
vertex 3 is removed there will be no path left to reach vertex 0 from any of
the vertices 1, 2, 4 or 5. And same goes for vertex 4 and 1. Removing
vertex 4 will disconnect 1 from all other vertices 0, 2, 3 and 4. So the graph
is not Biconnected.
So simply check if the given graph has any articulation point or not. If it has
no articulation point then it is Biconnected otherwise not. Here's the pseudo
code:
The code above is exactly same as that for Articulation Point with one
difference that it returns false as soon as it finds an Articulation Point.
87
The image below shows how the DFS tree will look like for the graph in Fig.
2 according to the algorithm, along with the value of the arrays low[] and
disc[].
Following image shows DFS tree, value of arrays low[] and disc[] for graph
in Fig.1
88
Clearly there does not exists any vertex x, such that low[x]≥disc[x], i.e. the
graph has no articulation point, so the algorithm returns true, that means
the graph is Biconnected.
89
Biconnected components in a graph can be determined by using the
previous algorithm with a slight modification. And that modification is to
maintain a Stack of edges. Keep adding edges to the stack in the order
they are visited and when an articulation point is detected i.e. say a vertex
u has a child v such that no vertex in the subtree rooted at v has a back
edge (low[v]≥disc[u]) then pop and print all the edges in the stack till the
u−v is found, as all those edges including the edge u−v will form one
biconnected component.
90
Let's see how it works for graph shown in Fig.2.
First it finds visited[0] is false so it starts with vertex 0 and first discovers
the edge 0-3 and pushes it in the stack.
91
It then discovers the fact that low[1]≥disc[4] i.e. discovers that 4 is an
articulation point. So all the edges inserted after the edge 4-1 along with
edge 4-1 will form first biconnected component. So it pops and print the
edges till last edge is 4-1 and then prints that too and pop it from the stack
For 5 it discovers the back edge 5-2 and pushes that in stack
92
After that no more edge is connected to 5 so it goes back to 4. For 4 also
no more edge is connected and also low[5]≱disc[4].
Now finally it discovers that for edge 0-3 also low[3]≥disc[0] so it pops it
from the stack and print it as the fourth biconnected component.
Then it checks visited[] value of other vertices and as for all vertices it is
true so the algorithm terminates.
93
Time complexity of the algorithm is same as that of DFS. If V is the number
of vertices and E is the number of edges then complexity is O(V+E).
Practice:
94
Strongly connected components can be found one by one, that is first the
strongly connected component including node 1 is found. Then, if node 2 is
not included in the strongly connected component of node 1, similar
process which will be outlined below can be used for node 2, else the
process moves on to node 3 and so on.
So, how to find the strongly connected component which includes node 1?
Let there be a list which contains all nodes, these nodes will be deleted one
by one once it is sure that the particular node does not belong to the
strongly connected component of node 1. So, initially all nodes from 1 to N
are in the list. Let length of list be LEN, current index be IND and the
element at current index ELE. Now for each of the elements at index
IND+1,...,LEN, assume the element is OtherElement, it can be checked if
there is a directed path from OtherElement to ELE by a single O(V+E) DFS,
and if there is a directed path from ELE to OtherElement, again by a single
O(V+E) DFS. If not, OtherElement can be safely deleted from the list.
After all these steps, the list has the following property: every element can
reach ELE, and ELE can reach every element via a directed path. But the
elements of this list may or may not form a strongly connected component,
because it is not confirmed that there is a path from other vertices in the list
excluding ELE to the all other vertices of the list excluding ELE.
In the end, list will contain a Strongly Connected Component that includes
node 1. Now, to find the other Strongly Connected Components, a similar
95
process must be applied on the next element(that is 2), only if it has not
already been a part of some previous Strongly Connected
Component(here, the Strongly Connected Component of 1). Else, the
process continues to node 3 and so on.
This algorithm just does DFS twice, and has a lot better complexity O(V+E),
than the brute force approach. First define a Condensed Component Graph
as a graph with ≤V nodes and ≤E edges, in which every node is a Strongly
Connected Component and there is an edge from C to C′, where C and C′
are Strongly Connected Components, if there is an edge from any node of
C to any node of C′.
Now a property can be proven for any two nodes C and C′ of the
Condensed Component Graph that share an edge, that is let C→C′ be an
edge. The property is that the finish time of DFS of some node in C will be
always higher than the finish time of all nodes of C′.
Proof: There are 2 cases, when DFS first discovers either a node in C or a
node in C′.
Case 1: When DFS first discovers a node in C: Now at some time during
the DFS, nodes of C′ will start getting discovered(because there is an edge
from C to C′), then all nodes of C′ will be discovered and their DFS will be
finished in sometime (Why? Because it is a Strongly Connected
Component and will visit everything it can, before it backtracks to the node
in C, from where the first visited node of C′ was called). Therefore for this
case, the finish time of some node of C will always be higher than finish
time of all nodes of C′.
97
Case 2: When DFS first discovers a node in C′: Now, no node of C has
been discovered yet. DFS of C′ will visit every node of C′ and maybe more
of other Strongly Connected Component's if there is an edge from C′ to that
Strongly Connected Component. Observe that now any node of C will
never be discovered because there is no edge from C′ to C. Therefore DFS
of every node of C′ is already finished and DFS of any node of C has not
even started yet. So clearly finish time of some node(in this case all) of C,
will be higher than the finish time of all nodes of C′.
Now the only problem left is how to find some node in the sink Strongly
Connected Component of the condensed component graph. The
condensed component graph can be reversed, then all the sources will
become sinks and all the sinks will become sources. Note that the Strongly
Connected Component's of the reversed graph will be same as the
Strongly Connected Components of the original graph.
98
Now a DFS can be done on the new sinks, which will again lead to finding
Strongly Connected Components. And now the order in which DFS on the
new sinks needs to be done, is known. The order is that of decreasing
finishing times in the DFS of the original graph. This is because it was
already proved that an edge from C to C′ in the original condensed
component graph means that finish time of some node of C is always
higher than finish time of all nodes of C′. So when the graph is reversed,
sink will be that Strongly Connected Component in which there is a node
with the highest finishing time. Since edges are reversed, DFS from the
node with highest finishing time, will visit only its own Strongly Connected
Component.
Now a DFS can be done from the next valid node(valid means which is not
visited yet, in previous DFSs) which has the next highest finishing time. In
this way all Strongly Connected Component's will be found. The complexity
of the above algorithm is O(V+E), and it only requires 2DFSs.
The algorithm in steps can be described as below:
1) Do a DFS on the original graph, keeping track of the finish times of each
node. This can be done with a stack, when some DFS finishes put the
source vertex on the stack. This way node with highest finishing time will be
on top of the stack.
99
3) Do DFS on the reversed graph, with the source vertex as the vertex on
top of the stack. When DFS finishes, all nodes visited will form one Strongly
Connected Component. If any more nodes remain unvisited, this means
there are more Strongly Connected Component's, so pop vertices from top
of the stack until a valid unvisited node is found. This will have the highest
finishing time of all currently unvisited nodes. This step is repeated until all
nodes are visited.
Practice:
100
Topological Sort
101
vi. We'll maintain an array T that will denote our topological sorting. So, let's
say for a graph having N vertices, we have an array in_degree[] of size N
whose ith element tells the number of vertices which are not already
inserted in T and there is an edge from them incident on vertex numbered i.
We'll append vertices vi to the array T, and when we do that we'll decrease
the value of in_degree[vj] by 1 for every edge from vi to vj. Doing this will
mean that we have inserted one vertex having edge directed towards vj. So
at any point we can insert only those vertices for which the value of
in_degree[] is 0.
Let's take a graph and see the algorithm in action. Consider the graph
given below:
102
Initially in_degree[0]=0 and T is empty
103
So, we continue doing like this, and further iterations looks like as follows:
104
105
So at last we get our Topological sorting in T i.e. : 0, 1, 2, 3, 4, 5
Solution using a DFS traversal, unlike the one using BFS, does not need
any special in_degree[] array. Following is the pseudo code of the DFS
solution:
The following image of shows the state of stack and of array T in the above
code for the same graph shown above.
106
107
Practice:
Hamiltonian Path
108
Graph shown in Fig.1 does not contain any Hamiltonian Path. Graph shown
in Fig. 2 contains two Hamiltonian Paths which are highlighted in Fig. 3 and
Fig. 4
109
represents a Hamiltonian Path or not. For example, for the graph
given in Fig. 2 there are 4 vertices, which means total 24 possible
permutations, out of which only following represents a Hamiltonian
Path.
0-1-2-3
3-2-1-0
0-1-3-2
2-3-1-0
110
Time complexity of the above method can be easily derived. For a graph
having N vertices it visits all the permutations of the vertices, i.e. N!
iterations and in each of those iterations it traverses the permutation to see
if adjacent vertices are connected or not i.e N iterations, so the complexity
is O( N * N! ).
2) There is one algorithm given by Bellman, Held, and Karp which uses
dynamic programming to check whether a Hamiltonian Path exists in a
graph or not. Here's the idea, for every subset S of vertices check whether
there is a path that visits "EACH and ONLY" the vertices in S exactly once
and ends at a vertex v. Do this for all v ϵ S. A path exists that visits each
vertex in subset S and ends at vertex v ϵ S iff v has a neighbor w in S and
there is a path that visits each vertex in set S-{v} exactly once and ends at
w. If there is such a path, then adding the edge w-v to it will extend it to visit
v and as it is already visiting every vertex in S-{v}, so the new path will visit
every vertex in S.
For example, consider the graph given in Fig. 2, let S={0, 1, 2} and v=2.
Clearly 2 has a neighbor in the set i.e. 1. A path exists that visits 0, 1, and 2
exactly once and ends at 2, if there is a path that visits each vertex in the
set (S-{2})={0, 1} exactly once and ends at 1. Well yes, there exists such a
path i.e. 0-1, and adding the edge 1-2 to it will make the new path look like
111
0-1-2. So there is a path that visits 0, 1 and 2 exactly once and ends at 2.
Following is the pseudo code for the above
algorithm, it uses bitmasking to represent subsets ( Learn about bitmasking
here):
Let's try to understand it. The cell dp[j][i] checks if there is a path that visits
each vertex in subset represented by mask i and ends at vertex j. In the
first 3 lines every cell of table dp is initialized as false and in the following
two lines the cells (i, 2i), 0 ≤ i < n are initialized as true. In the binary
conversion of 2i only ith bit is 1. That means 2i represents a subset
containing only the vertex i. And so the cell dp[i][2i] represents whether
there is a path that visits the vertex i exactly once and ends at vertex i. And
ofcourse for every vertex it should be true.
The next loop iterates over 0 to 2n-1, all the bitmasks, that means all the
subsets of the vertices. And the loop inside it check which of the vertices
from 0 to n-1 are present in subset S represented by a bitmask i. And the
third loop inside it checks for every vertex j present in S, which of the
vertices from 0 to n-1 are present in S and are neighbors of j. Then for
every such vertex k it checks whether the cell dp[k][ i XOR 2j ] is true or not.
112
What does this cell represents? In binary conversion of i XOR 2j every bit
which is 1 in binary conversion of i remains 1 except the jth bit. So i XOR 2j
represents the subset S-{j} and the cell dp[k][ i XOR 2j ] represents whether
there is a path that visits each vertex in the subset S-{j} exactly once and
ends at k. If there is a path that visits each vertex in S-{j} exactly once and
ends at k than adding the edge k-j will extend that path to visit each vertex
in S exactly once and end at j. So dp[j][i] will be true if there is such a path.
Finally there is a loop that iterates over all the vertices 0 to n-1 and checks
if the cell dp[i][2n-1] is true or not, where 0 ≤ i < n. In the binary conversion
of 2n-1 every bit is 1, that means it represents the set containing all the
vertices and cell dp[i][2n-1] represents whether there is a path that visits
every vertex exactly once and ends at i. If there is such a path it returns
true i.e. there is a Hamiltonian path in the given graph. In the last line it
returns false, that means no Hamiltonian path is found in the given graph.
Following is the C++ implementation of the above method:
Here's how the table dp looks like for the graph given in Fig. 2 filled upto
the mask 6.
113
Let's fill it for the mask 7 i.e. for the subset S={0, 1, 2}.
For cell dp[ 0 ][ 7 ], 0 has a neighbor in S i.e. 1, check if there is a path that
visits each vertex in subset represented by 7 XOR 20 = 6 i.e. {1, 2}, exactly
once and ends at 1, i.e. the cell dp[ 1 ][ 6 ]. It is true so dp[ 0 ][ 7 ] will also
be true.
For cell dp[ 1 ][ 7 ], 1 has two neighbors in S i.e. 0, 2. So check for the
bitmask 7 XOR 21 = 5 i.e. the subset {0, 2}. Here both the cells dp[ 0 ][ 5 ]
and dp[ 2 ][ 5 ] are false. So the cell dp[ 1 ][ 7 ] will remain false.
For cell dp[ 2 ][ 7 ], 2 has a neighbor in S i.e. 1, check if there is a path that
visits each vertex in subset represented by 7 XOR 22 = 3 i.e. {0, 1}, exactly
once and ends at 1, i.e. the cell dp[ 1 ][ 3 ]. It is true so dp[ 2 ][ 7 ] will also
be true.
For cell dp[ 3 ][ 7 ], clearly 3 ∉ {0, 1, 2}. So dp[ 3 ][ 7 ] will remain false.
Now clearly the cells dp[ 0 ][ 15 ], dp[ 2 ][ 15 ], dp[ 3 ][ 15 ] are true so the
graph contains a Hamiltonian Path.
114
3) Depth first search and backtracking can also help to check whether a
Hamiltonian path exists in a graph or not. Simply apply depth first search
starting from every vertex v and do labeling of all the vertices. All the
vertices are labelled as either "IN STACK" or "NOT IN STACK". A vertex is
labelled "IN STACK" if it is visited but some of its adjacent vertices are not
yet visited and is labelled "NOT IN STACK" if it is not visited.
If at any instant the number of vertices with label "IN STACK" is equal to
the total number of vertices in the graph then a Hamiltonian Path exists in
the graph.
Following image shows how this algorithm will work for graph shown in Fig.
2.
115
The above image shows how it will work when DFS is strated with vertex 1.
So clearly, the number of vertices having label IN_STACK is not equal to 4
at any stage. That means there is no Hamiltonian Path that starts with 1.
When DFS is applied starting from vertices 2, 3 and 4 same result will be
obtained. So there is no Hamiltonian Path in the given graph.
Following is the C++ implementation:
116
Worst case complexity of using DFS and backtracking is O(N!).
Practice:
117
Maximum flow
118
An augmenting path is a simple path from source to sink which do not
include any cycles and that pass only through positive weighted edges. A
residual network graph indicates how much more flow is allowed in each
edge in the network graph. If there are no augmenting paths possible from
S to T, then the flow is maximum. The result i.e. the maximum flow will be
the total flow out of source node which is also equal to total flow in to the
sink node.
119
120
Implementation:
Dinic's Algorithm
121
Level graph is one where value of each node is its shortest distance from
source.
Blocking flow includes finding the new path from the bottleneck node.
Residual graph and augmenting paths are previously discussed.
Inputs required are network graph G, source node S and sink node T.
122
123
Practice:
This algorithm is used to find min cost flow along the flow network. Pseudo
code for this algorithm is provided below.
Negative cycle in cost network(Gc) are cycle with sum of costs of all the
edges in the cycle is negative. They can be detected using Bellman Ford
algorithm. They should be eliminated because, practically, flow through
124
such cycles cannot be allowed. Consider a negative cost cycle, if at all flow
has to pass through this cycle, the total cost is always reducing for every
cycle completed. And so would result in an infinite loop in desire of
minimizing the total cost. So, whenever a cost network includes a negative
cycle, it implies, the cost can further be minimized (By flowing through other
side of cycle instead of the side currently considered). A negative cycle
once detected are removed by flowing a bottleneck capacity through all the
edges in the cycle.
There are various applications of minimum cost flow problem. One of which
is solving the minimum weighted bipartite matching. Bipartite graph(B) is a
graph whose nodes can be divided into two disjoint sets(P and Q) and all
the edges of graph joins a node in P to node in Q. Matching means that no
two edges in final flow touch each other(share common node). It can be
considered as a multi-source multi-destination graph. Convert such graphs
to single source and single destination by creating a source node S and
join all nodes in set P with Sand a destination node T and join all nodes in
set Q with T. Now, above algorithm can be applied to find min cost max
flow in graph B.
Hungarian Algorithm:
A variant of weighted bipartite matching problem is known as assignment
problem. In simple terms, assignment problem can be described as having
N jobs and N workers, each worker does a job for particular cost. Also,
each worker should be given only one job and each job should be assigned
to only worker. This can be solved using Hungarian algorithm. Pseudocode
for this problem is given below.
Input will be an N×N matrix showing cost charged by each worker for each
job.
125
FindMinCost does an optimal selection of 0s in matrix X such that N cells
are selected and non of them lie in same row or column. The values in cells
in C corresponding to selected cells in X, are added and are returned as
answer for minimum cost that is to be calculated.
Practice:
126
Min-cut
127
rest of the graph. Minimum value of cut_of_the_phase is the required
result.
Last two vertices added to the set A are merged by creating a single node
for both of them and edges joining them are deleted. Edges connecting
these vertices to other vertices are replaced with edges weighing sum of
edges to both the vertex. Ex: Let vertices P and Q are the two vertices
added last to the set A. Let there are three edges connecting P and Q to
the set A, E(X,P),W(X,P)=10;E(X,Q),W(X,Q)=20;E(Y,Q),W(Y,Q)=15. Now
as they are added last, vertices P and Q are merged to a single node, say
R. Now, edges connecting to R will be,
E(X,R),W(X,R)=30;E(Y,R),W(Y,R)=15.
128
Few possible cuts in the graph are shown in the graph and weights of each
cut are as follows: Cut1:25,Cut2:12,Cut3:16,Cut4:10,Cut5:15. As
mentioned these are only few possible cuts but considering any valid cut
would not have weight less than Cut4. And it is the min-cut of this graph
which is also the max-flow of the graph as explained above.
Practice:
129