Lecture 4

Analysis, Design of Algorithms
CS3610
Dr. Islam Hegazy

Associate Professor
F2023 Analysis, Design of Algorithms 1

Objectives
By the end of this lecture, you will:
Know common exhaustive search problems
Know the graph definitions and usages
Understand the depth-first search

Agenda
01 Exhaustive search 02 Summary of brute force
03 Graph traversal 04 DFS and BFS

1. Exhaustive search
A brute-force approach to combinatorial problems.

Steps:
Generate each and every element of the problem domain,
Select those of them that satisfy all the constraints, and then
Find a desired element (e.g., the one that optimizes some objective
function).

Advantages:
widely applicable
Easy
good for small problem sizes

Disadvantages:
often inefficient for large inputs

Traveling Salesman Problem
Find the shortest tour through a given set of n cities that visits each city exactly
once before returning to the city where it starts.



Knapsack Problem
Given n objects, each object i has weight wi and value vi, and a knapsack of
capacity W
Find most valuable items that fit into the knapsack
Items are not splittable

Knapsack Problem
Example: Knapsack capacity W = 16
Item Weight Value

1 2 $20
2 5 $30
3 10 $50
4 5 $10
Solution: Consider every possible subset of items, calculate total value and total
weight and discard if more than W; then choose remaining subset with
maximum total value.

Knapsack Problem

Knapsack Problem
Analysis
Input size: n
Running time: C(n) = 2n (number of subsets including Φ)

Assignment Problem
There are n people to execute n jobs, one person per job.

If ith person is assigned the jth job, the cost is C[i, j], i, j = 1, . . . , n.
Find the assignment with the minimum total cost.
Job 1 Job 2 Job 3 Job 4

Person 1 9 2 7 8
Person 2 6 4 3 7
Person 3 5 8 1 8
Person 4 7 6 9 4

Assignment Problem
Solution: compute all permutations and choose the one with least cost

Assignment Problem
Analysis
Input size: n
Running time: C(n) = n!

Agenda

Summary of brute-force
Strengths
Wide applicability
Simplicity
Yields reasonable algorithms for some important problems (e.g., matrix
2. Summary
multiplication, sorting, searching, string matching)

Weaknesses
Rarely yields efficient algorithms
Some brute-force algorithms are unacceptably slow
Not as constructive as some other design techniques

Summary of brute-force
Exhaustive-search algorithms run in a realistic amount of time only on very

small instances.
In some cases, there are much better alternatives
Shortest paths (greedy)
2. Summary
Minimum spanning tree (greedy)

Assignment problem (iterative improvement)
In many cases, exhaustive search or its variation is the only known way to get
exact solution

Agenda

Graph traversal
A graph is an abstract representation of a set of objects where some pairs of the

objects are connected by links.
Vertices: interconnected objects
3. Graph traversal
Edges: links that connect the vertices

A graph G = (V, E) V
V = set of vertices E
E = set of edges = subset of V × V

Graph traversal
Adjacency: For each edge {u, v} the vertices u and v are said to be adjacent to
one another, denoted u ~ v.
Graph Order: number of vertices |V|
3. Graph traversal
Graph Size: number of edges |E|

Max (E) = V(V-1)/2 V
Vertex Degree: number of edges that connect to it. E
In-degree
Out-degree

Graph traversal – Types
Connected graph: path from every vertex to every other

3. Graph traversal
undirected graph: (Symmetrical)

u v
Edge (u,v) = Edge (v,u)
No self-loops
Directed graph (Unsymmetrical) u v
Edge (u,v) goes from vertex u to vertex v, notated u→v
Graph traversal – Types
Weighted graph: associates weights with either the edges or the vertices
e.g. a road map: edges might be weighted w/distance
Dense graph: If |E| = O(|V|2): a graph with large number of edges

3. Graph traversal
Sparse graph: If |E| = O(|V|): a graph with relatively few edges

Complexity: in terms of |E| and |V|

Graph traversal – Applications
Web Crawling: (how Google find pages)
Vertices: web pages
Edges: hyperlinks
3. Graph traversal
Social Networks: (Facebook friend finder)

Vertices: people
Edges: friend/follow relationships
Internet Routing:
Vertices: end hosts & routers
Edges: direct physical or wireless connections

Graph traversal – Applications
Maps (e.g. Google map):
Vertices: intersection points
Edges: roads
3. Graph traversal
Image Apps (segmentation, intelligent scissor…):

Vertices: pixels
Edges: direct connection between pixel and its neighbors
Solving puzzles and games
Vertices: each possible state
Edges: each basic move from one state to another

Graph traversal – Representation
How to represent G(V,E) on computers?
Adjacency matrix: represents the graph as a |V| x |V| matrix A:

A[i, j] = 1 if edge (i, j) ∈ E (or weight of edge)
= 0 if edge (i, j) ∉ E
3. Graph traversal
Example:
A 1 2 3 4
1 1 0 1 1 0
2 4 2 0 0 1 0
3 0 0 0 0
3 4 0 0 1 0

Adjacency list: for each vertex v ∈ V, store a list of vertices adjacent to v
Example:
3. Graph traversal
1 Adj[1] = {2,3}
Adj[2] = {3}
2 4 Adj[3] = {}
Adj[4] = {3}
3

Comparison
Comparison Matrix List

Θ(|V|2) Θ(|V|+|E|)
Storage size
= O(|V|2)
3. Graph traversal
Check existence of an edge Θ(1) Θ(adj(u))

(u, v)? = O(|V|)
Θ(|V|) Θ(adj(u))
Find neighbors of a vertex u?
= O(|V|)
Simple (array) Complex (lists & ptrs)
Data structure
Suitable for Dense graph Sparse graph

Agenda

DFS and BFS
Exhaustive search can also be applied to two very important algorithms that
systematically process all vertices and edges of a graph.
Depth-First Search (DFS)
4. DFS and BFS
Breadth-First Search (BFS)

Useful in many applications involving graphs in artificial intelligence and
operations research.
Indispensable for efficient investigation of fundamental properties of graphs
such as connectivity and cycle presence.

DFS and BFS
DFS and BFS consist of
Visiting a vertex
Exploration of a vertex
4. DFS and BFS
Depth-first search: 1, 2, 3, 6, 7, 4, 5
Breadth-first search : 1, 2, 4, 5, 3, 6, 7 1
2 4 5
3 6 7

Search as deep as possible first.
Choose a vertex v (source, usually root of tree)
Explore edges out of v

4. DFS and BFS
When all edges of v have been explored, backtrack to predecessor of v and

explore all edges from predecessor
Continue until all vertices reachable from the original source are discovered

Many problems in computer science can be solved using DFS
Analyzing networks
Mapping routes
4. DFS and BFS
Scheduling
Finding spanning trees
DFS can also be used as a subroutine to solve complex problems
Matching algorithm
Hopcroft–Karp
Traveling Salesman problem

It is convenient to use a stack to trace the operation of depth-first search.
Push a vertex onto the stack when the vertex is reached for the first time (i.e
the visit of the vertex starts)
4. DFS and BFS
Pop a vertex off the stack when it becomes a dead end (i.e., the visit of the
vertex ends).

0 3
Visited
2
4. DFS and BFS
Stack
1 4

0 3
0 Visited
2
4. DFS and BFS
1 2 3 Stack
1 4

0 3
0 1 Visited
2
4. DFS and BFS
2 3 Stack
1 4

0 3
0 1 2 Visited
2
4. DFS and BFS
4 3 Stack
1 4

0 3
0 1 2 4 Visited
2
4. DFS and BFS
3 Stack
1 4

0 3
0 1 2 4 3 Visited
2
4. DFS and BFS
Stack
1 4

A standard DFS implementation puts each vertex of the graph into one of two
categories:
Visited
4. DFS and BFS
Not Visited
The purpose of the algorithm is to mark each vertex as visited while avoiding
cycles.

The DFS algorithm works as follows:
Start by putting any one of the graph's vertices on top of a stack.
Take the top item of the stack and add it to the visited list.
4. DFS and BFS
Create a list of that vertex's adjacent nodes. Add the ones which aren't in the
visited list to the top of the stack.
Keep repeating steps 2 and 3 until the stack is empty.

Implements a depth-first search traversal of a given graph
Input: Graph G = (V, E)
Output: Graph G with its vertices marked with consecutive integers in the order
4. DFS and BFS
they are first encountered by the DFS traversal
DFS(G)
mark each vertex in V with 0 as a mark of being “unvisited”
count ←0
for each vertex v in V do
if v is marked with 0
dfs(v)

Visit recursively all the unvisited vertices connected to vertex v by a path and
numbers them in the order they are encountered via global variable count
dfs(v)
count ←count + 1;
4. DFS and BFS
mark v with count

for each vertex w in V adjacent to v do
if w is marked with 0
dfs(w)

Time complexity of DFS
If represented in adjacency matrix = O(V2)
If represented in adjacency list = O(V + E)

4. DFS and BFS

Breadth-First Search (BFS)
Time complexit
4. DFS and BFS


Lecture 4

Uploaded by

Copyright:

Available Formats

Lecture 4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 4

Uploaded by

Copyright:

Available Formats

Analysis, Design of Algorithms

Dr. Islam Hegazy

F2023 Analysis, Design of Algorithms 1

By the end of this lecture, you will:

Know common exhaustive search problems

Know the graph definitions and usages

Understand the depth-first search

F2023 Analysis, Design of Algorithms 2

01 Exhaustive search 02 Summary of brute force

03 Graph traversal 04 DFS and BFS

F2023 Analysis, Design of Algorithms 3

A brute-force approach to combinatorial problems.

F2023 Analysis, Design of Algorithms 4

good for small problem sizes

F2023 Analysis, Design of Algorithms 5

F2023 Analysis, Design of Algorithms 6

F2023 Analysis, Design of Algorithms 7

F2023 Analysis, Design of Algorithms 8

Items are not splittable

F2023 Analysis, Design of Algorithms 9

Example: Knapsack capacity W = 16

Item Weight Value

F2023 Analysis, Design of Algorithms 10

F2023 Analysis, Design of Algorithms 11

F2023 Analysis, Design of Algorithms 12

There are n people to execute n jobs, one person per job.

Job 1 Job 2 Job 3 Job 4

F2023 Analysis, Design of Algorithms 13

F2023 Analysis, Design of Algorithms 14

F2023 Analysis, Design of Algorithms 15

01 Exhaustive search 02 Summary of brute force

03 Graph traversal 04 DFS and BFS

F2023 Analysis, Design of Algorithms 16

multiplication, sorting, searching, string matching)

F2023 Analysis, Design of Algorithms 17

Exhaustive-search algorithms run in a realistic amount of time only on very

Minimum spanning tree (greedy)

F2023 Analysis, Design of Algorithms 18

01 Exhaustive search 02 Summary of brute force

03 Graph traversal 04 DFS and BFS

F2023 Analysis, Design of Algorithms 19

A graph is an abstract representation of a set of objects where some pairs of the

Edges: links that connect the vertices

E = set of edges = subset of V × V

F2023 Analysis, Design of Algorithms 20

Graph Size: number of edges |E|

Vertex Degree: number of edges that connect to it. E

F2023 Analysis, Design of Algorithms 21

Connected graph: path from every vertex to every other

undirected graph: (Symmetrical)

e.g. a road map: edges might be weighted w/distance

Dense graph: If |E| = O(|V|2): a graph with large number of edges

Sparse graph: If |E| = O(|V|): a graph with relatively few edges

F2023 Analysis, Design of Algorithms 23

Web Crawling: (how Google find pages)

Vertices: web pages

Social Networks: (Facebook friend finder)

Edges: direct physical or wireless connections

Maps (e.g. Google map):