Data Structures and Algorithm

Lecture 5 – Algorithm Design Techniques

Algorithm Design Techniques
Objectives
• Brute Force
• Greedy Algorithms
• Divide and Conquer
• Dynamic Programming
• Randomized Algorithms
• Backtracking Algorithms
Algorithm Design Techniques
Optimization Problem
• Algorithms are used to solve problems.
• Involves taking sequence of decisions.
• Optimization problems have results that range from:
• Minimum results
• Maximum/ best results.
• Optimization strives to get better and refined solutions to
problems.
 Brute Force Algorithm
Photosynthesis
Brute Force Algorithm is a problem solving technique that relies on
computing power and trying every possibility rather than advanced
techniques to improve efficiency.
• Also called exhaustive search.
• Brute force algorithms are slow.
• The don't employ a lot of logic.
• But, brute force algorithms are fairly easy to implement as a first pass
solution
• Backtracking is a form of a brute force algorithm.
Brute Force Algorithm…
Task
Question
• What is brute force algorithm?
• How is it implemented in C/C++?
• Describe applications of brute force algorithms.
 Greedy Algorithm
Photosynthesis
Greedy Algorithm a technique that makes the optimal choice at each step
as it attempts to find the overall optimal way to solve the entire problem.

• They seek to optimize a function by making choices which are the best
locally or immediately.
• They do not consider the global problem.
• The result is a good solution but not necessarily the best one.
• The greedy algorithm does not always guarantee the optimal solution
however it generally produces solutions that are very close in value to
the optimal.
Greedy Algorithm…
Task
Question
• What greedy algorithm?
• How is it implemented in C/C++?
• Describe applications of greedy algorithms.
 Divide and Conquer
Photosynthesis
Divide and Conquer is a method of recursively breaking down a problem
into two or more sub-problems of the same or related type, until these
become simple enough to be solved directly.
Task
• Break down
• Solve the problem
Partition Task
• Combine.

Complete
smaller tasks
(Conquer)

Consolidate

Completed Task!
Divide and Conquer…
Task
Question
• What is divide and conquer algorithm?
• How is it implemented in C/C++?
• Describe applications of divide and conquer algorithms.
 Dynamic Programming
Photosynthesis
Dynamic Programming is a is a method for solving a complex problem by
breaking it down into a collection of simpler sub-problems, solving each of
those sub-problems just once, and storing their solutions using a memory-
based data structure.

• Data structures such as arrays can be used in the storage.

• Finds all the possible feasible solutions.
• Picks the best solution.
• Consumes more time compared to greedy method.

Solved Problem
Start
Dynamic Programming…
Task
Question
• What is dynamic programming?
• How is it implemented in C/C++?
• Describe applications of dynamic programming.
 Randomized Algorithms
Photosynthesis
Randomized Algorithms is a problem solving technique that uses some
degree of “randomness” as part of its logic
• A randomized algorithm is an algorithm where a random number is used
to make a decision at least once during the execution of the algorithm.
• The running time of the algorithm depends not only on the particular
input, but also on the random numbers that occur.
• The random numbers to decide what to do next anywhere in its logic.

Input Output
Algorithm

Random Number
Randomized Algorithms…
Task
Question
• What is randomized algorithm?
• How is it implemented in C/C++?
• Describe applications of randomized algorithms.
 Backtracking Algorithms
Photosynthesis
Backtracking Algorithms are set of steps for solving problems recursively
by trying to build a solution incrementally by removing or ignoring those
solutions that fail to satisfy the constraints of problems.

• Define a solution space or search space for the problem.

• Organize the solution space so that it can be reached easily. The typical
organization is either a graph or a tree.
• This space is searched in a depth-first manner beginning at a start node.

Success

Start

Failure

Failure Success
Backtracking Algorithms…
• Problem space consists of states(nodes) and actions(paths that
lead to new states).
• During search there are live nodes and expansion node(E-node).
• When in a node, can only see paths to connected nodes.
• If a node only leads to failure go back to its "parent“ node.
• Try other alternatives.
• If these all lead to failure then more backtracking may be
necessary.
• Searching terminates when we have found the answer or
when we run out of live nodes to back up to.
Backtracking Algorithms…
• Goals of Backtracking
• Find a path to success
• Find all paths to success
• Find the best path to success
• Not all problems are exactly alike, and finding one success node
may not be the end of the search.
Backtracking Algorithm…
Task
Question
• What is backtracking algorithm?
• How is it implemented in C/C++?
• Describe applications of backtracking algorithms.
Applications of Algorithm Design Techniques
• SUDOKU Problem
• Entering numbers in a logic-based, combinatorial puzzle.
• 8 Queens Problem
• Place 8 queen pieces on a chess board so that none of them can attack one
another.
• CD/DVD Problem
• Storing different file types with different sizes optimally.
• Knapsack Problem
• Keeping items of different sizes optimally. Example Different fruits with
different sizes.
• Knights Tour Problem
• Sequence of moves of a knight on a chessboard such that the knight visits
every square exactly once.
• …
Research
• Algorithm design for industrial applications.
• Applications of recursive backtracking.
• Implementation of all the algorithm design techniques.
Review Questions
• Define:
• Recursion
• Backtracking
• Algorithm Design
• Research on a C program that implements any backtracking
algorithm.
• Assume that you are in Eldoret and your final destination is
Mombasa. How can you implement a backtracking algorithm in
this journey so that next time you travel, you have better
travelling experience.
• What are the goals of backtracking algorithm.
• List the disadvantages of Brute Force algorithm.
• Discuss the algorithm design techniques.
References
• Online
• E-resources

• Books
• E-books
• Library