SEARCHING AND

SORTING
ALGORITHMS
Nagur Ramesh

NAGUR RAMESH 1
SEARCH ALGORITHMS

▪search algorithm – method for finding an item or

group of items with specific properties within a
collection of items
▪

2
SEARCHING ALGORITHMS
▪ linear search
• brute force search (aka British Museum algorithm)
• list does not have to be sorted
▪ bisection search
• list MUST be sorted to give correct answer
• saw two different implementations of the algorithm

3
LINEAR SEARCH
ON UNSORTED LIST: RECAP
def linear_search(L, e):
found = False
for i in range(len(L)):
if e == L[i]:
found = True
return found

▪ must look through all elements to decide it’s not there

▪ O(len(L)) for the loop * O(1) to test if e == L[i]
▪ overall complexity is O(n) – where n is len(L)

4
LINEAR SEARCH
ON SORTED LIST: RECAP
def search(L, e):
for i in range(len(L)):
if L[i] == e:
return True
if L[i] > e:
return False
return False

▪ must only look until reach a number greater than e

▪ O(len(L)) for the loop * O(1) to test if e == L[i]
▪ overall complexity is O(n) – where n is len(L)

5
 USE BISECTION SEARCH:
RECAP
1. Pick an index, i, that divides list in half
2. Ask if L[i] == e
3. If not, ask if L[i] is larger or smaller than e
4. Depending on answer, search left or right half of L for
e

A new version of a divide-and-conquer algorithm

▪ Break into smaller version of problem (smaller list), plus
some simple operations
▪ Answer to smaller version is answer to original problem

6
BISECTION SEARCH
IMPLEMENTATION: RECAP
def bisect_search2(L, e):
def bisect_search_helper(L, e, low, high):
if high == low:
return L[low] == e
mid = (low + high)//2
if L[mid] == e:
return True
elif L[mid] > e:
left to search
if low == mid: #nothing
return False
else:
return bisect_search_helper(L, e, low, mid - 1)
else:
return bisect_search_helper(L, e, mid + 1, high)
if len(L) == 0:
return False
else:
return bisect_search_helper(L, e, 0, len(L) - 1)

7
COMPLEXITY OF BISECTION
SEARCH: RECAP
▪ bisect_search2 and its helper
• O(log n) bisection search calls
• reduce size of problem by factor of 2 on each step
• pass list and indices as parameters
• list never copied, just re-passed as pointer
• constant work inside function
• 🡪 O(log n)

8
SEARCHING A SORTED LIST
-- n is len(L)
▪ using linear search, search for an element is O(n)
▪ using binary search, can search for an element in O(log n)
• assumes the list is sorted!
▪ when does it make sense to sort first then search?
• SORT + O(log n) < O(n) 🡪 SORT < O(n) – O(log n)
• when sorting is less than O(n)
•NEVER TRUE!
• to sort a collection of n elements must look at each one at
least once!

9
SORT ALGORITHMS
▪ Want to efficiently sort a list of entries (typically
numbers)
▪Will see a range of methods, including one that is
quite efficient

10
MONKEY SORT
▪aka bogosort, stupid
sort, slowsort,
permutation sort,
shotgun sort
▪ to sort a deck of cards
• throw them in the air
• pick them up
• are they sorted?
• repeat if not sorted

11
COMPLEXITY OF BOGO SORT
def bogo_sort(L):
while not is_sorted(L):
random.shuffle(L)

▪ best case: O(n) where n is len(L) to check if sorted

▪ worst case: O(?) it is unbounded if really unlucky

12
BUBBLE SORT
▪ compare consecutive pairs
of elements
▪swap elements in pair such
that smaller is first
▪ when reach end of list,
start over again
▪ stop when no more swaps
have been made
▪largest unsorted element
always at end after pass, so
at most n passes
CC-BY Hydrargyrum
https://commons.wikimedia.org/wiki/File:Bubble_sort_animation.gif 13
COMPLEXITY OF BUBBLE SORT
def bubble_sort(L):
swap = False
while not swap:
swap = True
for j in range(1, len(L)):
if L[j-1] > L[j]:
swap = False
temp = L[j]
L[j] = L[j-1]
L[j-1]
= temp
▪ inner for loop is for doing the comparisons
▪outer while loop is for doing multiple passes until no more
swaps
▪ O(n2) where n is len(L)
to do len(L)-1 comparisons and len(L)-1 passes
14
SELECTION SORT
▪ first step
• extract minimum element
• swap it with element at index 0
▪ subsequent step
• in remaining sublist, extract minimum element
• swap it with the element at index 1
▪ keep the left portion of the list sorted
• at i’th step, first i elements in list are sorted
• all other elements are bigger than first i elements

15
ANALYZING SELECTION SORT
▪ loop invariant
◦ given prefix of list L[0:i] and suffix L[i+1:len(L)], then
prefix is sorted and no element in prefix is larger than
smallest element in suffix
1. base case: prefix empty, suffix whole list – invariant
true
2. induction step: move minimum element from suffix
to end of prefix. Since invariant true before move,
prefix sorted after append
3. when exit, prefix is entire list, suffix empty, so sorted

16
 COMPLEXITY OF SELECTION
SORT
def selection_sort(L):
suffixSt = 0
while suffixSt != len(L):
for i in range(suffixSt, len(L)):
if L[i] < L[suffixSt]:
L[suffixSt], L[i] = L[i], L[suffixSt]
suffixSt += 1

▪ outer loop executes len(L) times

▪ inner loop executes len(L) – i times
▪ complexity of selection sort is O(n2) where n is len(L)

17
MERGE SORT
▪ use a divide-and-conquer approach:
1. if list is of length 0 or 1, already sorted
2. if list has more than one element, split into two lists,
and sort each
3. merge sorted sublists
1. look at first element of each, move smaller to end of the
result
2. when one list empty, just copy rest of other list

18
MERGE SORT
▪ divide and conquer
unsorted

unsorted unsorted

unsorted unsorted unsorted unsorted

unsor unsor unsor unsor unsor unsor unsor unsor
ted ted ted ted ted ted ted ted

merge merge merge merge merge merge merge merge

▪ split list in half until have sublists of only 1 element

22
MERGE SORT
▪ divide and conquer
unsorted

unsorted unsorted

unsorted unsorted unsorted unsorted

sort sort sort sort sort sort sort sort

merge merge merge merge

▪ merge such that sublists will be sorted after merge

23
MERGE SORT
▪ divide and conquer
unsorted

unsorted unsorted

sorted sorted sorted sorted

merge merge

▪ merge sorted sublists

▪ sublists will be sorted after merge

21
MERGE SORT
▪ divide and conquer
unsorted

sorted sorted

merge

▪ merge sorted sublists

▪ sublists will be sorted after merge

22
MERGE SORT
▪ divide and conquer – done!
sorted

23
 EXAMPLE OF MERGING
Left in list 1 Left in list 2 Compare Result
[1,5,12,18,19,20] [2,3,4,17] 1, 2 []
[5,12,18,19,20] [2,3,4,17] 5, 2 [1]
[5,12,18,19,20] [3,4,17] 5, 3 [1,2]
[5,12,18,19,20] [4,17] 5, 4 [1,2,3]
[5,12,18,19,20] [17] 5, 17 [1,2,3,4]
[12,18,19,20] [17] 12, 17 [1,2,3,4,5]
[18,19,20] [17] 18, 17 [1,2,3,4,5,12]
[18,19,20] [] 18, -- [1,2,3,4,5,12,17]
[] [] [1,2,3,4,5,12,17,18,19,20]

24
MERGING SUBLISTS STEP
def merge(left, right):
result = []
i,j = 0,0
while i < len(left) and j <
len(right): if left[i] <
right[j]:
result.append(left[i])
i += 1
else:
result.append(right[j])
j += 1
while
while(j(i< <len(right)):
len(left)):
result.append(right[j])
result.append(left[i])
j += 1
i
return result

25
COMPLEXITY OF
MERGING SUBLISTS STEP
▪ go through two lists, only one pass
▪ compare only smallest elements in each sublist
▪ O(len(left) + len(right)) copied elements
▪ O(len(longer list)) comparisons
▪ linear in length of the lists

26
MERGE SORT ALGORITHM
-- RECURSIVE
def merge_sort(L):
if len(L) < 2:
return L[:]
else:
middle = len(L)//2
left = merge_sort(L[:middle])
right = merge_sort(L[middle:])
return merge(left, right)
▪ divide list successively into halves
▪depth-first such that conquer smallest pieces down
one branch first before moving to larger pieces
27
 84165920
Merge
1468&0259
01245689

8416 5920
Merge Merge
48&16 59&02
1468 0259

84 16 59 20

Merge Merge Merge Merge

48 16 59 02

8 4 1 6 5 9 2 0

base base base base base base base base

case case case case case case case case

28
COMPLEXITY OF MERGE SORT
▪ at first recursion level
• n/2 elements in each list
• O(n) + O(n) = O(n) where n is len(L)
▪ at second recursion level
• n/4 elements in each list
• two merges 🡪 O(n) where n is len(L)
▪ each recursion level is O(n) where n is len(L)
▪ dividing list in half with each recursive call
• O(log(n)) where n is len(L)
▪ overall complexity is O(n log(n)) where n is len(L)
29
SORTING SUMMARY
-- n is len(L)
▪ bogo sort
• randomness, unbounded O()
▪ bubble sort
• O(n2)
▪ selection sort
• O(n2)
• guaranteed the first i elements were sorted
▪ merge sort
• O(n log(n))
▪ O(n log(n)) is the fastest a sort can be
30
KEY TOPICS
▪ represent knowledge with data structures
▪ iteration and recursion as computational metaphors
▪ abstraction of procedures and data types
▪organize and modularize systems using object classes
and methods
▪ different classes of algorithms, searching and sorting
▪ complexity of algorithms

31
OVERVIEW OF COURSE
▪learn computational modes of
thinking
▪begin to master the art of
computational problem solving
▪make computers do what you want
them to do

Hope we have started you down the

path to being able to think and act
like a computer scientist
32
WHAT DO COMPUTER
SCIENTISTS DO?
▪ they think computationally
◦ abstractions, algorithms,
automated execution
▪ just like the three r’s: reading,
‘riting, and ‘rithmetic – Alan Turing Ada Lovelace
Image in the Public Image in the Public

computational thinking is Domain, courtesy of

Wikipedia Commons.
Domain, courtesy of
Wikipedia Commons.

becoming a fundamental skill that

every well-educated person will
need

33
 THE THREE A’S OF
COMPUTATIONAL
▪ abstraction
Person

THINKING
◦ choosing the right abstractions
◦ operating in multiple layers of abstraction
MITPerson

Student
simultaneously
◦ defining the relationships between the abstraction UG Grad
layers
▪ automation
◦ think in terms of mechanizing our abstractions
◦ mechanization is possible – because we have precise
and exacting notations and models; and because there is
some “machine” that can interpret our notations
▪ algorithms
◦ language for describing automated processes
◦ also allows abstraction of details
◦ language for communicating ideas & processes

34
ASPECTS OF
COMPUTATIONAL THINKING
▪how difficult is this problem O(log n) ; O(n) ;
and how best can I solve it?
◦ theoretical computer science O(n log n) ;
gives precise meaning to these O(n2); O(cn)
and related questions and their
answers
▪ thinking recursively
◦ reformulating a seemingly
difficult problem into one
which we know how to solve
◦reduction, embedding,
transformation, simulation

Image Licensed CC-BY, Courtesy of Robson# on Flickr.

35