Sorting Algorithms

11/24/2024 1
 Sorting: Definition

Sorting: an operation that segregates items

into groups according to specified criterion.

Unsorted: A = { 3 1 6 2 1 3 4 5 9 0 }

Ascending: A = { 0 1 1 2 3 3 4 5 6 9 }

11/24/2024 2
Sorting
• Sorting = ordering.
• Sorted = ordered based on a particular way.
• Generally, collections of data are presented in a sorted
manner.
• Examples of Sorting:
• Words in a dictionary are sorted (and case distinctions are
ignored).
• Files in a directory are often listed in sorted order.
• The index of a book is sorted (and case distinctions are ignored).

11/24/2024 3
 Review of Complexity
• Most of the primary sorting algorithms run on different
space and time complexity.
• Time Complexity is defined to be the time the computer
takes to run a program (or algorithm in our case).
• Space complexity is defined to be the amount of memory
the computer needs to run a program.

11/24/2024 4
 Complexity (cont.)
• Complexity in general, measures the algorithms efficiency in
internal factors such as the time needed to run an algorithm.
• External Factors (not related to complexity):
 Size of the input of the algorithm
 Speed of the Computer
 Quality of the Compiler

11/24/2024 5
 O(n), Ω(n), & Θ(n)

An algorithm or function f(n) is O(g(n)) whenever f(n)'s
rate of growth is less than or equal to g(n)'s rate.

An algorithm or function f(n) is Ω(g(n)) whenever f(n)'s
rate of growth is greater than or equal to g(n)'s rate.

An algorithm or function f(n) is Θ(g(n)) if and only if
the rate of growth of f(n) is equal to g(n).

11/24/2024 6
Sorting Algorithms
• There are many sorting algorithms, such as:
Selection Sort
Insertion Sort
Bubble Sort
Merge Sort
Quick Sort
• The first three are the foundations for faster and more
efficient algorithms.

11/24/2024 7
Selection Sort: Idea
1. We have two group of items:
• sorted group, and
• unsorted group
2. Initially, all items are in the unsorted group. The sorted
group is empty.
• We assume that items in the unsorted group are unsorted.
• We have to keep items in the sorted group sorted.

11/24/2024 8
Selection Sort: Cont’d
1. Select the “best” (eg. smallest) item from the unsorted
group, then put the “best” item at the end of the sorted
group.
2. Repeat the process until the unsorted group becomes
empty.

11/24/2024 9
 Sorted Unsorted
Selection (min)
23 78 45 8 32 56 Original List

8 78 45 23 32 56 After pass 1

8 23 45 78 32 56 After pass 2

8 23 32 78 45 56 After pass 3

8 23 32 45 78 56
After pass 4

8 23 32 45 56 78
After pass 5

11/24/2024 10
Selection Sort (cont’)
selectionSort(a[n])
{ for (i = 0 to <n-1)
{
min = i;
for (j = i+1 to <n)
if (a[j] < a[min])
min = j;
swap(a[i], a[min]);
}
}
11/24/2024 11
 Selection Sort -- Analysis
• In general, we compare keys and move items (or exchange
items) in a sorting algorithm (which uses key comparisons).
 So, to analyze a sorting algorithm we should count the
number of key comparisons and the number of moves.
• In selectionSort function, the outer for loop executes n-1
times.
• We invoke swap function once at each iteration.
 Total Swaps: n-1
 Total Moves: 3*(n-1) (Each swap has three
moves)
11/24/2024 12
Selection Sort – Analysis (cont’)
• The inner for loop executes the size of the unsorted part
minus 1 (from 1 to n-1), and in each iteration we make one
key comparison.
 number of key comparisons = 1+2+...+n-1 = n*(n-1)/2
 So, Selection sort is O(n2)
• The best case, the worst case, and the average case of the
selection sort algorithm are same.  all of them are O(n2)

11/24/2024 13
Selection sort is Theta, best is
worst
• This means that the behavior of the selection sort algorithm
does not depend on the initial organization of data.
• Since O(n2) grows so rapidly, the selection sort algorithm is
appropriate only for small n.
• Although the selection sort algorithm requires O(n2) key
comparisons, it only requires O(n) moves.
• A selection sort could be a good choice if data moves are
costly but key comparisons are not costly (short keys, long
records).

11/24/2024 14
Insertion Sort
• Insertion sort is a simple sorting algorithm that is appropriate
for small inputs.
Most common sorting technique used by card players.
• The list is divided into two parts: sorted and unsorted.
• In each pass, the first element of the unsorted part is picked
up, transferred to the sorted sublist, and inserted at the
appropriate place.
• A list of n elements will take at most n-1 passes to sort the
data.
11/24/2024 15
Insertion Sort: Idea
1. We have two group of items:
• sorted group, and
• unsorted group
2. Initially, all items in the unsorted group and the sorted group is
empty.
• We assume that items in the unsorted group are unsorted.
• We have to keep items in the sorted group sorted.
3. Pick any item from, then insert the item at the right position in the
sorted group to maintain sorted property.
4. Repeat the process until the unsorted group becomes empty.
11/24/2024 16
 Sorted Unsorted

23 78 45 8 32 56 Original List
23 78 45 8 32 56 After pass 1

23 45 78 8 32 56 After pass 2

8 23 45 78 32 56 After pass 3

8 23 32 45 78 56
After pass 4
8 23 32 45 56 78
After pass 5
11/24/2024 17
Insertion sort
insertionSort(a[n])
{
for (i = 1 to <n)
{
tmp = a[i];
for (j=i; j>0 && tmp < a[j-1]; j--)
{
a[j] = a[j-1];
}
a[j] = tmp;
}
}
11/24/2024 18
INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8
do key ← A[ j ] key
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
11/24/2024 19
Insertion Sort – Analysis
• Running time depends on not only the size of the array but
also the contents of the array.
• Best-case:  O(n)
Array is already sorted in ascending order.
Inner loop will not be executed.
The number of moves: 2*(n-1)  O(n)
The number of key comparisons: (n-1)  O(n)

11/24/2024 20
Insertion Sort – Analysis
• Worst-case:  O(n2)
• Array is in reverse order:
• Inner loop is executed i-1 times, for i = 2,3, …, n
• The number of moves: 2*(n-1)+(1+2+...+n-1)= 2*(n-1)+ n*(n-1)/2
 O(n2)
• The number of key comparisons: (1+2+...+n-1)= n*(n-1)/2
 O(n2 )
• Average-case:  O(n2)
• We have to look at all possible initial data organizations.
• So, Insertion Sort is O(n2)
11/24/2024 21
 Analysis of insertion sort
• Which running time will be used to characterize this algorithm?
• Best, worst or average?
• Worst:
• Longest running time (this is the upper limit for the algorithm)
• It is guaranteed that the algorithm will not be worse than this.
• Sometimes we are interested in average case. But there are
some problems with the average case.
• It is difficult to figure out the average case. i.e. what is average input?
• Are we going to assume all possible inputs are equally likely?
• In fact for most algorithms average case is same as the worst case.

11/24/2024 22
Bubble Sort: Idea
• Idea: bubble in water.
• Bubble in water moves upward. Why?
• How?
• When a bubble moves upward, the water from above will move
downward to fill in the space left by the bubble.

11/24/2024 23
 Bubble Sort: Example

1 40 2 1 43 3 65 0 -1 58 3 42 4

2 2 1 40 3 43 0 -1 58 3 42 4 65

3 1 2 3 40 0 -1 43 3 42 4 58 65

4 1 2 3 0 -1 40 3 42 4 43 58 65

• Notice that at least one element will be in the correct

position each iteration.
11/24/2024 24
 Bubble Sort: Example
5 1 2 0 -1 3 3 40 4 42 43 58 65

6 1 0 -1 2 3 3 4 40 42 43 58 65

7 0 -1 1 2 3 3 4 40 42 43 58 65

8 -1 0 1 2 3 3 4 40 42 43 58 65

11/24/2024 25
Bubble sort Right to Left
• Bubble(A[n])
{
For(i=n-1 to 0) //i--
For(j=0 to i-1) //i++
If(A[j]>A[j+1]
Swap(A[j],A[j+1])
}

11/24/2024 26
Bubble sort Left to Right
23 78 45 8 32 56

8 23 78 45 32 56

8 23 32 78 45 56

8 23 32 45 78 56

8 23 32 45 56 78

11/24/2024 27
Bubble Sort – Analysis
• Best-case:  O(n)
• Array is already sorted in ascending order.
• The number of moves: 0  O(1)
• The number of key comparisons: (n-1)  O(n)

11/24/2024 28
Bubble Sort – Analysis
• Worst-case:  O(n2)
• Array is in reverse order:
• Outer loop is executed n-1 times,
• The number of moves: 3*(1+2+...+n-1) = 3 * n*(n-1)/2  O(n2)
• The number of key comparisons: (1+2+...+n-1)= n*(n-1)/2  O(n2)
• Average-case:  O(n2)
• We have to look at all possible initial data organizations.
• So, Bubble Sort is O(n2)

11/24/2024 29
Searching Algorithms
• The input data can be sorted or unsorted data
• There are mainly two types of searching algorithms
• Linear search or binary search algorithms
• You are required to understand at least; any two linear
search algorithms and any two binary search algorithms
• At the end, be able to determine the space and time
complexities of the algorithms
• NB: See you in examination with your searching algorithms.
11/24/2024 30