CMP 318 – ANALYSIS & COMPLEXITY OF ALGORITHMS

LECTURE NOTES

BY

CHARLES OKONJI
 Sorting Algorithm
A Sorting Algorithm is used to rearrange a given array or list of elements according to a comparison operator on the elements. The
comparison operator is used to decide the new order of elements in the respective data structure. For example, the list of characters
below is sorted in increasing order of their ASCII values, such that the character with a lesser ASCII value will be placed first than the
character with a higher ASCII value.

Selection Sort
This simple and efficient sorting algorithm works by repeatedly selecting the smallest (or largest) element from the unsorted portion of the
list and moving it to the sorted portion of the list.

Let us illustrate using, as an example, the array arr[] = {64, 25, 12, 22, 11}

First pass:
 For the first position in the sorted array, the whole array is traversed from index 0 to 4 sequentially. The first position where 64 is
stored presently, after traversing whole array it is clear that 11 is the lowest value.
 Thus, replace 64 with 11. After one iteration 11, which happens to be the least value in the array, tends to appear in the first position
of the sorted list.
 Sorting Algorithm
Second Pass:
 For the second position, where 25 is present, again traverse the rest of the array in a sequential manner.
 After traversing, we found that 12 is the second lowest value in the array and it should appear at the second place in the array, thus
swap these values.

Third Pass:
 Now, for third place, where 25 is present again traverse the rest of the array and find the third least value present in the array.
 While traversing, 22 came out to be the third least value and it should appear at the third place in the array, thus swap 22 with
element present at third position.

Fourth pass:
 Similarly, for fourth position traverse the rest of the array and find the fourth least element in the array
 As 25 is the 4th lowest value hence, it will place at the fourth position.

Fifth Pass:
 At last the largest value present in the array automatically get placed at the last position in the array
 The resulted array is the sorted array.
 Sorting Algorithm
procedure selection sort
list : array of items
n : size of list
for i = 1 to n - 1
/* set current element as minimum*/
min = i
/* check the element to be minimum */
for j = i+1 to n
if list[j] < list[min] then
min = j;
end if
end for
/* swap the minimum element with the current element*/
if indexMin != i then
swap list[min] and list[i]
end if
end for
end procedure
 Sorting Algorithm

SELECTION SORT(arr, n)

Step 1: Repeat Steps 2 and 3 for i = 0 to n-1

Step 2: CALL SMALLEST(arr, i, n, pos)
Step 3: SWAP arr[i] with arr[pos]
[END OF LOOP]
Step 4: EXIT

SMALLEST (arr, i, n, pos)

Step 1: [INITIALIZE] SET SMALL = arr[i]
Step 2: [INITIALIZE] SET pos = i
Step 3: Repeat for j = i+1 to n
if (SMALL > arr[j])
SET SMALL = arr[j]
SET pos = j
[END OF if]
[END OF LOOP]
Step 4: RETURN pos
 Sorting Algorithm
Below is the C++ implementation of the Selection Sort approach:

// C++ program for implementation of

// selection sort

#include <bits/stdc++.h>

using namespace std;

// Function for Selection sort

void selectionSort(int arr[], int n)

int i, j, min_idx;

// One by one move boundary of

// unsorted subarray

for (i = 0; i < n - 1; i++) {

// Find the minimum element in

// unsorted array

min_idx = i;

for (j = i + 1; j < n; j++) {

if (arr[j] < arr[min_idx])

min_idx = j;

}
 Sorting Algorithm
// Swap the found minimum element

// with the first element

if (min_idx != i)

swap(arr[min_idx], arr[i]);

// Function to print an array

void printArray(int arr[], int size)

int i;

for (i = 0; i < size; i++) {

cout << arr[i] << " ";

cout << endl;

// Driver program

int main()

{
 Sorting Algorithm
int arr[] = { 64, 25, 12, 22, 11 };

int n = sizeof(arr) / sizeof(arr[0]);

// Function Call

selectionSort(arr, n);

cout << "Sorted array: \n";

printArray(arr, n);

return 0;

Output
Sorted array:

11 12 22 25 64

Complexity Analysis of Selection Sort

Time Complexity: The time complexity of Selection Sort is O(N2) as there are two nested loops:
 One loop to select an element of Array one by one = O(N)
 Another loop to compare that element with every other Array element = O(N)

Therefore, overall complexity = O(N) * O(N) = O(N*N) = O(N2)

 Sorting Algorithm
Advantages of Selection Sort Algorithm
 Simple and easy to understand.
 Works well with small datasets.

Disadvantages of the Selection Sort Algorithm

 Selection sort has a time complexity of O(n^2) in the worst and average case.
 Does not work well on large datasets.
 Does not preserve the relative order of items with equal keys which means it is not stable.

Bubble Sort
This algorithm works by repeatedly swapping the adjacent elements if they are in the wrong order. This algorithm is not suitable for large
data sets as its average and worst-case time complexity is quite high.

• In this algorithm,
o traverse from left and compare adjacent elements and the higher one is placed at right side.
o In this way, the largest element is moved to the rightmost end at first.

Let us illustrate using an array arr[] = {14, 33, 27, 35, 10}
 Sorting Algorithm

Bubble sort starts with very first two elements, comparing them to check which one is greater. In this case, value 33 is greater than 14, so
it is already in sorted locations.

Next, we compare 33 with 27. We find that 27 is smaller than 33 and these two values must be swapped. The new array should look like
this −

Next we compare 33 and 35. We find that both are in already sorted positions.
 Sorting Algorithm

Then we move to the next two values, 35 and 10. We know then that 10 is smaller 35. Hence they are not sorted, so we swap these
values.

Thus, after one iteration, the array should look like this −

After the second iteration, it should look like this −

 Sorting Algorithm

Notice that after each iteration, at least one value moves at the end.

And when there's no swap required, bubble sorts learns that an array is completely sorted.

Thus, the output is {10, 14, 27, 33, 35}

 Sorting Algorithm
Suppose arr is an array of n elements. The assumed swap function in the algorithm will
swap the values of given array elements.

begin BubbleSort(arr)
for all array elements
if arr[i] > arr[i+1]
swap(arr[i], arr[i+1])
end if
end for
return arr
end BubbleSort
 Sorting Algorithm
for (j = 0; j < n - i - 1; j++) {
if (arr[j] > arr[j + 1]) {
swap(&arr[j], &arr[j + 1]);
swapped = true;
}
}

// If no two elements were swapped by inner loop,

// then break
if (swapped == false)
break;
}
}

// Function to print an array

void printArray(int arr[], int size)
{
int i;
for (i = 0; i < size; i++)
printf("%d ", arr[i]);
}
 Sorting Algorithm
// Driver program to test above functions
int main()
{
int arr[] = { 14, 33, 27, 35, 10};
int n = sizeof(arr) / sizeof(arr[0]);
bubbleSort(arr, n);
printf("Sorted array: \n");
printArray(arr, n);
return 0;
}

Output
Sorted array: 10, 14, 27, 33, 35

Complexity Analysis of Bubble Sort:

Time Complexity: O(N2)

Advantages of Bubble Sort:

 Bubble sort is easy to understand and implement.
 It does not require any additional memory space.
 It is a stable sorting algorithm, meaning that elements with the same key value maintain their relative order in the sorted output.

Disadvantages of Bubble Sort:

 Bubble sort has a time complexity of O(N2) which makes it very slow for large data sets.
 Bubble sort is a comparison-based sorting algorithm, which means that it requires a comparison operator to determine the relative
order of elements in the input data set. It can limit the efficiency of the algorithm in certain cases.