SORTING

Sorting is the process of arranging the elements in some logical order. Sorting are classified into following categories: External sorting: deals with sorting of the data stored in data files. This method is used when the volume of data is very large and cannot be held in computer main memory. Internal sorting: deals with sorting the data held in memory of the computer

SORTING METHODS
1. 2. 3. 4. 5. 6. 7. 8. 9. Bubble sort Selection sort Insertion sort Bucket sort Merge sort Quick sort Heap sort Tree sort Shell sort

BUBBLE SORT
It requires n-1 passes to sort an array. In each pass every element a[i] is compared with a[i+1], for i=0 to (n-k), where k is the pass number and if they are out of order i. e. if a[i]>a[i+1], they are swapped. This will cause the largest element move up or bubble up. Thus after the end of the first pass the largest element in the array will be placed in the nth position and on each successive pass, the next largest element is placed at position (n-1),(n2).,2 respectively

BUBBLE SORT
Pass1. Step1. if a[0]>a[1] then swap a[0] and a[1]. Step2. if a[1]>a[2] then swap a[1] and a[2]. Stepn-1. if a[n-2]>a[n-1] then swap a[n-2] and a[n-1]. Pass2. Step1. if a[0]>a[1] then swap a[0] and a[1]. Step2. if a[1]>a[2] then swap a[1] and a[2]. Stepn-2. if a[n-3]>a[n-2] then swap a[n-3] and a[n-2].

BUBBLE SORT
. . Pass k. Step1. if a[0]>a[1] then swap a[0] and a[1]. Step2. if a[1]>a[2] then swap a[1] and a[2]. Step n-k. if a[n-k+1]>a[n-k] then swap a[n-k+1] and a[n-k]. Pass n-1 Step 1 if a[0]>a[1] then swap a[0] and a[1].

BUBBLE SORT
Example: 12 40 3 2 15

12 40 3 2 15
Given array

12 40 3 2 15
Pass 1

12 3 40 2 15

12 3 2 40 15

12 3 2 15 40

BUBBLE SORT
3 12 2 15 40
Pass 2

3 2 12 15 40

3
2

12
15 40

BUBBLE SORT
2 3
12 15 40

2
3 12 15 40

2
3

12
15 40

Pass 3

pass 4

ALGORITHM
Bubblesort(a,n) for k=1 to (n-1) by 1 do for j=0 to (n-k-1) by 1 do if(a[j]>a[j+1]) then set temp=[j] set a[j]=a[j+1] set a[j]=temp endif endfor Endfor end

ANALYSIS OF BUBBLE SORT

First pass require n-1 comparison Second pass requires n-2 comparison Kth pass requires n-k comparisons Last pass requires only one comparison Therefore total comparisons are: F(n)=(n-1)+(n-2)++(n-k)+3+2+1 =n(n-1)/2 =O(n2)

SELECTION SORT
The selection sort also requires (n-1) passes to sort an array. In the first pass, find the smallest element from elements a[0], a[1], a[2],.., a[n-1] and swap with the first element, i.e. a[0]. In the second pass, find the smallest element from elements a[1], a[2], a[3].. a[n-1] and swap with a[1] and so on.

SELECTION SORT
Pass1. Find the location loc of the smallest element in the entire array, i.e. a[0],[1],a[2]a[n-1] Interchange a[0] & a[loc]. Then a[0] is trivially sorted. Pass2. Find the location loc of the smallest element in the entire array, i.e. a[1],a[2]a[n-1] Interchange a[1] & a[loc]. Then a[0], a[1] are sorted. Passk. Find the location loc of the smallest element in the entire array, i.e. a[k],a[k+1],a[k+2]a[n-1] Interchange a[k] & a[loc]. Then a[0],a[1],a[2],a[k] are sorted. Passn-1. Find the location loc of the smaller of the element a[n-2],a[n-1] Interchange a[n-2] & a[loc]. Then elements a[0],a[1],a[2].a[n-1].

EXAMPLE
Given array: 20 35 40 100 3 10 15 a[0] 20
Pass 1:

a[1] 35

a[2] 40

a[3] 100

a[4] 3

a[5] 10

a[6] 15

a[0] 20

a[1] 35

a[2] 40

a[3] 100

a[4] 3

a[5] 10

a[6] 15

Loc=4

Interchange elements a[0] & a[4] i.e. 20 and 3

SELECTION SORT
Pass 2

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 35 40 100 20 10 15

Loc=5

Interchange elements a[1] & a[5] i.e. 35 and 10

Pass3

a[0] a[1] a[2] a[3] a[4] a[5] a[6]

3 10 40 100 20 35 15

Loc=6

Interchange elements a[2] & a[6] i.e. 40 and 15

Pass 4

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 10 15 100 20 35 15

Loc=4

Interchange elements a[3] & a[4] i.e. 100 and 20

SELECTION SORT
Pass 5

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 10 15 20 100 35 40

Interchange elements a[4] & a[5] i.e. 100 and 35

Loc=5

Pass 6

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 10 15 20 35 100 40

Interchange elements a[5] & a[6] i.e. 100 and 40

Loc=6

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 10 15 20 35 40 100

ALGORITHM
Smallestelement(a,n,k,loc) Here a is linear array of size n. this sub algorithm finds the location loc of smallest element among a[k-1],a[k+1],a[k+2]a[n-1]. Temporary variable small is used to hold the current smllest element nd j is used loop control variable. Begin set small=a[k-1] set loc=k-1 for j=k to (n-1) by 1 do if(a[j]<small) then set small = a[j] set loc=j endif endfor end

ALGORITHM
Selectionsort(a,n) Here a is the linear array with n elements in memory. This algorithm sorts elements into ascending order. It uses a temporary variable temp to facilitate the exchange of two values and variable I is used loop control variable Begin for i=1 to (n-1) by 1 do call smllest element(a,n,I,loc) set temp=a[i-1] set a[i-1]=a[loc] set a[loc]=temp endfor end

ANALYSIS OF SELECTION SORT

First pass require n-1 comparison to find the location loc of smallest element Second pass requires n-2 comparison Kth pass requires n-k comparisons Last pass requires only one comparison Therefore total comparisons are: F(n)=(n-1)+(n-2)++(n-k)+3+2+1 =n(n-1)/2 =O(n2)

INSERTION SORT
This algorithm is very popular with bridge players when they sort their cards. In this procedure, we pick up a particular value and then insert it at the appropriate place in the sorted sub list. This algorithm also requires n-1 passes

INSERTION SORT
Pass1: a[1] is inserted either before or after a[0] so that a[0] and a[1] are sorted. Pass2: a[2] is inserted either before a[0] or between a[0] and a[1] or after a[1] so that the elements a[0], a[1], a[2] are sorted. Pass3: a[3] is inserted either before a[0] or between a[0] and a[1] or between a[1] and a[2] or after a[2] so that the elements a[0], a[1], a[2], a[3] are sorted. Passk: a[k] is inserted in proper place in the sorted sub array a[0], a[1], a[2],a[k-1] so that the elements a[0], a[1], a[2],a[k-1],a[k] are sorted. Passn-1: a[n-1] is inserted in proper place in the sorted sub array a[0], a[1], a[2],a[n-2] so that the elements a[0], a[1], a[2],a[n-1] are sorted.

EXAMPLE
Given array: 35 20 40 100 3 10 15 a[0] 35
Pass 1:

a[1] 20

a[2] 40

a[3] 100

a[4] 3

a[5] 10

a[6] 15

a[0] 35

a[1] 20

a[2] 40

a[3] 100

a[4] 3

a[5] 10

a[6] 15

Since a[1]< a[0] insert element a[1] before a[0]

SELECTION SORT
Pass 2

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 20 35 40 100 3 10 15

Since a[2]>a[1] no action is performed

Pass3

a[0] a[1] a[2] a[3] a[4] a[5] a[6]

20 35 40 100 3 10 15

Since a[3]>a[2] no action is performed

Pass 4

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 20 35 40 100 3 10 15

Since a[4] is less than a[3], a[2], a[1] as well as a[0] therefore insert a[4] before a[0]

SELECTION SORT
Pass 5

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 20 35 40 100 10 15

Since a[5] is less than a[4], a[3], a[2] as well as a[1] therefore insert a[5] before a[1]

Pass 6

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 10 20 35 40 100 15

Since a[6] is less than a[5], a[4], a[3] as well as a[2] therefore insert a[6] before a[2]

a[0] a[1] a[2] a[3] a[4] a[5] a[6] 3 10 15 20 35 40 100

ALGORITHM
insertionsort(a,n) Here a is the linear array with n elements in memory. This algorithm sorts elements into ascending order. It uses a temporary variable temp to facilitate the exchange of two values and variable j and k are used loop control variables. Begin for k=1 to (n-1) by 1 do set temp=a[k] set a[j]=k-1 while((temp<a[j]) and j>=0) do set a[j+1]=a[j] set j=j-1 endwhile set a[j+1]=temp endfor end

ANALYSIS OF INSERTION SORT

The worst case performance occurs when the elements of the input array are in descending order First pass require 1 comparison to find the location loc of smallest element Second pass requires 2 comparison Kth pass requires k-1 comparisons Last pass requires (n-1) comparison Therefore total comparisons are: F(n)=1+2+3+..+(n-k)+.+(n-3)+(n-2)+(n-1) =n(n-1)/2 =O(n2)

Bucket/Radix sort
This is used by most of the people when sorting a list of names in alphabetical order. The procedure is: First, the names are grouped according to the first letter, thus the names are arranged in 26 classes, one for each letter of alphabet. first class consists of those names that begin with letter A, the second class consists of those names that begins with letter B, and so on. Next, the names are grouped according to the second letter. After this step, the list of name will be sorted on first two letter. This process is continued for number of times depending on the length of the names with maximum letters.

Bucket/Radix sort
Since there are 26 letter of alphabet, we make use of 26 buckets, one for each letter of the alphabet. After grouping these names according to their specific letter, we collect them according to order of bucket. This new list becomes input for the next pass i.e to separate them on the next letter from left. To sort decimal number where base (radix) is 10, we need 10 buckets are numbered from 0-9. Unlike sorting names, decimal numbers are sorted from right to left i.e. first on unit digits, then on ten digit and so on.

example
321, 150, 235, 65, 573, 789, 928, 542
321 150 235 65 573 789 928 65 150
0

542
Input

321
1

542
2 Pass 1

573
3 4

235
5 6 7

928
8

789
9

150

321
542

573
235 65 928 789
Input 0 1

928

321
2

235
3

542 150
4 5

65
6

573
7

789
8 9

Pass 2

321

928
235

542
150 65 573 789
Input

573

65
0

150
1

235
2

321
3 4

542
5 6

789
7 8

928
9

Pass 3

Bucket/Radix sort
After pass three, when the numbers are collected, they re in following order 65, 150, 235, 321, 542, 573, 789, 928 thus the numbers are sorted

Algorithm
Bucketsort(a,n) Here a is linear array of integer with n elements, the variable digitcount is used to store the number of digits in the largest number in order to control the number of passes to be performed. Begin find the largest number of the array set digitcount=no. of digits of the largest no. for pass=1 to digitcount by 1 do initialize buckets for i=1 to n-1 by 1 do set digit=obtain digit no. pass of a[i] put a[i] in bucket no. digit increment bucket count for bucket no. digit endfor collect all the numbers from buckets in order endfor end

Analysis of bucket sort

Let us suppose the number of digits in the largest element of the given array is S. the number of passes to be performed is n. Then , the umber of comparisons, f(n), needed to sort the given array are f(n)=<=n*s*10 ----here 10 base of decimal number Though, s is independent of n , but if s=n, then in worst case. f(n)=O(n2) But on the other hand, if s=log10n, then f(n)=O(nlog10n) Thus from the above discussion , we conclude that bucket sort performs well only when the number of digits in the elements are very small.

Merge Sort

Divide and Conquer

Divide-and-conquer method for algorithm design:
Divide: If the input size is too large to deal with in a straightforward manner, divide the problem into two or more disjoint subproblems Conquer: Use divide and conquer recursively to solve the subproblems Combine: Take the solutions to the subproblems and merge these solutions into a solution for the original problem

Merge Sort Algorithm

Divide: If S has at least two elements (nothing needs to be done if S has zero or one elements), remove all the elements from S and put them into two sequences, S1 and S2 , each containing about half of the elements of S. (i.e. S1 contains the first n/2elements and S2 contains the remaining n/2elements). Conquer: Sort sequences S1 and S2 using Merge Sort. Combine: Put back the elements into S by merging the sorted sequences S1 and S2 into one sorted sequence

Merge Sort: Algorithm

Merge-Sort(A, p, r) if p < r then q(p+r)/2 Merge-Sort(A, p, q) Merge-Sort(A, q+1, r) Merge(A, p, q, r)

Merge(A, p, q, r) Take the smallest of the two topmost elements of sequences A[p..q] and A[q+1..r] and put into the resulting sequence. Repeat this, until both sequences are empty. Copy the resulting sequence into A[p..r].

MergeSort (Example) - 1

MergeSort (Example) - 2

MergeSort (Example) - 3

MergeSort (Example) - 4

MergeSort (Example) - 5

MergeSort (Example) - 6

MergeSort (Example) - 7

MergeSort (Example) - 8

MergeSort (Example) - 9

MergeSort (Example) - 10

MergeSort (Example) - 11

MergeSort (Example) - 12

MergeSort (Example) - 13

MergeSort (Example) - 14

MergeSort (Example) - 15

MergeSort (Example) - 16

MergeSort (Example) - 17

MergeSort (Example) - 18

MergeSort (Example) - 19

MergeSort (Example) - 20

MergeSort (Example) - 21

MergeSort (Example) - 22

Merge Sort Revisited

To sort n numbers
if n=1 done! recursively sort 2 lists of numbers n/2 and n/2 elements merge 2 sorted lists in Q(n) time

Strategy
break problem into similar (smaller) subproblems recursively solve subproblems combine solutions to answer

Analysis of merge Sort

The major work is done in the merge procedure, which is an O(n) operation. Merge procedure is called from merge sort procedure after the array is divided into two halves, and each halves has been sorted. In each recursive calls, one for the left half and one for the right half, array is divided into four segments. At each level number of segments doubles, therefore the total division are log2n , hence total number of comparisons f(n)= n*log2n =O(nlog2n)

Quick-Sort

Another divide-and-conquer sorting algorihm To understand quick-sort, lets look at a high-level description of the algorithm 1) Divide : If the sequence S has 2 or more elements, select an element x from S to be your pivot. Any arbitrary element, like the last, will do. Remove all the elements of S and divide them into 3 sequences: L, holds Ss elements less than x E, holds Ss elements equal to x G, holds Ss elements greater than x 2) Recurse: Recursively sort L and G 3) Conquer: Finally, to put elements back into S in order, first inserts the elements of L, then those of E, and those of G. Here are some diagrams....

Idea of Quick Sort

1) Select: pick an element

2) Divide: rearrange elements so that x goes to its final position E 3) Recurse and Conquer: recursively sort

In-Place Quick-Sort
Divide step: l scans the sequence from the left, and r from the right.

A swap is performed when l is at an element larger than the pivot and r is at one smaller than the pivot.

In Place Quick Sort (contd)

A final swap with the pivot completes the divide step

Algorithm
quicksort(a , l, r) Begin If(l<r) then splitarray(a,l,r,loc) quicksort(a,l,loc-1) quicksort(a,loc+1,r) endif end

//partition the array //recursively sort left sub array //recursively sort right sub array

Analysis of the quick sort

To find the location of the element that splits the array into two sections is an O(n) operation, because every element in the array is compared to the dividing element. After the division each section is examined separately. If the array is split approximately in half (which is no usually), then there will be log2n splits. therefore the total comparisons are: f(n)=n*log2n = O(nlog2n)

Tree Sort
Tree sort method, in order to sort an array of size n in ascending order, works in following phases: 1. Build a binary search tree by using n calls to insert operation 2. Print elements using inorder traversal.

Tree Sort
Array : 50, 60, 40, 45, 31, 75, 53, 65 Solution: When these elements are inserted in binary search tree one by one, the final binary search tree looks like in next slide;

Given Array: 50, 60, 40, 45, 31, 75, 53, 65

50

40

60

31

45

53

75

65

Inorder traversal produces the following listing of elements: 31, 40, 45, 50, 53, 60, 65, 75

Tree Sort v/s Quick Sort

In tree sort , the first item is inserted into the root of the tree and all subsequent elements are partitioned to the left or right depending on their relation to the first element. This is analogous to quick sort, if the first element in the array is used as the pivot element for partitioned. Further in tree sort, second element becomes the root of the sub tree. It becomes the pivot element to partition all subsequent elements in that subtree. This is analogous to quick sort for partitioning of one of the sublist.

Analysis of Tree Sort

All the comparisons for tree sort are done during insert() calls. The insert() function does the same number of comparisons as quick sort. Therefore , tree sort has the same running time as quick sort, i.e. average case complexity is O(nlogn) and worst case complexity is O(n2)

Tree Sort
It does not require that all the elements to be present in the array at the beginning of sorting Elements can be added gradually as they become available It also works on a linked structure that allows easier insertions and deletions than a contiguous list

 Invented by Donald Shell in 1959, the shell sort is the most efficient of the O(n2) class of sorting algorithms. Of course shell sort is also the most complex of the O(n2) algorithms. This algorithm is similar to bubble sort in the sense it also moves elements by exchanges. It begins by comparing elements that are at a distance d. With this the elements that are quite away from their place will move rapidly than the simple bubble sort. In each pass, the value of d is reduced to half i.e. di+1=(di+1)/2 In each pass, each element is compared with element that is located d position away from it, and exchange is made if required. The next iteration starts with new value of d. The algorithm terminates when d=1

Shell sort

Example
12,9,-10,22,2,35,40 Starting value of d=n/2=7/2=3 12 9 -10

22 2 35 40
Given array

12 9 -10 22 2
35 40

12 9 -10 22 2 35 40
Pass 1

12 2 -10 22 9 35 40

12 2 -10 22 9 35 40

12 2 -10 22 9 35 40

Example
Starting value of di+1=(di+1)/2=(3+1)/2=2

12 2 -10

22 9 35 40

-10 2 12 22 9
35 40

-10 2 12 22 9 35 40
Pass 2

-10 2 9 22 12 35 40

-10 2 9 22 12 35 40

-10 2 9 22 12 35 40

Example
Starting value of di+1=(di+1)/2=(2+1)/2=1

-10 2 9

22 12 35 40

-10 2 9 22 12
35 40

-10 2 9 22 12 35 40
Pass 3

-10 2 9 22 12 35 40

-10 2 9 12 22 35 40

-10 2 9 22 12 35 40

Analysis of shell sort

It is difficult to predict the complexity of shell sort, as it is very difficult to show the effect of one pass on another pass. One thing is very clear that if the new distance d is computed using above formula, then the number of passes will approximately be log2d since d=1 will complete the sort. Empirical studies have shown that the worst case complexity of shell sort is O(n2)