COMPLEXITY , SEARCHING & SORTING

TIME COMPLEXITY:

The Number of machine instructions which a program executes during its running
time is called its “Time complexity”. This number depends primarily on the size of the
program’s input.

 Time taken by a program is the sum of the compile time and runtime. In the time
complexity, we consider run time only.

 Time complexity of an algorithm signifies the total time required by the program
to run till its completion.

 The time complexity of algorithms is most commonly expressed using the big O
notation. It's an asymptotic notation to represent the time complexity.

 Time Complexity is most commonly estimated by counting the number of

elementary steps performed by any algorithm to finish execution.

And since the algorithm's performance may vary with different types of input data, hence
for an algorithm we usually use the worst-case Time complexity of an algorithm
because that is the maximum time taken for any input size.

Types of Notations for Time Complexity

1. Big Oh
2. Big Omega
3. Big Theta
4. Little Oh
5. Little Omega
 O (expression):

It indicates the maximum required by an algorithm for all input values. It

represents the worst case of an algorithm's time complexity. To denote, we use O
-notation. For a given function g(n), we denote by O(g(n)) (Pronounced “big-oh
of g of n”).

Omega (expression)

It indicates the minimum time required by an algorithm for all input values. It
represents the best case of an algorithm's time complexity. To denote, we use Ω -
notation. For a given function g(n), we denote by Ω(g(n)) (pronounced “big-
omega of g of n”).

Theta (expression)

It consist of all the functions that lie in both O(expression) and

Omega(expression).It indicates the average bound of an algorithm. It represents
the average case of an algorithm's time complexity. To denote, we use Θ-
notation. For a given function g(n), we denote by Θ(g(n)) (pronounced “big-
theta of g of n”).

Calculating Time Complexity

Example:
#include <stdio.h>
int main ( )
{
printf("Hello World");
}
Output:
Hello World

In above code “Hello World!!!” print only once on a screen. So, time
complexity is constant: O (1) i.e. every time constant amount of time require
 executing code, no matter which operating system or which machine configurations
you are using.

Example:
#include <stdio.h>
void main ( )
{
int i, n = 8;
for (i = 1; i <= n; i++)
{
printf("Hello Word !!!");
}

}
Output:
Hello Word !!!Hello Word !!!Hello Word !!!Hello Word !!!
Hello Word !!!Hello Word !!!Hello Word !!!Hello Word !!!

In above code “Hello World!!!” will print N times. The time complexity
for the above algorithm will be Linear. So, time complexity of above code is
O(N).

Example:
#include <stdio.h>
void main ( )
{
int i, n = 5;
for (i = 1; i <= n; i++)
{
for (i = 1; i <= n; i++)
{
printf ( " Hello Word !!! " ) ;
}

}
 In above code “Hello World!!!” will print N * N times. The time complexity for
the above code will be Quadratic. The running time of the two loops is proportional to
the square of N. When N doubles, the running time increases by N * N. So, time
2
complexity of above code is O(N ).

NOTE: In general, doing something with every item in one dimension is linear, doing
something with every item in two dimensions is quadratic, and dividing the
working area in half is logarithmic.
SPACE COMPLEXITY:

Whenever a solution to a problem is written some memory is required to

complete. For any algorithm memory may be used for the following:

1. Variables (This include the constant values, temporary values)

2. Program Instruction
3. Execution

Space complexity is the amount of memory used by the algorithm (including the input
values to the algorithm) to execute and produce the result.

This number computed with respect to the size of the input data. The space
occupied by an algorithm is determined by the number and sizes of the variables and data
structures used by the algorithm.

Calculating the Space Complexity

For calculating the space complexity, we need to know the value of memory used
by different type of data type variables, which generally varies for different operating
systems, but the method for calculating the space complexity remains the same.

Type Size

bool, char, unsigned char, signed char, __int8 1 byte

__int16, short, unsigned short, wchar_t, __wchar_t 2 bytes

float, __int32, int, unsigned int, long, unsigned long 4 bytes

double, __int64, long double, long long 8 bytes

Now let's learn how to compute space complexity by taking a few examples:

{
int z = a + b + c;
return(z);
}

In the above expression, variables a, b, c and z are all integer types, hence they
will take up 4 bytes each, so total memory requirement will be (4(4) + 4) = 20 bytes, this
additional 4 bytes is for return value. And because this space requirement is fixed for the
above example, hence it is called Constant Space Complexity.

Let's have another example, this time a bit complex one,

int sum(int a[ ] , int n)
{
int x = 0; // 4 bytes for x
for(int i = 0; i < n; i++) // 4 bytes for i
{
x = x + a[i];
}
return(x);
}

 In the above code, 4*n bytes of space is required for the array a[] elements.
 4 bytes each for x, n, i and the return value.

Hence the total memory requirement will be (4n + 12), which is increasing
linearly with the increase in the input value n, hence it is called as Linear Space
Complexity.
Similarly, we can have quadratic and other complex space complexity as well, as
the complexity of an algorithm increases.
The following is the list that illustrates the time complexities of various algorithms.
SEARCHING:

 A Searching is a process used to find the location of a target element among a list of
objects.
 The search is claimed as successful or unsuccessful depending on whether the targeted
element is found or not.
 Depending on the location of the records to be searched, searching techniques are
classified into two types:
1. Linear Search
2. Binary Search

LINEAR SEARCH:

 This is a simple search technique also known as sequential search.

 The Sequential search is used whenever the list is unordered.
 This simplest method can be used either for sorted or unsorted list of elements.
 We start searching for the target from the beginning of the list, and we continue until
we find the target or until we are sure that it is not there in the list.

ALGORITHM: LINEAR – SEARCH ( array a[ ] , num , key )

Step 1: Start
Step 2: Set i from 0 to num
Step 3: check if i > num then go to step 8 else go to step 4
Step 4: check if A [ i ] = = key then go to step 6
Step 5: Set i to i + 1 and go to Step 3
Step 6: Print Element key Found at index i and go to step 8
Step 7: Print element not found
Step 8: Exit
Example:
Assume the listed elements, in which we would like to search the element 56.
The element 56 is compared with each of the element from beginning. Once the element
is found then it stops searching.
PROGRAM:

/*Program to implement linear search*/

#include<stdio.h>
#include<conio.h>
#include<stdlib.h>
void main ( )
{
int a[10] , i , key , n ;
clrscr ( ) ;
printf ( " \n How many Elements you want to Enter : " ) ;
scanf ( " %d " , & n ) ;
printf ( " \n Enter the elements: " ) ;
for ( i = 0 ; i < n ; i + + )
{
scanf ( " %d " , &a [ i ] ) ;
}
printf ( " \n Enter the Key element to search: " ) ;
scanf ( " %d " , &key ) ;
for ( i = 0 ; i < n ; i + + )
{
if ( a [ i ] = = key )
{
printf ( " \n Element %d found in position %d " , key , i + 1 ) ;
getch ( ) ;
exit ( 1 ) ;
}
}
printf ( " \n Element %d not Found " , key ) ;
getch ( ) ;
}

OUTPUT:

How many Elements you want to enter: 6

Enter the elements: 23 67 89 56 19 12
Enter the Key element to search: 56
Element 56 found in position 4
 Advantages:

 It is simplest technique.
 The elements in the list can be in any order.

Disadvantages:

 This method is inefficient when large numbers of elements are present in list.
Because time taken for search is more.

Complexity of linear search:

 The worst and average case complexity of linear search is O ( n ) , where n is the
number of elements present in the list.
 If the data are distributed randomly, an average of N/2comparison is needed.
 The worst case is that the value is not there in the list or it is the last element,
which takes n comparisons to search.

BINARY SEARCH:

 To implement binary search method, the elements must be in sorted order.

 The most efficient technique that can be applied to list of sorted elements is
Binary Search.
 This technique is faster than the other searching techniques.
 In this method, the given sorted list of elements will be divided into three parts.
 The first element is treated as low and last as high, if low is less than high then a
mid will be considered by taking ( low + last ) / 2.
 The key is first compared with the middle record (mid) of array.
 If the match is found, the key index is returned.
 If it is doesn’t match, then the required key must be either in the left half or right
half of the middle.
 If the key is less than the middle record, the key is searched in the left part, else it
is checked in the right part of the middle element.
ALGORITHUM:

Binary search ( a , n , key) [ low = 0 , high = n - 1 ]

Step 1 – Start
Step 2 – Check while ( low <= high) if it is true go to step 3 otherwise go to step 4
Step 3 – Find mid value (mid = (low + high)/2 )
Step 4 − If match found (key = = a [ mid ] )
then print position of key item, and go to step 7.
Step 5 – [ Compare to search for item]
if key < a [ mid ] then
high = mid -1
else
low = mid + 1
go to step 2 ( Repeat until match found )
Step 6 – [ for Un successful search] print match not found.
Step 7 – exit
PROGRAM :

/*Program to implement the binary search algorithm-non-recursive*/

#include<stdio.h>
#include<conio.h>
#include<stdlib.h>
void main ( )
{
int a[10] , i , key , n , low , mid , high ;
clrscr ( ) ;
printf ( " \n How many Elements you enter : " ) ;
scanf ( " %d " , & n ) ;
printf ( " \n Enter the Elements ascending order: " ) ;
for ( i = 0 ; i < n ; i + + )
{
Scanf ( " % d " , &a [ i ] ) ;
}
printf ( " \n Enter the Key Element to search : " ) ;
scanf ( " %d " , &key ) ;
low = 0 ;
high = n - 1 ;
while ( low < = high )
{
mid = ( low + high ) / 2 ;
if ( key = = a [ mid ] )
{
printf ( " \n Element %d found in %d position " , key , mid + 1 ) ;
getch ( ) ;
exit ( ) ;
}
else
if ( key < a [ mid ] )
high = mid - 1 ;
else
low = mid + 1 ;
}
printf ( " \n Element %d not found " , key ) ;
getch ( ) ;
}
OUTPUT:

How many Elements you enter: 6

Enter the Elements ascending order: 5 7 9 10 12 15
Enter the Key Element to search: 10
Element 10 found in 4 position

Note:
if the elements entered by the user are distinct then the above code gives the
position of occurrence of the no. if some of them are repeated then it may give be
any position of the same no., but may not be the first occurrence

/*Program to implement the binary search algorithm using recursion*/

#include<stdio.h>
#include<conio.h>
#include<stdlib.h>
int key ;
int binary ( int [ ] , int , int ) ;
void main ( )
{
int a [ 10 ] , i , n , p ;
clrscr ( ) ;
printf ( " \n How many Elements you enter: " ) ;
scanf ( " %d " , & n ) ;
printf ( " \n Enter the Elements ascending order: " ) ;
for ( i = 0 ; i < n ; i + + )
{
Scanf ( " % d " , &a [ i ] ) ;
}
printf ( " \n Enter the Key Element to search : " ) ;
scanf ( " %d " , &key ) ;
p = binary ( a , 0 , n - 1 ) ;
if ( p = = 0 )
printf ( " \n Element not Found " ) ;
else
printf ( " \n Element %d Found in %d position " , key , p ) ;
 getch ( ) ;
}
binary ( int a [ 10 ] , int low , int high )
{
int mid ;
if ( low < = high )
{
mid = ( low + high ) / 2 ;
if ( key = = a [ mid ] )
return mid + 1 ;
else
if ( key < a [ mid ] )
binary ( a , low , mid - 1 ) ;
else
binary ( a , mid + 1 , high ) ;
}
else
return 0 ;
}

OUTPUT:
How many Elements you enter: 6
Enter the Elements: 2 4 6 8 10 12
Enter the Key Element to search: 8
Element 8 Found in 4 position.

Complexity of Binary Search:

The binary search algorithm is having the efficiency of O ( log 2 n ) .

Advantages:
 When the number of elements in the list is large, binary search executes faster
than linear search.
Disadvantages:
 To implement this method the elements in the list must always in sorted order
otherwise it fails.
SORTING:

Sorting is a method of arranging elements in a file or list in ascending or descending

order.
OR

The process through which data are arranged according to their values in some
order is called Sorting.

A Sorting Algorithm is used to rearrange a given array elements according to a

comparison operator on the elements.
Sorting makes handling of records in a file or list easier. These are very common
applications in computer science.

Sorting can be classified into two types, Internal sorting and External sorting.

Internal Sorting:

An internal sort is any data sorting process in a file or list which is stored in the main
memory of a computer.
Internal Sorting algorithms are of following types:
Bubble Sort, Insertion Sort, Quick Sort, Heap Sort, merge Sort, Radix Sort,
Selection sort.

External sorting:

An external sort is data sorting process in a file or list which is stored in the
secondary memory.

External Sorting algorithms are of following types:

K-way Merge Sort, Multi-way / k-way merge sort, Balanced Merge Sort, Poly-
Phase Merge Sort
Need of External Sorting:

 Entire data to be sorted might not fit in the available internal memory Considerations
 When data resides in internal memory
 Data access time << Computation time
 Need to reduce the number of CPU operations
 When data resides on external storage devices
 Data access time >> Computation time
 Need to reduce disk accesses

Difference between internal and external sorting:

 Internal Sorting takes place in the main memory of a computer. The internal sorting
methods are applied to small collection of data. It means that, the entire collection of
data to be sorted in small enough that the sorting can take place within main memory.
 The External sorting methods are applied only when the number of data elements to
be sorted is too large. These methods involve as much external processing as
processing in the CPU. This sorting requires auxiliary (secondary) storage.
 Internal sorting takes small input, whereas external sorting can take as much as large
input...

We have the following sorting techniques

1. Bubble Sort
2. Selection Sort
3. Insertion Sort
4. Merge Sort
5. Quick Sort
6. Radix sort
7. Heap sort
8. Tree sort
BUBBLE SORT:

 It is also known as exchange sort.

 The way of putting the largest element at the highest index in the array uses an algorithm
called Bubble sort.
 In this sorting technique the adjacent elements are compared and exchanged if they are
not in order.
 If there are n elements then there will be n-1 passes or iterations to place each element in
proper positions.
 Bubble sort is very inefficient and should only be used on small data sets.

Bubble sort is performed as given below, consider an array of n elements a[0] to a[n-1]

 During the first pass the first and second elements are compared and swapped if they are
not in order. After the first pass the largest element will move to its correct position i.e.
the last position.
 Next, the 2 nd, 3 rd and 3 rd , 4th and so on are compared and the process will be repeated
until comparison of all the elements is finished.
 The same process will be repeated for the next passes until all the elements are placed in
proper positions.

ALGORITHM:

Assume list is an array of n elements.

for i - 0 to n-1 do
for j - 0 to n-1-i do
if (a[j] > a[j+1]) then
Exchange a[j] < = > a[j+1].

This algorithm needs n-1 iterations for n number of elements. In the first iteration we do
n-1 comparisons among n elements and in the second iteration we need n-2 comparisons
among n-1 elements and so on one comparison among 2 elements.
 So total no. of comparisons are

Complexity can be expressed by using Big-oh (O) notation, i.e., O (n2)

The time complexity can be expressed by using the three cases

Best Case : The minimum number of comparisons needed to sort or list is
already sorted. [Big-omega]: O(n)
Average Case : The Average number of comparisons needed to sort. [Big-theta]: O(n2)
Worst Case : The maximum number of comparisons needed to sort. [ Big-O ]: O(n2)

The space complexity for Bubble Sort is O(1), because only a single additional memory
space is required i.e. for temp variable.

Implementation of Bubble Sort

/*PROGRAM TO IMPLEMENT BUBBLE SORT*/

# include < stdio.h >

# include < conio.h >
# include < stdlib.h >
void main ( )
{
int a [ 10 ] , i , n , f , j , temp ;
clrscr ( ) ;
printf ( " \n How many elements you want to enter: " ) ;
scanf ( " %d " , & n ) ;
printf ( " \n Enter elements in order " ) ;
for ( i = 0 ; i < n ; i + + )
scanf ( " %d " , & a [ i ] ) ;
for ( i = 0 ; i < n ; i + + )
{
f=0;
for ( j = 0 ; j < ( n - i - 1 ) ; j + + )
{
if ( a [ j ] > a [ j + 1 ] )
{
f=1;
temp = a [ j ] ;
a[j]=a[j+1];
a [ j + 1 ] = temp ;
}
if ( f = = 0 )
break ;
}
}
printf ( " \n Sorted elements are: \n " ) ;
for ( i = 0 ; i < n ; i + + )
{
printf (" \t %d " , a [ i ] ) ;
}
getch ( ) ;
}
 OUTPUT:

How many elements you want to enter: 6

Enter elements:
23 67 89 56 19 12
Sorted elements are:
12 19 23 56 67 89

Advantages of bubble sort:

 It is relatively easy to write and understand.

 The performance is good for nearly sorted list.
 Works well for smaller list of elements.

Disadvantages of bubble sort:

 It is relatively slow algorithm.

 It is not used for sorting the list of larger size.
 It is inefficient algorithm because the number of iterations increases with the increase in
number of elements to be sorted.
 The time taken for sorting is more than selection and insertion sort.
SELECTION SORT:

This is the simplest method of sorting. For sorting in ascending order the
following process is used,
In each pass, the smallest element in the list placed in right position if there are N
elements in the list.
 In this, the 1st element is compared with all the other elements.
 If the 1st element is found to be greater than the compared element then they are
interchanged.
 So after the first iteration the smallest element is placed at the 1st position.
 The same procedure is repeated for the 2nd, 3rd elements and so on.
 This process is continued until all the elements are sorted.

Note: if there are N elements in list, we require N - 1 passes.

Algorithm: Selection sort ( a [ ] , N )

Step -1 : Repeat thru step 2 for i = 0 to n-1 do

Step - 2: Repeat thru step 3 for j = 1 to n do

Step - 3: check if ( a [ i ] > a [ j ] ) do
Exchange a [ i ] and a [ j ]
Step - 4: [ End of the Algorithm ]
/*PROGRAM TO IMPLEMENT THE SELECTION SORT*/
# include < stdio.h >
# include < conio.h >
# include < stdlib.h >
void main ( )
{
 int a [ 10 ] , i , n , j , temp ;
clrscr ( ) ;
printf ( " \n How many elements you want to enter: " ) ;
scanf ( " %d " , & n ) ;
printf ( " \n Enter elements in order " ) ;
for ( i = 0 ; i < n ; i + + )
scanf ( " %d " , & a [ i ] ) ;
for ( i = 0 ; i < n - 1 ; i + + )
{
for ( j = i + 1 ; j < n ; j + + )
{
if ( a [ i ] > a [ j ] )
{
temp = a [ i ] ;
a[i]=a[j];
a [ j ] = temp ;
}
}
}
printf ( " \n Sorted elements are: \n " ) ;
for ( i = 0 ; i < n ; i + + )
{
printf (" \t %d " , a [ i ] ) ;
}
getch ( ) ;
}

OUTPUT:

How many elements you want to enter: 6

Enter the elements: 23 67 89 56 8 12
Sorted Elements are:
8 12 23 56 67 89
Advantages of Selection sort:

Selection sort minimizes data movement by putting one entry in its first position
at every pass. Hence this algorithm is useful for continuous list of large elements where
movement of entries is expensive.

Disadvantages of Selection sort:

If the elements are sorted using linked lists, then insertion sort is more efficient
than selection sort.

Complexity of Selection Sort:

Selection Sort requires two nested for loops to complete itself, Hence for a given
input size of n, following will be the time and space complexity for selection sort
algorithm:

 Worst Case Time Complexity [ Big-O ]: O(n2)

 Best Case Time Complexity [Big-omega]: O(n2)
 Average Time Complexity [Big-theta]: O(n2)
 Space Complexity: O(1)
INSERTION SORT:

In each pass an item is compared with its predecessors and if it is not in the right
position, it is placed in the right place among the elements being compared.

For sorting in ascending order the following process is used,

 In first pass, second element is compared with the first. If the second number is
smaller than first number, then it is inserted in first position by pushing first number
to second position. After first step, first two elements of an array will be sorted.
 In second pass, 3rd element is compared with before two numbers. If third element is
smaller than first element, it is inserted in the position of first element. If third
element is larger than first element but, smaller than second element, it is inserted in
the position of second element. If third element is larger than both the elements, it is
kept in the position as it is. After second step, first three elements of an array will be
sorted.
 In third pass, 4th number is compared with previous three elements and the same
procedure is applied and that element is inserted in the proper position among the
numbers which are compared. After third step, first four elements of an array will be
sorted.
 This process is continued for ( N - 1 ) passes.
/* PROGRAM TO IMPLEMENT THE INSERTION SORT */
# include < stdio.h >
# include < conio.h >
# include < stdlib.h >
void main ( )
{
int a [ 10 ] , i , n , j , temp ;
clrscr ( ) ;
printf ( " \n How many elements you want to enter: " ) ;
scanf ( " %d " , & n ) ;
printf ( " \n Enter elements in order " ) ;
for ( i = 0 ; i < n ; i + + )
scanf ( " %d " , & a [ i ] ) ;
for ( i = 0 ; i < n ; i + + )
 {
j=i;
while ( ( j > 0 ) && ( a [ j – 1 ] > a [ j ] ) )
{
temp = a [ j ] ;
a[j]=a[j-1];
a [ j - 1 ] = temp ;
j=j–1;
}
}
printf ( " \n Sorted elements are: \n " ) ;
for ( i = 0 ; i < n ; i + + )
{
printf (" \t %d " , a [ i ] ) ;
}
getch ( ) ;
}

OUTPUT:

How many elements you want to enter: 5

Enter the elements: 5 4 3 2 1
Sorted Elements are:
1 2 3 4 5

Advantages of insertion sort:

 It is simple sorting algorithm in which the elements are sorted by considering one
item at a time.
 The implementation is simple.
 It is efficient for small data set and for the data set that had been substantially
sorted before.
 It reduces unnecessary travel through the array.
 It requires constant amount of extra memory space.
Disadvantages of insertion sort:

 It is less efficient on list containing more number of elements.

 As the number of elements increases the performance of the program would be
slow.

Complexity of insertion Sort:

 This algorithm is not suitable for large data sets as its Best case [Big-omega]
complexity of insertion sort is O(n) and average [Big-theta] and worst case [ Big-
O ] complexity are of Ο(n2), where n is the number of items.

 Space Complexity: O(1)