Algorithm

An algorithm is a step-by-step set of instructions designed to perform a specific task or

solve a particular problem.

Time and space trade off:

The time and space trade-off is a concept in computing where we balance the speed
(time) and the memory usage (space) of an algorithm. Improving one often affects the
other. Here’s how it works:

1. Time (Speed): Refers to how fast an algorithm runs. We aim to minimize the time it
takes to reach a result.
2. Space (Memory): Refers to the amount of memory an algorithm uses to store data.
We aim to use as little memory as possible.

The Trade-Off

• More Space, Less Time: Using extra memory can help an algorithm run faster. For
example, storing precomputed values in a lookup table (caching) can speed up
access to data, reducing the time required for calculations. However, this approach
uses more memory.
• Less Space, More Time: Using less memory can slow down the algorithm. For
example, if we don’t store intermediate results, the algorithm might need to
recalculate values repeatedly, which can slow down execution but save memory.

The efficiency of an algorithm is all about how fast it works and how much memory it
uses.

Why Efficiency is Important

Efficient algorithms can:

• Run faster, which is important when you have a lot of data or need quick results
(like in video games or large apps).
• Use less memory, which is helpful on devices with limited storage or memory.
Measuring Efficiency

We usually look at two main things:

1. Time Complexity (Speed): How much time does the algorithm take as the amount
of data grows?
a. Fast algorithms: These take little time even if there's a lot of data.
b. Slow algorithms: These can take a long time if there's a lot of data.
2. Space Complexity (Memory): How much memory does the algorithm use as it
works on larger amounts of data?
a. Low-memory algorithms: Use very little memory, which is good for memory-
limited devices.
b. High-memory algorithms: Use more memory, which can be fine if memory
isn't an issue.

The rate of growth describes how quickly the time or space requirements of an algorithm
increase as the input size grows. It’s a key concept in analyzing algorithm efficiency,
because it helps us predict performance on larger inputs.

Why Rate of Growth Matters

As input size increases (more data), we want to know how much more time or memory the
algorithm will need. A slower rate of growth means the algorithm handles larger inputs
more efficiently.

Big-O Notation and Rate of Growth

Big-O notation is used to describe the rate of growth of an algorithm in the worst case.
Here are some common rates of growth, from fastest to slowest:

1. O(1) - Constant Growth:

a. No matter the input size, the time or space doesn’t increase.
b. Example: Accessing an item in an array.
2. O(log n) - Logarithmic Growth:
a. Grows slowly as input size increases. Doubling the input size only adds a
small amount of time or space.
b. Example: Binary search.
 3. O(n) - Linear Growth:
a. Time or space increases in direct proportion to input size.
b. Example: Going through each item in a list once.
4. O(n log n) - Log-Linear Growth:
a. Grows a bit faster than linear but much slower than quadratic.
b. Example: Efficient sorting algorithms like mergesort.
5. O(n²) - Quadratic Growth:
a. Time or space grows proportional to the square of the input size.
b. Example: Checking every pair in a list (like in bubble sort).
6. O(2^n) - Exponential Growth:
a. Grows very quickly. Even slightly larger inputs make time or space explode.
b. Example: Certain recursive algorithms without optimization.
7. O(n!) - Factorial Growth:
a. The slowest rate. Very small inputs make it unmanageable.
b. Example: Algorithms that check all possible combinations, like the brute-
force approach to the traveling salesman problem.

Asymptotic notation:
1. Big O Notation O(f(n))O(f(n))O(f(n))

• Definition: Describes an upper bound on the growth rate of a function.

It provides a worst-case scenario for the performance of an algorithm.

• Formal Definition: A function g(n) is O(f(n)) if there exist constants

C>0 and n0 such that for all n>n0 :

g(n)≤C⋅f(n)

• Example: If an algorithm has a runtime of g(n)=3n2+2n+1, it can be said

to be O(n2).
2. Omega Notation Ω(f(n))\Omega(f(n))Ω(f(n))

• Definition: Describes a lower bound on the growth rate of a function. It

provides the best-case scenario for the performance of an algorithm.
•
• Formal Definition: A function g(n) is said to be Ω(f(n)) if there exist
constants C>0 and n0 such that for all n>n0 :

g(n)≥C⋅f(n)g(n)

• Example: If an algorithm has a runtime of g(n)=ng(n) = ng(n)=n, it can

be said to be Ω(n).

3. Theta Notation Θ(f(n))\Theta(f(n))Θ(f(n))

• Definition: Describes a tight bound on the growth rate of a function. It

indicates that the function grows at the same rate as f(n).

• Formal Definition: A function g(n) is Θ(f(n)) if there exist constants

C1>0, C2 >0, and n0 such that for all n>n0 :

C1⋅f(n) ≤ g(n) ≤ C2⋅f(n)C

• Example: If an algorithm has a runtime of g(n)=5n+3, it can be said to be

Θ(n).

Sorting and searching algorithm:

Merge Sort
 • Description: Divides the array into halves, sorts each half, and then
merges them back together.
• Time Complexity:
 Best Case: O(n log ⁡n)
 Average Case: O(n log⁡ n)
 Worst Case: O(n log⁡ n)

Quick Sort

• Description: Selects a 'pivot' element and partitions the array into

elements less than and greater than the pivot, then recursively sorts the
partitions.
• Time Complexity:
 Best Case: O(n log⁡ n)
 Average Case: O(n log⁡ n)
 Worst Case: O(n^2) (when the pivot is the smallest or largest
element)

Practical Use of Asymthotic Notation:

Asymptotic notation allows programmers and computer scientists to:

• Analyze and compare the efficiency of different algorithms.

• Make predictions about performance without needing to implement the algorithms.
• Focus on the growth rates of functions, especially for large inputs, which helps in
understanding the scalability of algorithms.

Complexity:

the input increases. Understanding complexity helps us figure out how well an algorithm
performs.
Types of Complexity

1. Time Complexity: This tells us how long an algorithm takes to complete as the
input size grows. It's like asking, "How much time will it take to finish if I have more
items to process?"
a. Examples:
i. O(1): Constant time – takes the same time regardless of input size.
ii. O(n): Linear time – takes longer as the number of items increases (if
you have twice as many items, it takes about twice as long).
iii. O(n²): Quadratic time – time increases dramatically with more items
(if you double the items, the time increases four times).
2. Space Complexity: This tells us how much memory an algorithm needs as the
input size grows. It answers the question, "How much extra memory will I need for
more items?"
a. Examples:
i. O(1): Constant space – uses the same amount of memory no matter
how many items there are.
ii. O(n): Linear space – uses more memory as the number of items
increases.

Why It Matters

Understanding the complexity helps us choose the right algorithm for the job. An algorithm
that works well for small datasets might become too slow or use too much memory for
larger datasets. So, we look for algorithms that can handle larger inputs efficiently.

1. Brute Force

• What it is: Trying every possible option to find the solution.

• Example: Checking each number one by one to find the biggest one in a list.

2. Divide and Conquer

• What it is: Breaking a big problem into smaller parts, solving each part, and then
combining the results.
• Example: Sorting a list by first dividing it into smaller lists, sorting those, and then
merging them back together (like in merge sort).
3. Dynamic Programming

• What it is: Solving problems by breaking them into smaller pieces that repeat and
storing the results so you don’t have to solve the same piece more than once.
• Example: Finding the Fibonacci numbers by saving the results of previous numbers
instead of recalculating them.

4. Greedy Algorithms

• What it is: Making the best choice at each step without worrying about the overall
best solution. This works well for some problems.
• Example: Making change for a dollar by using the largest coins first (if you have
coins of 1, 5, 10, and 25 cents).