Introduction to Algorithm

What is an Algorithm?

 A step-by-step procedure for solving a problem or accomplishing a

task
 Algorithms are generally created independent of underlying
languages,
 i.e. an algorithm can be implemented in more than one
programming language.
 Key Elements:
 Inputs
 Outputs
 Steps
Types of Algorithms

• Brute Force: Tries all possible solutions.

• Divide and Conquer: Divides problems into smaller ones.
• Greedy: Chooses the most beneficial option at each step.
• Dynamic Programming: Solves subproblems and stores
their solutions.
• Backtracking: Removes solutions that don’t fit constraints.
• Recursive: Calls itself with smaller inputs.
From data structure point of view, following are
some important categories of algorithms:

 Search Algorithms: to search an item in a data structure

 Sort Algorithms: to sort items in certain order
 Insert Algorithms: to insert item in a data structure
 Update Algorithms: to update an existing item in a data
structure
 Delete Algorithms: to delete an existing item from a data
structure
Importance of Algorithms

 Efficiency: Optimizes time and space

 Scalability: Handles large inputs and complex
tasks
 Reliability: Ensures consistency and accuracy
Applications of Algorithms

 Sorting and Searching: Quick Sort, Merge Sort,

Binary Search
 Graph Algorithms: Shortest path, Minimum spanning
tree
 Cryptography: Secure communication
 Machine Learning: Data-driven predictions
 How to write an algorithm?

 There are no well-defined standards for writing algorithms.

 Rather, it is problem and resource dependent.
 Algorithms are never written to support a particular programming code.
 As we know, all programming languages share basic code constructs like
loops (do, for, while), flow-control (if-else) etc. these common constructs
can be used to write an algorithm.
 We write algorithms in step by step manner, but it is not always the case.
 Algorithm writing is a process and is executed after the problem domain
is well-defined.
 That is, we should know the problem domain, for which we are designing
a solution.
 Example

 Problem: design an algorithm to add two numbers

and display result.
 Step 1: START
 Step 2: declare three integers a, b & c
 Step 3: define values of a & b
 Step 4: add values of a & b
 Step 5: store output of step 4 to c
 Step 6: print c
 Step 7: STOP
 Algorithms tell the programmers how to code
the program. Alternatively the algorithms can be
written as:

 Step 1: START ADD

 Step 2: get values of a & b
 Step 3: c <- a + b
 Step 4: display c
 Step 5: STOP
 In design and analysis of algorithms, usually the
second method is used to describe an algorithm.
 It makes it easy of the analyst to analyze the
algorithm ignoring all unwanted definitions.
 He can observe what operations are being used an
how the process is flowing.
 Writing step numbers, is optional.
 We design an algorithm to get solution of a given
problem.
 A problem can be solved in more than one ways.
 Hence, many solution algorithms can be derived for a
given problem
 Next step is to analyse those proposed solution
algorithms and implement the best suitable
Properties of Algorithms:

 The fundamental characteristics that every algorithm

should have to function correctly.
 Not all procedures can be called an algorithm. An
algorithm should have the below mentioned
characteristics:
1. Input

 An algorithm may have zero or more inputs. These

inputs are the external data that the algorithm
processes to produce the desired output.
 Example: In a sorting algorithm, the input could be a
list of numbers that need to be arranged in
ascending order.
2. Output

 An algorithm produces at least one output, which is

the result of the computation or process. The output
is directly related to the input and reflects the goal of
the algorithm.
 Example: In the sorting algorithm mentioned above,
the output is the sorted list of numbers.
3. Unambiguous

 Each step in an algorithm must be clearly and

unambiguously defined. The actions that the
algorithm takes should be specified in such a way
that they can be performed without confusion.
 Example: If an algorithm instructs to "subtract 5
from the total," it must be clear what the "total" is at
that point in the algorithm.
4. Finiteness

 An algorithm must always terminate after a finite

number of steps. It cannot run indefinitely; it must
reach a conclusion after a certain point.
 Example: A loop that continues until a certain
condition is met is finite if that condition will definitely
be met after a certain number of iterations.
5. Effectiveness

 Each step of an algorithm must be simple enough to

be performed, in theory, by a person using basic
operations. The operations should be doable within
a reasonable time frame and without requiring any
superhuman abilities.
 Example: Basic arithmetic operations like addition,
subtraction, and comparison are considered
effective because they can be performed manually.
6. Generality

 An algorithm should be general enough to solve a

class of problems, not just a single instance. It
should be applicable to various inputs and scenarios
within its defined problem domain.
 Example: A sorting algorithm should be able to sort
lists of different sizes and types of elements, not
just one specific list.
7. Correctness

 An algorithm is correct if, for every input instance, it

halts with the correct output. This means that it
should always produce the desired output for any
valid input.
 Example: If an algorithm is designed to compute the
factorial of a number, it should return the correct
factorial for any non-negative integer input.
 Summary of properties

 Input: Data the algorithm processes

 Output: Result produced by the algorithm
 Unambiguous: Clarity and unambiguity of each step
 Finiteness: The algorithm must terminate
 Effectiveness: Simple enough operations that can be
manually executed
 Generality: Applicability to a broad class of problems
 Correctness: Produces the right output for all valid inputs
 These properties ensure that an algorithm is not only capable
of solving a problem but also does so efficiently and reliably.
Features of Algorithms:

 When discussing the "features" of algorithms, it

typically refers to the characteristics or aspects that
define how algorithms are designed, how they
perform, and how they are applied.
 The qualities that can describe the efficiency, clarity,
and practicality of an algorithm.
 Here’s a detailed explanation:
1. Determinism

 An algorithm is deterministic if, given a particular

input, it always produces the same output and
follows the same sequence of steps.
 Example: The Quick Sort algorithm, given the same
list of numbers, will always produce the same sorted
list.
2. Non-determinism

 A non-deterministic algorithm may produce different

outputs for the same input on different executions.
This can happen due to elements like randomness or
parallel processing.
 Example: Randomized algorithms, such as
Randomized Quick Sort
3. Parallelism

 Some algorithms are designed to perform multiple

operations simultaneously by dividing the task into
sub-tasks that can be executed in parallel. This is
often used to speed up processing.
 Parallel sorting algorithms can sort different parts of a
dataset simultaneously using multiple processors.
4. Scalability

 An algorithm's ability to handle growing amounts of

work or its capacity to be efficiently scaled up when
additional resources (like processors or memory) are
added.
 Example: A scalable algorithm can process a large
dataset without a significant drop in performance,
such as the Merge Sort algorithm which scales well
with large inputs.
5. Recursiveness

 A recursive algorithm is one that calls itself with a

subset of the original problem, gradually reducing the
problem until it is simple enough to solve directly.
 Example: The classic example is the algorithm for
calculating the factorial of a number: ‘n! = n * (n-1)!’ ,
with the base case being ‘0! = 1’
6. Iterativeness

 An iterative algorithm solves a problem by

repeatedly applying a set of operations, typically in a
loop, until a certain condition is met.
 Example: The Linear Search algorithm iteratively
checks each element of a list until it finds the target
value or reaches the end of the list.
7. Greediness

 Greedy algorithms make a series of choices, each of

which looks best at the moment (locally optimal),
with the hope that these choices lead to a globally
optimal solution.
 Example: The Greedy algorithm for the coin change
problem.
8. Dynamic Programming

 Dynamic programming algorithms solve problems by

breaking them down into simpler subproblems and
storing the results of these subproblems to avoid
redundant calculations.
 Example: The algorithm for finding the shortest path
in a weighted graph using the Bellman-Ford algorithm
employs dynamic programming principles.
9. Divide and Conquer

 This approach involves dividing the problem into

smaller subproblems of the same type, solving each
subproblem independently, and then combining
their solutions to solve the original problem.
 Example: The Merge Sort algorithm is a classic
example, where the list is divided into halves, each
half is sorted, and then the halves are merged.
10. Probabilistic

 Probabilistic algorithms use randomness as part of

their logic, which means their behaviour can vary
even with the same input. These algorithms are
particularly useful when deterministic solutions are
impractical.
 Example: The Monte Carlo method is a probabilistic
algorithm used for numerical simulations, relying on
repeated random sampling to obtain results.
11. Approximation

 Approximation algorithms are designed to find near-

optimal solutions to complex problems where finding
the exact optimal solution is computationally
infeasible.
 Example: The Traveling Salesman Problem (TSP)
often uses approximation algorithms to find a route
that is close to the shortest possible, rather than the
exact shortest route.
12. Heuristic

 Heuristic algorithms use practical methods to find

good-enough solutions for complex problems, often
with less computational effort than exact methods.
These methods don’t guarantee an optimal solution
but are effective in practice.
 Example: Algorithms used in Artificial Intelligence
(AI), such as those for pathfinding in games, often
use heuristics to make decisions.
Summary of features
 Determinism/Non-determinism: Whether the algorithm consistently
produces the same result for the same input
 Parallelism: Ability to perform operations simultaneously
 Scalability: Ability to handle increased workload or resources
 Recursiveness/Iterativeness: Approach to solving the problem by self-calling
or repetition
 Greediness: Making locally optimal choices
 Dynamic Programming: Solving problems by using the solutions of
subproblems
 Divide and Conquer: Breaking the problem into subproblems and combining
their solutions
 Probabilistic: Involving randomness in decision-making
 Approximation: Finding near-optimal solutions where exact solutions are too
costly
 Heuristic: Using practical methods to find good-enough solutions
 These features help define the nature, performance, and applicability of
Difference between properties and
features of algorithms:
 The terms "properties" and "features" of algorithms are
sometimes used interchangeably, but they refer to slightly
different concepts
 Properties of algorithms are fundamental attributes that
describe the essential characteristics that all algorithms must
possess.
 Features of algorithms refer to specific aspects, methods, or
techniques used in designing and implementing algorithms.
 In summary, properties are the essential characteristics that
every algorithm must have, while features are additional
aspects that define how an algorithm works or solves a particular
problem.
Conclusion

 Recap:
 What is an Algorithm
 Types of Algorithms
 Importance of Algorithms
 Applications of Algorithms
 How to write an Algorithm
 Properties of Algorithms
 Features of Algorithms
Questions?