Design and Analysis of Algorithms Credits : 4

Course Objectives
1. Learn algorithms time complexity
2. Learn divide and conquer approach 3. Learn greedy method
4. Learn dynamic programming 5. Learn backtracking
Course Outcomes
1. Carry out algorithms time complexity
2. Explain divide and conquer approach 3. Illustrate greedy method
4. Elaborate dynamic programming 5. Explore backtracking

DAA stands for Design and Analysis of Algorithms — a foundational subject in computer
science and engineering that focuses on creating efficient algorithms and understanding how
they perform.

DAA is the study of designing algorithms (step-by-step computational procedures) and

analyzing their correctness, efficiency, and resource usage (time and space).

Purpose of Studying DAA

1. Design algorithms to solve complex computational problems.

2. Analyze them for performance in terms of:
o Time Complexity: How fast does the algorithm run?
o Space Complexity: How much memory does it use?
3. Compare multiple algorithms and choose the most efficient one for a given task.
4. Understand limits of computation, e.g., NP-hard problems (unsolvable in reasonable
time).

Key Areas Covered in DAA

1. Algorithm Design Techniques:

o Divide and Conquer
o Greedy Algorithms
o Dynamic Programming
o Backtracking and Branch-and-Bound
2. Algorithm Analysis:
o Asymptotic notation (Big O, Ω, Θ)
o Worst, average, and best-case performance
3. Data Structures used in algorithm implementation:
o Stacks, Queues, Trees, Graphs, Heaps
4. Complexity Classes:
o P, NP, NP-Complete, NP-Hard
 Why DAA is Important

• Helps in writing optimized and scalable code.

• Essential for competitive programming and technical interviews.
• Forms the backbone of many computer science applications like AI, Databases,
Operating Systems, and Networking.

+-------------------+
| DAA |
+-------------------+
|
+----------------------+------------------------+
| | |
Algorithm Design Algorithm Analysis Data Structures
Techniques | |
| | |
+-------+------+ +--------+--------+ +------+--------+
| Divide & | | Time Complexity | | Stacks |
| Conquer | | Space Complexity| | Queues |
| Greedy | | Big O, Θ, Ω | | Trees |
| Dynamic Prog | | Graphs |
| Backtracking | | Heaps |

|
+----------+----------+
| Complexity Classes |
+---------------------+
| P, NP, NP-Complete |
| NP-Hard |
|
+---------+---------+
| Applications |
+--------------------+
| AI, Databases, |
| Networking, OS |

What is P?
P (Polynomial Time) is the set of decision problems (yes/no problems) that can be solved
efficiently — that is, by an algorithm that runs in polynomial time.

Characteristics:

• Problems in P can be solved in time O(n^k) for some constant k, where n is the size
of the input.
• These are considered "tractable" or "easy" problems.
 Examples:

• Sorting numbers (e.g., Merge Sort)

• Searching in an array (e.g., Binary Search)
• Finding the shortest path in a graph (e.g., Dijkstra’s algorithm)

What is NP?
NP (Nondeterministic Polynomial time) is the class of decision problems for which a
solution can be verified in polynomial time, even if finding it may be hard.

Characteristics:

• Given a solution, we can verify it quickly (in polynomial time).

• It might take a long time to find that solution.
• NP includes all problems in P, but might also include harder ones.

Examples:

• Sudoku puzzle (verifying a solution is easy)

• Hamiltonian Cycle Problem (is there a cycle that visits all vertices?)
• 0/1 Knapsack Problem

Unit I

Introduction to Algorithms
1. What is an Algorithm?

• A finite set of well-defined instructions to solve a problem or perform a computation.

• Input: Zero or more values.
• Output: At least one value.
• Finiteness: Terminates after a finite number of steps.
• Definiteness: Each step is clearly and unambiguously defined.
• Effectiveness: Steps are basic enough to be carried out.

2. Algorithm Specification

• Formal representation of the algorithm.

• Includes:
o Input/output types.
 o Procedure description (in pseudocode or natural language).
o Constraints and assumptions.

Example (Pseudocode):

ALGORITHM LinearSearch(A, n, x)
Input: Array A of n elements, and target value x
Output: Index of x in A or -1 if not found
1. for i ← 1 to n do
2. if A[i] = x then
3. return i
4. return -1

3. Performance Analysis

Performance is measured in terms of time and space:

a. Time Complexity

• Indicates the number of operations executed.

• Uses Asymptotic Notations:
o Big O (O) – Worst-case scenario.
o Omega (Ω) – Best-case scenario.
o Theta (Θ) – Average-case or tight bound.
• Example: Linear Search has time complexity O(n).

b. Space Complexity

• Refers to the amount of memory used.

• Includes input storage, auxiliary space, and output storage.
• Constant auxiliary space → O(1), Linear → O(n), etc.

4. Randomized Algorithms

• Algorithms that make random choices during execution.

• Used to improve average performance or simplicity.
• Useful when deterministic solutions are too slow or complex.

Types:

1. Las Vegas Algorithms:

o Always produce the correct result.
o Running time is random.
o Example: Randomized QuickSort.
2. Monte Carlo Algorithms:
o May produce incorrect results with small probability.
o Running time is fixed or bounded.
 o Example: Randomized Primality Testing.

Example – Randomized QuickSort:

• Picks a pivot randomly, reduces chances of worst-case time O(n²).

• Expected time: O(n log n).

5. Importance of Algorithm Analysis

• Helps choose the most efficient algorithm for a problem.

• Critical for scalability and real-world applications.

Elementary Data Structures

Data structures are foundational tools that help organize and manipulate data efficiently in
algorithm design.

1. Stacks

• Definition: A linear data structure that follows LIFO (Last-In, First-Out) principle.
• Operations:
o push(x) – Insert element x
o pop() – Remove and return top element
o peek() – Return top element without removing
o isEmpty() – Check if the stack is empty
• Applications:
o Function calls (call stack)
o Expression evaluation and conversion
o Depth-First Search (DFS)

2. Queues

• Definition: A linear data structure that follows FIFO (First-In, First-Out) principle.
• Operations:
o enqueue(x) – Insert element x
o dequeue() – Remove and return the front element
o isEmpty() – Check if the queue is empty
• Variants:
o Circular Queue
o Deque (Double-Ended Queue)
• Applications:
o Scheduling (CPU, printer, etc.)
o Breadth-First Search (BFS)
3. Trees

• Definition: A hierarchical data structure with nodes connected by edges.

• Binary Tree: Each node has at most two children.
• Binary Search Tree (BST): For each node, left subtree < root < right subtree.
• Traversals:
o Inorder (LNR)
o Preorder (NLR)
o Postorder (LRN)
• Applications:
o Searching and sorting
o Syntax trees in compilers
o Priority queues (Heaps)

4. Dictionaries

• Definition: Data structure for key-value pair mappings.

• Implemented via:
o Hash tables: Fast average-case operations.
o Balanced binary search trees (e.g., AVL, Red-Black Tree)
• Operations:
o insert(key, value)
o delete(key)
o search(key)
• Applications:
o Symbol tables in compilers
o Databases, caches

5. Priority Queues

• Definition: Each element has a priority; the element with highest/lowest priority is
served first.
• Implemented using:
o Binary Heaps
o Fibonacci Heaps (for advanced applications)
• Operations:
o insert(x)
o extractMin() or extractMax()
• Applications:
o Dijkstra’s algorithm
o Huffman coding
 6. Sets and Disjoint Set Union (DSU)

• Set: Collection of unique elements.

• Disjoint Set Union (Union-Find): Used to manage partitioning of a set into disjoint
subsets.
• Operations:
o find(x) – Returns representative of the set containing x
o union(x, y) – Merges sets containing x and y
• Optimizations:
o Path compression
o Union by rank
• Applications:
o Kruskal’s algorithm (for MST)
o Detecting cycles in a graph

7. Graphs

• Definition: A set of nodes (vertices) connected by edges.

• Types:
o Directed / Undirected
o Weighted / Unweighted
• Representations:
o Adjacency Matrix
o Adjacency List
• Traversal Algorithms:
o BFS (Breadth-First Search)
o DFS (Depth-First Search)
• Applications:
o Social networks
o Routing and pathfinding
o Network topology

Unit II
Divide and Conquer

What is Divide and Conquer?

• A problem-solving paradigm that:

1. Divides the problem into smaller subproblems.
2. Conquers each subproblem by solving it recursively.
3. Combines the solutions of the subproblems to solve the original problem.
Steps in Divide and Conquer

1. Divide: Split the input into smaller parts.

2. Conquer: Solve each part recursively. If the part is small enough, solve it directly.
3. Combine: Merge or combine the solutions of the subproblems.

Advantages

• Breaks complex problems into simpler ones.

• Efficient for many problems with a recursive structure.
• Often leads to elegant and efficient algorithms.

Common Examples

1. Binary Search

• Problem: Search an element in a sorted array.

• Divide: Split the array into two halves.
• Conquer: Recursively search in one half where the element may be.
• Combine: Return the result from recursive call.
• Time Complexity: O(log n)

2. Finding Maximum and Minimum

• Divide the array into two halves.

• Recursively find max and min in each half.
• Combine: Compare and get overall max and min.
• Time Complexity: O(n)

3. Merge Sort

• Divide the array into two halves.

• Recursively sort both halves.
• Combine: Merge the two sorted halves into one sorted array.
• Time Complexity: O(n log n)
4. Quick Sort

• Divide by choosing a pivot element and partitioning the array into elements less than
and greater than pivot.
• Recursively sort the partitions.
• Combine: Subarrays are sorted in place.
• Average Time Complexity: O(n log n)
• Worst Case: O(n²) (can be improved by random pivot)

5. Selection Problem (Finding kth smallest element)

• Use partition-based approach similar to QuickSort (QuickSelect).

• Average Time Complexity: O(n)

6. Strassen’s Matrix Multiplication

• Multiplies two matrices faster than the standard O(n³).

• Divides matrices into submatrices and uses fewer recursive multiplications.
• Time Complexity: Approximately O(n^2.81)

7. Convex Hull (Divide and Conquer Approach)

• Divides points into two halves.

• Computes convex hulls recursively.
• Merges hulls to get overall convex hull.

Complexity Recurrence

• Many divide and conquer algorithms can be analyzed with recurrence relations:

T(n)=a⋅T(nb)+f(n)T(n) = a \cdot T\left(\frac{n}{b}\right) + f(n)T(n)=a⋅T(bn)+f(n)

Where:

oaaa = number of subproblems,

on/bn/bn/b = size of each subproblem,
of(n)f(n)f(n) = cost of dividing and combining.
• Use Master Theorem to solve and find time complexity.
The Greedy Method

What is the Greedy Method?

• An algorithmic paradigm that builds a solution step-by-step, making the best

(greedy) choice at each step.
• Assumes that locally optimal choices lead to a globally optimal solution.
• Unlike divide and conquer or dynamic programming, it does not revisit choices once
made.

Key Features

• Greedy choice is made based on current information.

• Simple and efficient.
• Works well when greedy-choice property and optimal substructure are satisfied.

When is Greedy Method Applicable?

1. Greedy-Choice Property:
o A global optimum can be arrived at by choosing a local optimum.
2. Optimal Substructure:
o Optimal solution to the problem contains optimal solutions to subproblems.

Common Problems Solved by Greedy Algorithms

1. Knapsack Problem (Fractional)

• Items have weights and values.

• Fractional quantities allowed.
• Greedy choice: pick items with highest value/weight ratio first.
• Optimal and efficient solution for fractional knapsack.

2. Tree Vertex Splitting

• Used in network reliability.

• Involves splitting vertices to optimize connectivity.
3. Job Sequencing with Deadlines

• Schedule jobs to maximize profit, each job with deadline and profit.
• Greedy approach: Sort jobs by profit, schedule as late as possible before deadline.
• Achieves optimal profit in O(n log n) time.

4. Minimum-Cost Spanning Trees

• Finds a spanning tree of a weighted graph with minimum total edge weight.
• Famous algorithms:
o Kruskal’s Algorithm (uses DSU)
o Prim’s Algorithm
• Both use greedy selection of edges with least cost.

5. Single Source Shortest Paths (for graphs with non-negative edges)

• Dijkstra’s Algorithm:
o Greedy approach picks the closest vertex not yet processed.
o Updates shortest paths incrementally.

Advantages of Greedy Algorithms

• Usually faster and simpler than other methods.

• Provides good approximate solutions even when optimal solution is hard.
• Easy to implement.

Limitations

• Doesn’t always produce the optimal solution (e.g., 0/1 Knapsack, some scheduling
problems).
• Requires proof that greedy choice leads to an optimal solution.
Summary Table

Problem Greedy Approach Optimal? Time Complexity

O(n log n)
Fractional Knapsack Pick by value/weight ratio Yes
(sorting)

Job Sequencing with

Sort by profit and schedule Yes O(n log n)
Deadlines

Minimum Spanning Tree Pick edges in increasing order Yes O(E log V)

Yes (non-negative
Single Source Shortest Path Pick nearest vertex each step O((V+E) log V)
weights)

Cannot be solved optimally by

0/1 Knapsack No N/A
greedy

Unit III
Dynamic Programming

What is Dynamic Programming?

• A method for solving complex problems by breaking them down into simpler
overlapping subproblems.
• Stores solutions of subproblems to avoid redundant computations (called
memoization).
• Useful when a problem exhibits:
o Optimal Substructure: Optimal solution can be constructed from optimal
solutions of subproblems.
o Overlapping Subproblems: Subproblems recur multiple times.

Key Idea

• Solve each subproblem once.

• Store its solution in a table (bottom-up approach) or cache (top-down recursion).
• Use stored results to build solutions to bigger problems.

Steps in Dynamic Programming

1. Characterize the structure of an optimal solution.

2. Define the value of an optimal solution recursively (recurrence relation).
3. Compute the value of optimal solutions bottom-up.
4. Construct the optimal solution from computed values.
Common Examples of Dynamic Programming

1. Multistage Graphs

• Find shortest path from source to destination through multiple stages.

• Recurrence:

d(j)=min i[d(i)+cost(i,j)]d(j) = \min_{i} [d(i) + cost(i,j)]d(j)=imin[d(i)+cost(i,j)]

• Build solutions from the last stage backward.

2. All-Pairs Shortest Paths (Floyd-Warshall Algorithm)

• Finds shortest paths between all pairs of vertices in a weighted graph.

• Uses a 2D DP table dist[i][j]dist[i][j]dist[i][j].
• Recurrence:

dist[i][j]=min (dist[i][j],dist[i][k]+dist[k][j])dist[i][j] = \min(dist[i][j], dist[i][k] +

dist[k][j])dist[i][j]=min(dist[i][j],dist[i][k]+dist[k][j])

• Time complexity: O(n3)O(n^3)O(n3).

3. Single-Source Shortest Paths (Bellman-Ford Algorithm)

• Finds shortest paths from a source vertex to all other vertices.

• Can handle negative weights.
• Recurrence:

dist[v]=min (u,v)∈E[dist[u]+w(u,v)]dist[v] = \min_{(u,v) \in E} [dist[u] +

w(u,v)]dist[v]=(u,v)∈Emin[dist[u]+w(u,v)]

• Time complexity: O(VE)O(VE)O(VE).

4. Optimal Binary Search Trees

• Given keys and their search probabilities, build a binary search tree with minimal
expected search cost.
• Uses recursive relation based on choosing root keys.
5. 0/1 Knapsack Problem

• Items with weights and values, choose items to maximize value without exceeding
capacity.
• Recurrence:

K(i,w)=max {K(i−1,w),vi+K(i−1,w−wi)if wi≤wK(i, w) = \max \begin{cases} K(i-1, w), \\ v_i +

K(i-1, w - w_i) \quad \text{if } w_i \leq w \end{cases}K(i,w)=max{K(i−1,w),vi+K(i−1,w−wi)if wi
≤w

• Time complexity: O(nW)O(nW)O(nW) where WWW is capacity.

6. Traveling Salesperson Problem (TSP)

• Find the shortest route visiting all cities exactly once and returning to start.
• DP approach uses bitmasking to represent visited cities.
• Time complexity: O(n22n)O(n^2 2^n)O(n22n).

Comparison with Other Techniques

Feature Divide & Conquer Dynamic Programming

Subproblems Independent Overlapping

Approach Top-down Bottom-up or Memoized Top-down

Recomputations Yes No (uses memoization)

Example QuickSort Fibonacci, Knapsack

Advantages

• Efficiently solves problems with overlapping subproblems.

• Reduces exponential time problems to polynomial time in many cases.
Basic Traversal and Search Techniques

These techniques are used to explore or search through data structures like trees and graphs,
which are core to many algorithmic problems.

1. Techniques for Binary Trees

Binary tree traversal means visiting all the nodes in a specific order. There are two main
categories:

a. Depth-First Traversals (DFT)

• Inorder (Left → Root → Right)

o Used in Binary Search Trees to get sorted order.
• Preorder (Root → Left → Right)
o Useful for copying trees, prefix expressions.
• Postorder (Left → Right → Root)
o Used for deleting or freeing trees, postfix expressions.

b. Breadth-First Traversal (Level Order)

• Visits nodes level by level (uses a queue).

• Example: Used in BFS for trees.

2. Techniques for Graphs

Graphs can be directed or undirected, cyclic or acyclic, and weighted or unweighted.

a. Depth-First Search (DFS)

• Explores as far as possible along a branch before backtracking.

• Implemented using recursion or a stack.
• Applications:
o Detecting cycles
o Topological sorting
o Connected components

b. Breadth-First Search (BFS)

• Explores all neighbors at the current depth before going deeper.

• Implemented using a queue.
• Applications:
o Shortest path in unweighted graphs
o Level-order traversal
 o Graph coloring

3. Connected Components and Spanning Trees

a. Connected Components

• In undirected graphs, a connected component is a subgraph in which any two

vertices are connected by a path.
• DFS or BFS can be used to find all components.

b. Spanning Trees

• A spanning tree is a subgraph that connects all vertices with the minimum number of
edges (no cycles).
• Minimum Spanning Trees (MST) are built using:
o Kruskal’s Algorithm (Greedy, edge-based)
o Prim’s Algorithm (Greedy, vertex-based)

4. Biconnected Components and DFS

a. Biconnected Graph

• A graph is biconnected if there are no articulation points (vertices whose removal

disconnects the graph).
• Biconnected components can be found using Tarjan’s Algorithm (based on DFS).

b. Articulation Points

• Found using DFS timestamps and low-link values.

• Used in network reliability analysis.
 Summary Table

Technique Structure Uses Stack/Queue Applications

DFS Graph/Tree Stack (or Recursion) Topological sort, Cycle detection

BFS Graph/Tree Queue Shortest path in unweighted graph

Inorder Binary Tree Stack/Recursion BST traversal (sorted order)

Preorder Binary Tree Stack/Recursion Tree construction, expression trees

Postorder Binary Tree Stack/Recursion Tree deletion, postfix expressions

Level Order Binary Tree Queue Level-wise traversal

Unit IV
Back Tracking

What is Backtracking?

Backtracking is a general algorithmic technique for solving combinatorial problems by

exploring all possible solutions and abandoning partial solutions ("backtracking") that do
not satisfy constraints.

It is a refined brute-force approach that incrementally builds candidates and prunes paths
that are guaranteed to fail.

Key Concepts

• Constructs solution step-by-step.

• If a partial solution fails, the algorithm backtracks to a previous step and tries a
different option.
• Often implemented using recursion and implicit state-space trees.

General Method (Template)

procedure BACKTRACKING(c):
if c is a solution:
print(c)
else:
for each choice x in possible extensions of c:
if x is promising:
BACKTRACKING(c + x)
 Classic Problems Solved Using Backtracking

♟ 1. 8-Queens Problem

• Place 8 queens on an 8×8 chessboard such that no two queens threaten each other.
• Use backtracking to place one queen per row, checking for column and diagonal
conflicts.
• State: Positions of queens placed so far.
• Constraint: No two queens share the same row, column, or diagonal.

2. Sum of Subsets

• Given a set of numbers and a target sum, find subsets that sum to the target.
• State: Current subset and remaining sum.
• Constraint: Sum should not exceed the target.

3. Graph Coloring

• Assign colors to vertices such that no adjacent vertices have the same color using ≤ k
colors.
• State: Colors assigned to vertices so far.
• Constraint: Adjacent vertices must not share the same color.
• Used in scheduling, register allocation, etc.

4. Hamiltonian Cycle

• Check if a graph contains a cycle that visits every vertex exactly once and returns to
the starting point.
• State: Current path.
• Constraint: Path must include each vertex once and return to start.

5. 0/1 Knapsack Problem (Backtracking Approach)

• Items can be either included or excluded.

• Use backtracking to explore all combinations.
• Constraint: Total weight ≤ capacity.
• Optimization: Use bounding functions to prune paths.
 Time Complexity
• Exponential in most cases (e.g., O(2ⁿ), O(n!)), but practical for small inputs.
• Can be made more efficient using:
o Pruning
o Bounding functions (for optimization problems)
o Heuristics

Backtracking vs Other Techniques

Feature Backtracking Dynamic Programming Greedy
Problem Type Constraint satisfaction Optimization, Overlap Optimization
Approach Search & prune Subproblem reuse Local optimal choices
Optimality Guaranteed (if complete) Guaranteed Not always
Complexity Often exponential Usually polynomial Usually fast (greedy)

Branch-Bound:

What is Branch and Bound?

Branch and Bound (B&B) is an algorithm design paradigm for solving combinatorial and
optimization problems, especially NP-hard problems. It systematically explores the
solution space by:

• Branching: Splitting the problem into smaller subproblems.

• Bounding: Using bounds to eliminate suboptimal solutions early.

It's like backtracking, but with an added component: pruning based on bounds to avoid
unnecessary computation.

Key Concepts

1. State Space Tree: A tree representing all possible solutions.

2. Branching: Expands nodes by creating subproblems (children).
3. Bounding Function: Computes the best possible solution (bound) from a node to
decide whether to proceed.
4. Pruning: Stops exploring a node if its bound is worse than the current best known
solution.
 Problems Solved by Branch and Bound

1. 0/1 Knapsack Problem

• Given items with weights and profits, select items to maximize profit without
exceeding the capacity.
• Branch: Include or exclude each item.
• Bound: Use Fractional Knapsack (Greedy method) to estimate the upper bound of
profit from a node.
• Prune: Skip nodes where:
o Total weight exceeds capacity.
o Bound ≤ current max profit.

Example Steps:

• Node = partial solution (e.g., items 1, 2 included; item 3 not yet decided).
• Calculate upper bound at each node.
• Use a priority queue to explore most promising node next (Best-First Search).

2. Traveling Salesperson Problem (TSP)

• Visit every city once and return to the start with minimum total cost.
• Branch: Fix the next city in the tour.
• Bound: Estimate the least possible cost using reduced cost matrix.
• Prune: Discard paths with cost more than current best.

Example Strategy:

• Represent problem using cost matrix.

• Reduce rows and columns to get initial bound.
• Use a min-heap or priority queue to pick the most promising node.

General Algorithm (Best-First Search Strategy)

1. Start with a root node representing an empty or partial solution.

2. Compute bound for the node.
3. Add node to priority queue.
4. While queue is not empty:
o Remove node with best (lowest) bound.
o If node is a complete solution and better than current best, update it.
o Else, generate child nodes and add those with bounds better than current best.
 Comparison: Backtracking vs Branch and Bound

Feature Backtracking Branch and Bound

Purpose Feasibility problems Optimization problems

Pruning Constraint-based Bound-based

Efficiency May explore more nodes Often fewer nodes (uses bounds)

Optimal Solution Not guaranteed unless all paths explored Guaranteed (if used fully)

Advantages

• Often more efficient than brute-force and backtracking.

• Guarantees an optimal solution (when complete).

Disadvantages

• May still be exponential in the worst case.

• Needs good bounding functions to be effective.

Unit V
NP-Hard and NP-Complete Problems
These are classes of computational problems that are generally difficult (intractable) to
solve efficiently. They lie at the heart of computational complexity theory.

Basic Complexity Classes

Class Definition

P Problems that can be solved in polynomial time.

NP Problems for which solutions can be verified in polynomial time.

NP-
The hardest problems in NP. If one can be solved in polynomial time, all can.
Complete

Problems that are at least as hard as NP-Complete problems, but not necessarily in NP
NP-Hard
(verification may not be polynomial).
Definitions

NP (Nondeterministic Polynomial time)

• Decision problems for which a proposed solution can be verified in polynomial

time.

NP-Complete

• A problem A is NP-Complete if:

1. A ∈ NP
2. Every problem in NP can be reduced to A in polynomial time (called NP-
Hardness).
• These are the most difficult problems within NP.

NP-Hard

• A problem B is NP-Hard if:

o Every NP problem can be reduced to B in polynomial time.
o B does not need to be in NP, i.e., it might not even be a decision problem
(could be an optimization problem).

Cook's Theorem
• Stephen Cook (1971) proved that the Boolean Satisfiability Problem (SAT) is NP-
Complete.
• First known NP-Complete problem.
• Foundation for proving other problems NP-Complete using polynomial-time
reduction.

Reducibility
• A problem A can be reduced to problem B if any instance of A can be transformed
into an instance of B in polynomial time.
• If B can be solved efficiently, so can A.
 Examples of NP-Complete Problems
Problem Description

SAT (Boolean
Is there a truth assignment that satisfies a Boolean formula?
Satisfiability)

3-SAT SAT where each clause has exactly 3 literals.

Hamiltonian Cycle Is there a cycle that visits each vertex once?

Can a graph be colored with k colors without adjacent nodes sharing

Graph Coloring
colors?

Clique Problem Is there a complete subgraph of size k?

Subset Sum Is there a subset with sum = target value?

0/1 Knapsack Can items be chosen to exactly fill the knapsack with max value?

NP-Hard Problems (Not necessarily in NP)

Problem Description

Travelling Salesperson (TSP) Find the shortest tour visiting each city once (Optimization).

Halting Problem Will a program halt or run forever? (Undecidable)

General Scheduling Assign jobs to machines to minimize time/cost.

Code Generation (Compiler) Generate optimal machine code – extremely complex.

How to Show a Problem is NP-Complete

1. Prove it belongs to NP (solution can be verified in poly time).

2. Choose a known NP-Complete problem.
3. Reduce it to the new problem in polynomial time.
 Why NP-Hard and NP-Complete Matter:

• No known polynomial-time algorithms for NP-Complete problems.

• If one NP-Complete problem is solved in P, P = NP, solving thousands of hard
problems.
• Used in:
o Cryptography (based on intractability)
o Operations research
o Scheduling, optimization

Here is a simple diagram showing the relationship between P, NP, NP-Complete, and NP-
Hard:

+-------------------------------+
| NP-Hard |
| +---------------------+ |
| | NP-Complete | |
| | +------------+ | |
| | | NP | | |
| | | +--------+ | | |
| | | | P | | | |
| | | +--------+ | | |
| | +------------+ | |
| +---------------------+ |
+-------------------------------+

Legend:

• P: Problems solvable in polynomial time.

• NP: Problems whose solutions are verifiable in polynomial time.
• NP-Complete:
o Subset of NP.
o Hardest problems in NP.
o If one is solved in P → all NP problems are in P.
• NP-Hard:
o At least as hard as NP-Complete.
o May not be in NP (e.g., optimization problems or undecidable problems).
o Includes NP-Complete and problems even harder.