MODULE-2

Search Algorithms in Artificial Intelligence

Search algorithms are one of the most important areas of Artificial Intelligence.

Search Algorithm Terminologies:

o Search: Searching is a step by step procedure to solve a search-problem in a given search space. A
search problem can have three main factors:
a. Search Space: Search space represents a set of possible solutions, which a system may have.
b. Start State: It is a state from where agent begins the search.
c. Goal test: It is a function which observe the current state and returns whether the goal state is
achieved or not.
Search tree: A tree representation of search problem is called Search tree. The root of the search tree is the
root node which is corresponding to the initial state.
Actions: It gives the description of all the available actions to the agent.
Transition model: A description of what each action do, can be represented as a transition model.
Path Cost: It is a function which assigns a numeric cost to each path.
Solution: It is an action sequence which leads from the start node to the goal node.
Optimal Solution: If a solution has the lowest cost among all solutions.

Properties of Search Algorithms:

Following are the four essential properties of search algorithms to compare the efficiency of these
algorithms:

Completeness: A search algorithm is said to be complete if it guarantees to return a solution if at

least any solution exists for any random input.

Optimality: If a solution found for an algorithm is guaranteed to be the best solution (lowest path
cost) among all other solutions, then such a solution for is said to be an optimal solution.

Time Complexity: Time complexity is a measure of time for an algorithm to complete its task.

Space Complexity: It is the maximum storage space required at any point during the search, as the
complexity of the problem.
Types of search algorithms

Based on the search problems we can classify the search algorithms into uninformed (Blind search)
search and informed search (Heuristic search) algorithms.

Uninformed/Blind Search:

The uninformed search does not contain any domain knowledge such as closeness, the location of the
goal. It operates in a brute-force way as it only includes information about how to traverse the tree
and how to identify leaf and goal nodes. Uninformed search applies a way in which search tree is
searched without any information about the search space like initial state operators and test for the
goal, so it is also called blind search. It examines each node of the tree until it achieves the goal node.

It can be divided into five main types:

o Breadth-first search
o Uniform cost search
o Depth-first search
o Iterative deepening depth-first search
o Bidirectional Search
Informed Search

Informed search algorithms use domain knowledge. In an informed search, problem information is
available which can guide the search. Informed search strategies can find a solution more efficiently
than an uninformed search strategy. Informed search is also called a Heuristic search.

A heuristic is a way which might not always be guaranteed for best solutions but guaranteed to find a
good solution in reasonable time.

Informed search can solve much complex problem which could not be solved in another way.

An example of informed search algorithms is a traveling salesman problem.

1. Greedy Search
2. A* Search

Random Search:

Random search is a technique where random combinations of the hyper parameters are used to find
the best solution for the built model. It is similar to grid search, and yet it has proven to yield better
results comparatively. The drawback of random search is that it yields high variance during computing.
Since the selection of parameters is completely random; and since no intelligence is used to sample
these combinations, luck plays its part. Below is a visual description of search pattern of the random
search.
As random values are selected at each instance, it is highly likely that the whole of action space has been
reached because of the randomness, which takes a huge amount of time to cover every aspect of the combination
during grid search. This works best under the assumption that not all hyper parameters are equally important. In
this search pattern, random combinations of parameters are considered in every iteration. The chances of finding
the optimal parameter are comparatively higher in random search because of the random search pattern where
the model might end up being trained on the optimised parameters without any aliasing.

Search with Open and Closed List

• The nodes that the algorithm has generated are kept in a data structure called OPEN or fringe. Initially only
the start node is in OPEN.
• The search starts with the root node. The algorithm picks a node from OPEN for expanding and generates all
the children of the node. Expanding a node from OPEN results in a closed node. Some search algorithms keep
track of the closed nodes in a data structure called CLOSED.

Pure heuristic search is the simplest form of heuristic search algorithms. It expands nodes based on their
heuristic value h(n). It maintains two lists, OPEN and CLOSED list. In the CLOSED list, it places those nodes
which have already expanded and in the OPEN list, it places nodes which have yet not been expanded.
On each iteration, each node n with the lowest heuristic value is expanded and generates all its successors and n
is placed to the closed list. The algorithm continues unit a goal state is found.
Example of 8 Puzzle Problem using :-
Depth – First – Search traversal and Breadth - First - Search traversal :
Uninformed Search Methods:
Breadth- First -Search: Breadth First Search

Algorithm
Breadth first search

Let fringe be a list containing the

initial state Loop

if fringe is empty
return failure Node
remove-first
(fringe)ifNode is agoal

then return the path from initial state

to Node else generate all successors of
Node, and
Note that in breadth first search the newly generated nodes are put at the back of fringe or the
OPEN list. What this implies is that the nodes will be expanded in a FIFO (First In First Out)
order. The node that enters OPEN earlier will be expanded earlier. This amounts to expanding
the shallowest nodes first.

BFS Illustrated
We will now consider the search space in Figure1, and show how breadth first search
works on this graph.

Step 1: Initially fringe contains only one node corresponding to the source state A.

B C

D E D G

C F B F

G G H E G H

FRINGE: A

Step2: A is removed from fringe. The node is expanded, and its children B and C are
generated. They are placed at the back of fringe.

B C

D E D G

C F B F

G G H E G H
FRINGE: BC

Step3: Node B is removed from fringe and is expanded. Its children D, E are generated
And put at the back of fringe.
 A

B C

D E D G

C F B F

G G H E G H

FRINGE: C DE

Step4: Node C is removed from fringe and is expanded. Its children D and G
are added to the back of fringe.

B C

D E D G

C F B F

G G H E G H

FRINGE: D EDG

Step5: Node D is removed from fringe. Its children C and F are generated and added
to the back of fringe.
 A

B C

D E D G

C F B F

G G H E H

FRINGE:E DGC F

Step6: Node E is removed from fringe. It has no children.

A

B C

D E D G

C F B F

G G H E G H

FRINGE:DG CF

Step 7: D is expanded, B and F are put in OPEN.

B C

D E D G

C F B F

G G H E G H
FRINGE:GCFBF
 A

B C

D E D G

C F B F
Goal!

G G H E G H

Step 8: G is selected for expansion. It is found to be a goal node. So the

algorithm returns the path ACG by following the parent pointers of the node
corresponding to G. The algorithm terminates.

Properties of Breadth-First Search

We will now explore certain properties of breadth first search. Let us consider a model of the
search tree as shown in Figure 3. We assume that every non-leaf node has b children. Suppose
that d is the depth o the shallowest goal node, and m is the depth of the node found first

Breadth first search is:

• Complete.
• The algorithm is optimal (i.e., admissible) if all operators have the same cost. Otherwise,
breadth first search finds a solution with the shortest path length.
• The algorithm has exponential time and space A B C D E D G C F B F G G H E G H FAI
Module - 2 19 complexity. Suppose the search tree can be modeled as a b-ary tree as shown in
Figure 3. Then the time and space complexity of the algorithm is O(bd) where d is the depth of
the solution and b is the branching factor (i.e., number of children) at each node.
Advantages of Breadth First Search
• Finds the path of minimal length to the goal.
Disadvantages of Breadth First Search
• Requires the generation and storage of a tree whose size is exponential

Depth First Search

Algorithm

Depth First Search

Let fringe be a list containing the
initial state Loop
if fringe is empty return
failure Node remove-first(fringe)
if Node is a goal
then return the path from initial state
to Node else generate all successors
of Node, and
merge the newly generated nodes into fringe
add generated nodes to the front of fringe
End Loop

The depth first search algorithm puts newly generated nodes in the front of
OPEN. This results in expanding the deepest node first. Thus the nodes in
OPEN follow a LIFO order (Last In First Out). OPEN is thus implemented
using a stack data structure.

DFS Illustrated

State Space Graph

B E

A D F H
C
G
 A

B C

D E D G

C F B F

G G H E G H

Search tree for the state space graph

Let us now run Depth First Search on the search space given ,and trace its
progress.

Step1: Initially fringe contains only the node for A.

A

B C

D E D G

C F B F

G G H E G H

FRINGE:A

Step2: A is removed from fringe. A is expanded and its children B and C

are put in front of fringe.

B C

D E D G

C F B F

G G H E G H

FRINGE:B C
Step3: Node B is removed from fringe, and its children D and E are pushed in
front of fringe.
A

B C

D E D G

C F B F

G G H E G H

FRINGE:DE C

Step4: Node D is removed from fringe. C and F are pushed in front of fringe.

B C

D E D G

C F B F

G G H E G H

Fringe:CFEC

Step5: Node C is removed from fringe. Its child G is pushed in front of

fringe.

B C

D E D G

C F B F

G G H E G H

FRINGE:G FE C
 Step6: Node G is expanded and found to be a goal node. The solution path A-
B-D-C-G is returned and the algorithm terminates.

B C

D E D G

C F B F

G G H E G H

Goal!

FRINGE:GFEC

Properties of Depth First Search :

Let us now examine some properties of the DFS algorithm.
• The algorithm takes exponential time.
• If N is the maximum depth of a node in the search space, in the worst case the algorithm will take time O(bd ).
• However the space taken is linear in the depth of the search tree, O(bN).
• Note that the time taken by the algorithm is related to the maximum depth of the search tree. If the search tree
has infinite depth, the algorithm may not terminate. This can happen if the search space is infinite. It can also
happen if the search space contains cycles. The latter case can be handled by checking for cycles in the
algorithm. Thus Depth First Search is not complete.

HeuristicSearch
Heuristics function: Heuristic is a function which is used in Informed Search, and it finds the most
promising path. It takes the current state of the agent as its input and produces the estimation of how close
agent is from the goal. The heuristic method, however, might not always give the best solution, but it
guaranteed to find a good solution in reasonable time. Heuristic function estimates how close a state is to
the goal. It is represented by h(n), and it calculates the cost of an optimal path between the pair of states.
The value of the heuristic function is always positive.
Admissibility of the heuristic function is given as:
h(n)<= h*(n)
Here h(n) is heuristic cost, and h*(n) is the estimated cost. Hence heuristic cost should be less than
or equal to the estimated cost.
Pure Heuristic Search:

Pure heuristic search is the simplest form of heuristic search algorithms. It expands nodes based on their
heuristic value h(n). It maintains two lists, OPEN and CLOSED list. In the CLOSED list, it places those
nodes which have already expanded and in the OPEN list, it places nodes which have yet not been
expanded.
On each iteration, each node n with the lowest heuristic value is expanded and generates all its successors
and n is placed to the closed list. The algorithm continues until a goal state is found.
In the informed search we will discuss two main algorithms which are given below:
• Best First Search Algorithm
• A* Search Algorithm

Example of Heuristic Function

A heuristic function at a node n is an estimate of the optimum cost from the current
node to a goal. It is denoted by h(n).
h(n)=estimated cost of the cheapest path from node n to a goal node
Example: 8-puzzle: Misplaced Tiles Heuristics is the number of tiles out of place.

2 8 3 1 2 3

1 6 4 8 4

7 5 7 6 5

Initial State Goal state

Figure: 8puzzle
The first picture shows the current state n, and the second picture the goal
state.
h(n) =5
because the tiles 2,8,1,6 and 7 are out of place.

o Manhattan Distance Heuristic: Another heuristic for 8-puzzle is the Manhattan

distance heuristic. This heuristic sums the distance that the tiles are out of
place. The distance of a tile is measured by the sum of the differences in the
 x-positions and they-positions.

For the above example, using the Manhattan distance heuristic,

h(n) =1+1+0+0+0 + 1+1+2= 6

Best First Search (Informed Search):

In BFS and DFS, when we are at a node, we can consider any of the adjacent as next node. So
both BFS and DFS blindly explore paths without considering any cost function. The idea of Best
First Search is to use an evaluation function to decide which adjacent is most promising and then
explore. Best First Search falls under the category of Heuristic Search or Informed Search.
We use a priority queue or heap to store costs of nodes which have lowest evaluation function
value. So the implementation is a variation of BFS, we just need to change Queue to Priority
Queue.

//
Let us consider the below example.
A* Algorithm Concepts and Implementation
What is an A* Algorithm?

It is a searching algorithm that is used to find the shortest path between an initial and a
final point.

It is a handy algorithm that is often used for map traversal to find the shortest path to be
taken. A* was initially designed as a graph traversal problem, to help build a robot that
can find its own course. It still remains a widely popular algorithm for graph traversal.

It searches for shorter paths first, thus making it an optimal and complete algorithm. An
optimal algorithm will find the least cost outcome for a problem, while a complete
algorithm finds all the possible outcomes of a problem.

Another aspect that makes A* so powerful is the use of weighted graphs in its
implementation. A weighted graph uses numbers to represent the cost of taking each
path or course of action. This means that the algorithms can take the path with the least
cost, and find the best route in terms of distance and time.

Figure 1: Weighted Graph

A major drawback of the algorithm is its space and time complexity. It takes a large amount of
space to store all possible paths and a lot of time to find them.

The Basic Concept of A* Algorithm

A heuristic algorithm sacrifices optimality, with precision and accuracy for speed, to solve
problems faster and more efficiently.

All graphs have different nodes or points which the algorithm has to take, to reach the final
node. The paths between these nodes all have a numerical value, which is considered as the
weight of the path. The total of all paths transverse gives you the cost of that route.

Initially, the Algorithm calculates the cost to all its immediate neighbouring nodes, n, and
chooses the one incurring the least cost. This process repeats until no new nodes can be chosen
and all paths have been traversed. Then, you should consider the best path among them. If f(n)
represents the final cost, then it can be denoted as :

f(n) = g(n) + h(n), where :

g(n) = cost of traversing from one node to another. This will vary from node to node

h(n) = heuristic approximation of the node's value. This is not a real value but an approximation
cost

How Does the A* Algorithm Work?

Figure 2: Weighted Graph 2

Consider the weighted graph depicted above, which contains nodes and the distance between
them. Let's say you start from A and have to go to D.

Now, since the start is at the source A, which will have some initial heuristic value. Hence, the
results are
f(A) = g(A) + h(A)
f(A) = 0 + 6 = 6

Next, take the path to other neighbouring vertices :

f(A-B) = 1 + 4

f(A-C) = 5 + 2

Now take the path to the destination from these nodes, and calculate the weights :

f(A-B-D) = (1+ 7) + 0

f(A-C-D) = (5 + 10) + 0

It is clear that node B gives you the best path, so that is the node you need to take to reach the
destination.

Pseudo code of A* Algorithm

The text below represents the pseudo code of the Algorithm. It can be used to implement the
algorithm in any programming language and is the basic logic behind the Algorithm.

• Make an open list containing starting node

• If it reaches the destination node :

• Make a closed empty list

• If it does not reach the destination node, then consider a node with the lowest f-score in the
open list

We are finished

• Else :

Put the current node in the list and check its neighbours

• For each neighbour of the current node :

• If the neighbour has a lower g value than the current node and is in the closed list:

Replace neighbour with this new node as the neighbour’s parent

• Else If (current g is lower and neighbour is in the open list):

Replace neighbour with the lower g value and change the neighbour’s parent to the current node.

• Else If the neighbour is not in both lists:

Add it to the open list and set its g

Game Playing

Game Playing is an important domain of artificial intelligence. Games don’t

require much knowledge; the only knowledge we need to provide is the rules,
legal moves and the conditions of winning or losing the game.
Both players try to win the game. So, both of them try to make the best move
possible at each turn. Searching techniques like BFS (Breadth First Search) are
not accurate for this as the branching factor is very high, so searching will take a
lot of time. So, we need another search procedures that improve–
• Generate procedure so that only good moves are generated.
• Test procedure so that the best move can be explored first.
Mini-Max Algorithm

• Mini-max algorithm is a recursive or backtracking algorithm

which is used in decision-making and game theory. It provides
an optimal move for the player assuming that opponent is also
playing optimally.
• Mini-Max algorithm uses recursion to search through the game-tree.
• Min-Max algorithm is mostly used for game playing in AI.
Such as Chess, Checkers, tic-tac-toe and various two-players
game. This Algorithm computes the minimax decision for the
current state.
• In this algorithm two players play the game, one is called MAX
and the other is called MIN.
• Both the players fight it as the opponent player gets the
minimum benefit while they get the maximum benefit.
• Both Players of the game are opponent of each other, where
MAX will select the maximized value and MIN will select
the Minimized value.
• The minimax algorithm performs a depth-first search
algorithm for the exploration of the complete game tree.
• The minimax algorithm proceeds all the way down to the
terminal node of the tree, then backtrack the tree as the
recursion.
Working of Min-Max Algorithm:

• The working of the minimax algorithm can be easily

described using an example. Below we have taken an
example of game-tree which is representing the two-player
game.
• In this example, there are two players one is called Maximizer
and other is called Minimizer.
 • Maximizer will try to get the Maximum possible score, and
Minimizer will try to get the minimum possible score.
• This algorithm applies DFS, so in this game-tree, we have to go
all the way through the leaves to reach the terminal nodes.
• At the terminal node, the terminal values are given so we will
compare those value and backtrack the tree until the initial state
occurs. Following are the main steps involved in solving the two-
player game tree:
Step-1: In the first step, the algorithm generates the entire game-tree
and apply the utility function to get the utility values for the terminal
states. In the below tree diagram, let's take A is the initial state of the
tree. Suppose maximizer takes first turn which has worst-case initial
value = - infinity, and minimizer will take next turn which has worst-
case initial value=+infinity.

Step2: Now, first we find the utilities value for the Maximizer, its initial
value is-∞, so we will compare each value in terminal state with initial
value of Maximizer and determines the higher nodes values. It will find
the maximum among the all.
For node D max(-1,- -∞) => max(-1,4)= 4
For Node E max(2, -∞) => max(2, 6)= 6
For Node F max(-3, -∞) => max(-3,-5)= -3
For Node G max(0, -∞) => max(0,7)=7
 Step3: In the next step, it's a turn for minimizer, so it will compare all
nodes value with +∞, and will find the 3rd layer node values.
For node B=min(4,6)=4
For node C=min(-3,7)=-3

Step4: Now it's a turn for Maximizer, and it will again choose the maximum
of all nodes value and find the maximum value for the root node. In this game
tree, there are only 4 layers, hence we reach immediately to the root node, but
in real games, there will be more than4layers.
For node A max(4,-3)=4

That was the complete work flow of the minimax two player game
Algorithm:
minimax (player, board)
if(game over in current board position)
return winner
children = all legal moves for player from this board
if(max's turn)
return maximal score of calling minimax on all the
children else (min'sturn)
return minimal score of calling minimax on all the children
Properties of Mini-Max Algorithm:
• Complete- Min-Max algorithm is Complete. It will definitely
find a solution (if exist), in the finite search tree.
• Optimal- Min-Max algorithm is optimal if both opponents
are playing optimally.
• Time complexity- As it performs DFS for the game-tree, so the
time complexity of Min-Max algorithm is O(bm), where b is
branching factor of the game-tree, and m is the maximum depth
of the tree.
• Space Complexity- Space complexity of Mini-max algorithm is
also similar to DFS which is O(bm).

Limitation of the minimax Algorithm:

The main drawback of the minimax algorithm is that it gets really
slow for complex games such as Chess etc., This type of games has a
huge branching factor, and the player has lots of choices to decide