Chapter Three

Problem solving
 Problem
• It is a collection of information that the
agent will use to decide what to do
• Problems have the general form: given
such-and-such information, find x
• A huge variety of problems are
addressed in AI, both:
– toy problems (e.g. 8-puzzle, vacuum cleaner
world, towers of Hanoi, …)
–real-life problems (like Route finding, Traveling
sales person, etc.)
 Example
1: Route-finding problem:
Consider the agent in the city of Arad, toward the end of
touring holiday. Flight leaves tomorrow from city Bucharist
airport. The agent has a ticket to fly out the following day.
The ticket is non-refundable, the agent’s visa is about to
expire after tomorrow and there are no seats available for
six weeks.
2. Vacuum cleaner world problem
An agent vacuum cleaner operates in a simple world with
only two locations to be cleaned.
– Each location may or may not contain dirt
– The agent may be in one location or the other
– The agent can sense
• Location: whether it is in room A or B
• Status: whether the room is Clean or Dirty

The agent is expected to clean all the dirt in the given

locations
 Solving a problem
Formalize the problem: Identify the collection of
information that the agent will use to decide what to do.
• Define states
– States describe distinguishable stages during the problem-
solving process
Example- What are the various states in route finding
problem?
– The various cities including the location of the agent
• Define the available operators/rules for getting from one
state to the next
– Operators cause an action that brings transitions from one
state to another by applying on a current state
• Suggest a suitable representation for the problem
space/state space
– Graph, table, list, set, … or a combination of them
 State space of the problem
• The state space defines the set of all relevant states
reachable from the initial state by (any) sequence of actions
through iterative application of all permutations and
combinations of operators
• State space of the problem includes the various states
– Initial state: defines where the agent starts in or begins with
– Goal state: describes the situation we want to achieve
– Transition states: other states in between initial and goal
states
• Also called search space/problem space and can be
represented in a tree/graph
Example – Find the state space for route finding problem
– Think of the states reachable from the initial state.
 Further example
• Coloring problem
– There are 3 rectangles. Both are initially
white. The problem is to change white color to
black color. Color change is one rectangle at
a time. Define state space for coloring
problem?
 The 8 puzzle problem
• Arrange the tiles so that all the
tiles are in the correct positions. 3 4 5
You do this by moving tiles. You 8 6
can move a tile up, down, left, or 2 1 7
right, so long as the following
conditions are met:
A) there's no other tile blocking
you in the direction of the 3 4 5
movement; and
2 6
B) you're not trying to move
1 8 7
outside of the boundaries/edges.
 Steps in problem solving
• Goal formulation
– is a step that specifies exactly what the agent is trying to achieve
– This step narrows down the state-space that the agent has to look at
• Problem formulation
– is a step that puts down the actions and states that the agent has to
consider given a goal (avoiding any redundant states), like:
• the initial state
• the allowable actions etc…
• Search
– is the process of looking for the various sequence of actions that
lead to a goal state, evaluating them and choosing the optimal
sequence.
• Execute
–is the final step that the agent executes the chosen sequence of
actions to get it to the solution/goal
 Problem Solving Agent Algorithm

function SIMPLE-PROBLEM-SOLVING-AGENT(percept)
returns an action
static: seq, an action sequence, initially empty
state, some description of the current world state
goal, a goal initially null
problem, a problem formulation
state  UPDATE-STATE(state,percept)
if seq is empty then do
goal  FORMULATE-GOAL(state)
problem FORMULATE-PROBLEM(state,goal)
seq  SEARCH(problem)
action  FIRST(seq)
seq  REST(seq)
return action
 Example: Route Finding
Scenario: On holiday in Romania: currently in Arad. Flight
leaves tomorrow from Bucharest. Since then no seats
available for the next six weeks. The ticket is non-refundable,
visa is about to expire, …
• Formalize problem:
–Define the possible states involved to solve the problem
• Formulate goal:
–be in Bucharest
• Formulate problem:
–initial state: in Arad
–states: various cities
–Operators/actions: driving between cities
• Find solution:
–Optimal sequence of cities, e.g., Arad, Sibiu, Fagaras,
Bucharest
State Space for Route Finding
 Example: Coloring Problem
There are 3 rectangles. Both are initially white. The problem
is to change white color to black color. Color change is one
rectangle at a time. Define state space for coloring
problem?
• Formalize problem:
– Define the possible states involved to solve the problem
• Formulate goal:
– All rectangles colored Black
• Formulate problem:
– initial state: all rectangles white
– states: all rectangles white or all black, one or more black/white
– Operators/actions: fill with black color
• Find solution:
– State Optimal sequence of coloring
 Exercise
Missionary-and-cannibal problem:
Three missionaries and three cannibals are on
one side of a river that they wish to cross.
There is a boat that can hold one or two
people. Find an action sequence that brings
everyone safely to the opposite bank (i.e.
Cross the river). But you must never leave a
group of missionaries outnumbered by
cannibals on the same bank (in any place).
Construct the state space of the problem
using suitable representation
 Knowledge and types of problems
• There are four types of problems:
–Single state problems (Type 1)
–Multiple state problems (Type 2)
–Exploration problems (Type 3)
–Contingency Problems (Type 4)
• This classification is based on the level of knowledge that
an agent can have concerning its action and the state of
the world
• Thus, the first step in formulating a problem for an agent
is to see what knowledge it has concerning
–The effects of its action on the environment
–Accessibility of the state of the world
• This knowledge depends on how it is connected to the
environment via its percepts and actions
Example: Vacuum world problem
To simplify the problem (rather than the full
version), let;
• The world has only two locations
– Each location may or may not contain dirt
– The agent may be in one location or the other
• Eight possible world states
• Three possible actions (Left, Right, Suck)
– Suck operator clean the dirt
– Left and Right operators move the agent from
location to location
• Goal: to clean up all the dirt
Clean House Task
Vacuum Cleaner state Space
 Single state problem
• Accessible: The world is accessible to the agent
– It can determine its exact state through its sensors
– The agent’s sensor knows which state it is in
• Deterministic: The agent knows exactly the effect of its actions
– It can then calculate exactly which state it will be in after any
sequence of actions
• Action sequence is completely planned
Example - Vacuum cleaner world
– What will happen if the agent is initially at state = 5 and
formulates action sequence - [Right, Suck]?
– Agent calculates and knows that it will get to a goal state
• Right  {6}
• Suck  {8}
 Multiple state problems
• Deterministic: The agent knows exactly what each of its actions do
– It can then calculate which state it will be in after any sequence of actions
• Inaccessible: The agent has limited access to the world state
– It might not have sensors to get full access to the environment states or as
an extreme, it can have no sensors at all (due to lack of percepts)
• If the agent has full knowledge of how its actions change the world,
but does not know of the state of the world, it can still solve the task
Example - Vacuum cleaner world
– Agent’s initial state is one of the 8 states: {1,2,3,4,5,6,7,8}
– Action sequence: {right, suck, left, suck}
– Because agent knows what its actions do, it can discover and reach to goal
state.
Right  [2.4.6.8.] Suck  {4,8}
Left  {3,7} Suck  {7}
 Contingency Problems
• Non-deterministic
– The agent is ignorant of the effect of its actions
• Inaccessible
– The agent has limited access to the world state
• Sometimes ignorance prevents the agent from finding a
guaranteed solution sequence.
• Suppose the agent is in Murphy’s law world
– The agent has to sense during the execution phase, since things
might have changed while it was carrying out an action. This
implies that
• the agent has to compute a tree of actions, rather than a linear sequence of
action
– Example - Vacuum cleaner world:
• action ‘Suck’ deposits dirt on the carpet, but only if there is no dirt already.
Depositing dirt rather than sucking returns from ignorance about the effects
of actions
 Contingency Problems (cont…)
• Example - Vacuum cleaner world
– What will happen given initial state {1,3}, and action
sequence: [Suck, Right, Suck]?
{1,3}  {5,7}  {6,8}  {8,6} (failure)
• Is there a way to solve this problem?
– Thus, solving this problem requires local sensing, i.e.
sensing the execution phase,
– Start from one of the states {1,3}, and take improved action
sequence [Suck, Right, Suck (only if there is dirt there)]
• Many problems in the real world are contingency
problems (exact prediction is impossible)
– For this reason many people keep their eyes open while
walking around or driving.
 Exploration problem
• The agent has no knowledge of the environment
– World Inaccessible: No knowledge of states (environment)
• Unknown state space (no map, no sensor)
– Non-deterministic: No knowledge of the effects of its actions
– Problem faced by (intelligent) agents (like, newborn babies)
• This is a kind of problem in the real world rather than in a model,
which may involve significant danger for an ignorant agent. If the
agent survives, it learns about the environment
• The agent must experiment, learn and build the model of the
environment through its results, gradually, discovering
– What sort of states exist and What its action do
– Then it can use these to solve subsequent (future) problems

Example: in solving Vacuum cleaner world problem the agent

learns the state space and effects of its action sequences say: [suck,
Right]
 Well-defined problems and solutions
To define a problem, we need the following elements
•The Initial state: is the state that the agent
starts in or begins with.
–Example- the initial state for each of the following:
•Route finding problem
– Arad (states are the various cities)
•Coloring problem
– All rectangles white
 Operators
• The set of possible actions available to the agent,
i.e.
–which state(s) will be reached by carrying out the action
in a particular state
–A Successor function S(x)
• Is a function that returns the set of states that are reachable from
a single state by any single action/operator
• Given state x, S(x) returns the set of states reachable from x by
any single action

• Example:
–Route finding problem: Drive between cities
drive (city x, city y)
–Coloring problem: Paint black color
paint (color w, color b)
 Goal test function
• The agent execute to determine if it has
reached the goal state or not
– Is a function which determines if a single state is a
goal state

• Example:
– Route finding problem:
Reach Bucharest airport on time (IsCity(x,
Bucharest))
– Coloring problem:
All rectangles black (IsBlack(x, y, z)
 Path cost function
• A function that assigns a cost to a path (sequence
of actions).
– Is often denoted by g. Usually, it is the sum of the costs
of the individual actions along the path (from one state to
another state)
– A sequence of actions in the state space. For example,
we may prefer paths with fewer or less costly actions

• Example:
– Route finding problem:
• Path cost from initial to goal state
– Coloring problem:
• One for each transition from state to state till goal state is
reached
 Example problems
We can group well defined problems into two:
• Toy Problems
–Are problems that are useful to test and demonstrate
methodologies
–Concise and exact description (abstract version) of real
problem
–Can be used by researchers to compare the performance of
different algorithms
Example:
– The vacuum Cleaner world, The 8-puzzle, The 8-queens problem

• Real-World Problem
–More difficult and complex to solve, and there is no single
agreed-upon description
–Have much greater commercial/economic impact if solved
Example:
– Route finding, Traveling sales person
 The Vacuum Cleaner World
• Problem formulation
– State : one or both of the eight states shown in the
Figure
– Operators: move left, move right, suck
– Goal test: no dirt left in any square
– Path cost: each action costs 1
 The 8-puzzle
• Problem formulation
–State: the location of each of the eight tiles in one of the
nine squares
–Operators: blank space moves left, right, up, or down
–Goal test: state matches the right figure (goal state)
–Path cost: each step costs 1, since we are moving one
step each time.

5 4 1 2 3
6 1 8 8 4
7 3 2 7 6 5
Initial State Goal State
 The 8-queens problem
• Problem formulation
– State : any arrangement of 1 to 8 queens on board
– Operators: move a queen to any square
– Goal test: 8 queens on board, none attacked
– Path cost: zero
Searching Strategies

Uninformed Search
 Examples of Search Problems
• Route Finding path: Search through set of paths
–Looking for one which will minimize distance
• Machine learning: Search through a set of
concepts
–Looking for a concept which achieves target categorisation
• Chess: search through set of possible moves
–Looking for one which will best improve position
• Theorem proving: Search through sets of
reasoning steps
–Looking for a reasoning progression which proves
theorem
• Missionaries and Cannibals: Search through set
of possible crossing the river
–Looking for one which transports missionaries & cannibals
 Problem Spaces
• For the given problem we have to identify:
– A set of states
– Special states: initial state(s), goal state(s)
– Successor function (also called operators)
• In combination, these create problem space or a
graph which can be used for searching

• What else might we know about a Problem?

– Constraints on operators
– Costs incurred by applying operators
– Metrics for how close we might be to a solution
 Search
• Examine different possible sequences of actions &
states, and come up with specific sequence of
operators/actions that will take you from the initial
state to the goal state
–Given an initial state and a goal, find the sequence of
actions leading through a sequence of states to the final
goal state

• During searching, the agent must use:

–All available knowledge about a problem to avoid
unnecessary moves during searching for optimal solution
–Some times, the agent also use heuristic knowledge
(knowledge that is true from experience)
 Where to Search
• State space
–For a well-defined problem we know states, operators
(actions), goal test function and cost function.
– Then we construct the state space that contains a set of
states.
–State space is the environment in which the search takes
place.
–To find the optimal solution to our problem, there is a need to
search through the search space.

• Searching State Space

– choose a node/state in the state space,
– test the state to know whether we reached to the goal state
or not,
– if not, expand the node further to identify its successor.
What does search refers to in route finding
problem ?
What does search refers to in route finding
problem ?
 Search Tree
• The searching process is like building the search tree that is
super imposed over the state space
– A search tree is a representation in which nodes denote paths and
branches connect paths. The node with no parent is the root node. The
nodes with no children are called leaf nodes.
Example: Route finding Problem
• Partial search tree for route finding from Arad to Bucharest.
goal test
(a) The initial state Arad

Arad
(b) After expanding Arad generating a new state
Sibiu Timisoara Zerind
choosing one Arad
option

(c) After expanding Sibiu Sibiu Timisoara Zerind

Fagaras Oradea Rimnicu Vilcea

 Searching for Solutions
Example 1: 8-puzzle: Given 8 tiles and a blank space,
in a square: Show the state space diagram that
generate new arrangement till the goal state is
reached
–Empty space can move up, down, left, right; Cannot move
diagonally

Initial state Goal state

7 2 4 1 2 3
5 8 6 8 4
3 1 7 6 5
 8-puzzle Search Tree
Initial state 7 2 4
5 8 6
3 1
up right

7 2 4 7 2 4
8 6 5 8 6
5 3 1 3 1

2 4 7 2 4 7 2 4 7 2 4 7 2 4 7 2 4
7 8 6 8 6 5 8 6 5 6 5 8 6 5 8 6
5 3 1 5 3 1 3 1 3 8 1 3 1 3 1
 Search algorithm
• Two functions needed for conducting search
–Generator (or successors) function: Given a state and action, produces
its successor states (in a state space)
–Tester (or IsGoal) function: Tells whether given state S is a goal state
IsGoal(S)  True/False
–IsGoal and Successors functions depend on problem domain.

• Two lists maintained during searching

–OPEN list: stores the nodes we have seen but not explored
–CLOSED list: the nodes we have seen and explored
–Generally, search proceeds by examining each node on the OPEN list,
performing some expansion operation that adds its children to the OPEN
list, and moving the node to the CLOSED list.

• Merge function: Given successor nodes, it either append,

prepend or arrange based on evaluation cost
• Path cost: function assigning a numeric cost to each path
–Either from initial node to current node and/or from current node to goal
node
 Search algorithm
• Input: a given problem (Initial State + Goal state + transit
states and actions) and
• Output: returns a solution, i.e. an optimal sequence of actions
to reach the goal.
function GeneralSearch (problem, strategy){
open = (C_0); //put initial state C_0 in the List
closed = {}; //maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
l = not-in-set ( Successors (f), closed );
open = merge ( rest(open), l); //append or pre_pend l to
open list
}
return ('fail')
}
 Algorithm Evaluation: Completeness
• Is the algorithm guarantees in finding a solution
whenever one exists
– Think about the density of solutions in space
– Searches guaranteed to find all solutions
• Particular tasks may require one/some/all
solutions
– E.g., how many different ways to get from A to B?

• Pruning versus exhaustive searches

– Exhaustive searches try all possibilities
– If only one solution required, can employ pruning
• Rule out certain operators on certain states
– If all solutions are required, we have to be careful with
pruning
• Check no solutions can be ruled out
 Algorithm Evaluation: Time and Space
Tradeoffs
• With many computing projects, we worry about:
– Speed versus memory
– Time complexity: how long does it take to find a solution
– Space complexity: how much space is used by the
algorithm
• Fast programs can be written
– But they use up too much memory
• Memory efficient programs can be written
– But they are slow
• We consider various search strategies
– In terms of their memory/speed tradeoffs
 Algorithm Evaluation
Optimality
• is the algorithm finding an optimal solution;
i.e. the one with minimum cost
– how good is our solution?

• Is the search strategies find solutions to

problems with no solutions
• Is it produces incorrect solutions to
problems
 Searching Strategies: classification
•Search strategy gives the order in which the search
space is examined
•Uninformed (= blind) search
–Also called weak search methods, most general methods
are brute-force because they do not need domain knowledge
that guide them to the right direction towards the goal
–Have no information about the number of steps or the path
cost from the current state to the goal
–It is important for problems for which there is no additional
information to consider
•Informed (= heuristic) search
–have problem-specific knowledge (knowledge that is true from
experience)
–Have knowledge about how far are the various state from the
goal
–Can find solutions more efficiently than uninformed search
 Search Methods:
• Uninformed search
– Breadth first
– Depth first
– Uniform cost, …
– Iterative deepening
– Etc.

• Informed search
– Greedy search
– A*-search
– Iterative improvement,
– Constraint satisfaction
– etc.
 Breadth first search
•Expand shallowest unexpanded node,
–i.e. expand all nodes on a given level of
the search tree before moving to the next
level
•Implementation: use queue data
structure to store the list:
–Expansion: put successors at the end of
queue
–Pop nodes from the front of the queue
•Properties:
–Takes space: keeps every node in
memory
–Optimal and complete: guarantees to
find solution
 Algorithm for Breadth first search
function BFS (problem){
open = (C_0); //put initial state C_0 in the List
closed = {}; /maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed );
open = merge ( rest(open), l); //append to the
list
}
return ('fail')
}
 Uniform cost Search
•So far we’ve ignored the issue of edge costs.
•The goal of this technique is to find the shortest path
to the goal in terms of cost.
–It modifies the BFS by always expanding least-cost
unexpanded node
•Implementation: nodes in list keep track of total path
length from start to that node
–List kept in priority queue ordered by path cost
A
S S S
1 10 S
5 B 5
S G 0 A B C A B C A B C
1 5 15 5 15 15
G G G
15 5
11 11 10
C
•Properties:
–This strategy finds the cheapest solution provided the cost of
a path must never decrease as we go along the path
g(successor(n)) ≥ g(n), for every node n
–
 Algorithm for Uniform Cost search
function uniform_cost (problem){
open = (C_0); //put initial state C_0 in the List
g(s) = 0;
closed = {}; /maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed );
g(f,li) = g(f) + c(f,li);
open = merge(rest(open), l, g(f,li)); //keep the open list sorted in
ascending order by edge cost
}
return ('fail')
}
 Depth-first search
•Expand one of the node at the deepest
level of the tree.
–Only when the search hits a non-goal dead
end does the search go back and expand
nodes at shallower levels
•Implementation: treat the list as stack
–Expansion: push successors at the top of
stack
–Pop nodes from the top of the stack
•Properties
–Incomplete and not optimal: fails in infinite-
depth spaces, spaces with loops.
• Modify to avoid repeated states along the path
–Takes less space (Linear): Only needs to
remember up to the depth exapanded
 Algorithm for Depth first search
function DFS (problem){
open = (C_0); //put initial state C_0 in the List
closed = {}; /maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed );
open = merge ( rest(open), l); //prepend to the
list
}
return ('fail')
}
Depth-First & Breadth-First Search
• In the case of DFS and
BFS a search is
determined by the order in
which the nodes are
examined.
• Consider the following
graph:

• DFS examines the nodes in the following order:

A, B, E, K, S, L, T, F, M, C, G, N, H, O, P, U, D, I, Q, J,
R
• BFS examines the nodes in the following order:
A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T,
 Iterative Deepening Search (IDS)
• IDS solves the issue of choosing the best depth limit by trying
all possible depth limit:
–Perform depth-first search to a bounded depth d, starting at d = 1 and
increasing it by 1 at each iteration.
Example: for route finding problem we can take the diameter of
the state space. In our example, at most 9 steps is enough to
reach any node
• This search combines the benefits of DFS and BFS
–DFS is efficient in space, but has no path-length guarantee
–BFS finds min-step path towards the goal, but requires memory space
–IDS performs a sequence of DFS searches with increasing depth-cutoff
until goal is found
Limit=0 Limit=1 Limit=2
 Algorithm for IDS
function IDS (problem){
open = (C_0); //put initial state C_0 in the List
closed = {}; /maintain list of nodes examined earlier
while (not reached maxDepth) {
while (not (empty (open))) {
f = remove_first(open);
if (IsGoal (f)) then return (f);
closed = append (closed, f);
l = not-in-set (Successors (f), closed);
if (depth(l) < maxDepth) then
open = merge (rest(open), l); //prepend to
the list
}
}
return ('fail')
}
 Bidirectional Search
• Simultaneously search both forward from the initial
state to the goal and backward from the goal to the
initial state, and stop when the two searches meet
somewhere in the middle
– Requires an explicit goal state and invertible operators (or
backward chaining).
– Decide what kind of search is going to take place in each
half using BFS, DFS, uniform cost search, etc.

Start Goal
 Bidirectional Search
• Advantages:
– Only need to go to half depth
– It can enormously reduce time complexity, but is not
always applicable

• Difficulties
– Do you really know solution? Unique?
– Cannot reverse operators
– Memory requirements may be important: Record all
paths to check they meet
• Memory intensive

• Note that if a heuristic function is inaccurate, the

two searches might miss one another.
 Comparing Uninformed Search
Strategies Complete Optimal Time Space
complexity complexity

Breadth first search yes yes O(bd) O(bd)

Depth first search no no O(bm) O(bm)
Uniform cost search yes yes O(bd) O(bd)
Iterative deepening yes yes O(bd) O(bd)
search
bi-directional search yes yes O(bd/2) O(bd/2)
• b is branching factor,
• d is depth of the shallowest solution,
• m is the maximum depth of the search tree,
• l is the depth limit
Informed Search
 Informed Search
•Search efficiency would improve greatly if there is a
way to order the choices so that the most promising
are explored first.
–This requires domain knowledge of the problem.
•Add domain specific knowledge to select what is the
best one to continue searching along the path
–Define a heuristic function, h(n) that estimates the
goodness of a node n, based on domain specific information
that is computable from the current state description.
•Heuristics is knowledge about the domain that helps
to undertake focused search
–It is rule of thumb, or guess that limits the search space
towards the goal
Example: 8- puzzle
 Heuristic function, h(n)
•It is an estimate of how close we are to a goal state
•Heuristic function h(n)
h(n) = estimated cost of path from state n to a goal state.
•What good heuristic function for the following
problems:
–Route finding: straight line distance
–8-puzzle: (i) Number of mismatched tiles and (ii) Manhattan
or city block distance (sum of distances each tile is from its
goal position)
–8-queens: Min-conflict or max conflict – (that is, the number
of attacking queens)
•Note that:
–h(n) ≥ 0 for all nodes n
–h(n) = 0 if n is a goal node
–h(n) = infinity if n is a dead-end from which a goal can’t be
reached
 Best first search
•It is a generic name to the class of informed methods
•Nodes are ordered in the open list so that the one
with the best evaluation (minimum cost) is expanded
first
–Evaluation function, f incorporates domain specific
information and determines the desirability of expanding the
node
–It aims to find low-cost solutions towards goal node
•There are two measures
–The use of the path cost g to decide which path to extend
further (e. g. uniform cost search).
–The use of some estimate of the cost of the path from
current state to the goal state (heuristics h)
–The first measure may not direct search towards the goal. It
needs to be supplemented by the second measure
 Best First Search Approaches
•Two approaches to find the shortest path:
–Greedy search: minimizes estimated cost to reach a goal
–A*-search: minimizes the total path cost
•When expanding a node n in the search tree, greedy
search uses the estimated cost to get from the
current state to the goal state, define as h(n).
–In route finding problem h(n) is the straight-line distance
•We also possess the sum of the cost to reach that
node from the start state, define as g(n).
–In route finding problem this is the sum of the step costs for
the search path.
•For each node in the search tree, an evaluation
function f(n) can be defined as the sum of these
functions.
f(n) = g(n) + h(n)
 Algorithm for Best First search
• Implementation:
– Sort states (in ascending order) by the heuristic estimate to the goal
– Expand first the one with the minimum cost for which the estimate is
minimized

function Best_first_search (problem, eval_fun){

open = (C_0); //put initial state C_0 in the List
closed = {}; /maintain list of nodes examined earlier
while (not (empty (open))) {
f = remove_first(open);
if IsGoal (f) then return (f); //goal test
closed = append (closed, f);
l = not-in-set (Successors (f), closed );
open = merge ( rest(open), l, eval_fun); //sort the list using
evaluation function
}
return ('fail')
}
 Admissibility
• Search algorithms (such as greedy and A*- search)
are admissible when the heuristic function never over
estimated the actual cost so that
–the algorithm always terminates in an optimal path from the
initial state to goal node if one exists.
• check whether estimated cost h(n) is not
overestimated
–h*(n): Actual cost of shortest path from n to z (not known)
–h(n) is said to be an admissible heuristic function if for all n,
h(n) ≤ h*(n)
–The closer h to h* the fewer extra nodes that will be
expanded
–Using an admissible heuristics guarantees that the solution
found by searching algorithm is optimal
 Greedy search
• A best first search that uses a heuristic function h(n)
alone to guide the search
–Selects node to expand that is closest (hence it’s greedy) to
a goal node
–The algorithm doesn’t take minimum path costs from initial
to current nodes into account, it just go ahead optimistically
and never looks back.
• Implementation: expand 1st the node closest to the goal
state, i.e. with evaluation function f(n) = h(n)
–h(n) = 0 if node n is the goal state
–Otherwise h(n) ≥ 0; an estimated cost of the cheapest path
from the state at node n to a goal state
Example 1: Route finding problem
Example 2: Can we arrange the search in the 8-puzzle so that
a move will minimize the distance b/n the initial state & the
goal arrangement
Exercise
 Solution to Route Finding
Arad f(n) = 366

Sibiu 253 Timisoara 329 Zerind 374

Arad 366 Fagaras 176 Oradea 380 Rimnicu Vilcea 193

Bucharest 0 Sibiu 253

Is Arad  Sibiu  Fagaras  Bucharest optimal?

Total cost = 450
• Incomplete:
Properties
– may not reach goal state because it may start down an infinite path and
never return to try other possibilities
• Not optimal
– if estimated heuristic cost is not admissible
• Takes memory
– It retains all the nodes in memory
– Space complexity is exponential: O(bm), where m is the maximum depth
of the search space and b is branching factor
• Time complexity
– It takes exponential time: O(bm)
• With good heuristic function the space and time complexity can
be reduced substantially
• The problem with greedy search is that it didn’t take costs so
far into account.
– We need a search algorithm (like A* search) that takes into consideration
this cost so that it avoids expanding paths that are already expensive
 A*-search Algorithm
•It considers both estimated cost of getting from n to
the goal node h(n), and cost of getting from initial
node to node n, g(n)
•Apply three functions over every nodes
–g(n): Cost of path found so far from initial state to n
–h(n): Estimated cost of shortest path from n to z
–f(n): Estimated total cost of shortest path from a to z via n
–Evaluation function f(n) = h(n) + g(n)
•Implementation: Expand the node for which the evaluation
function f(n) is lowest
–Rank nodes by f(n) that goes from the start node to goal node
via given node
Example: Route finding problem
Solution to Route Finding
Arad f(n) = 0 + 366

Sibiu 393 Timisoara 447 Zerind 449

=140+253

413 =
220+193
Arad 646 Fagaras 415 Oradea 671
Rimnicu Vilcea

415
=317+98
Pitesti Craiova 573

418
=418+0 Bucharest Craiova 615 Rimnicu 607
 Properties
• A* search is complete, optimal, and optimally
efficient for any given admissible heuristic function
• Complete
– It guarantees to reach to goal state
• Optimality
– If the given heuristic function is admissible, then A* search
is optimal.
– Proof optimality of A* search
• Takes memory:
– It keeps all generated nodes in memory
– Both time and space complexity is exponential
• Time and space complexity of heuristic algorithms
depends on the quality of the heuristic function.
– Worst case is exponential
 Local Search
• In many problems, the path to the goal is irrelevant; the goal state itself is
the solution
• Being able to ignore the path offers two main advantages:
–Less memory is used, generally a constant amount.
–A reasonable solution can be found in infinite search spaces for which
systematic search would be unsuited.
–They are also very useful for optimization problems where the goal is to find the
best state according to a function. Many of these problems are not suitable for
standard search.

Example: n-queens problem: Put n queens on an n × n board with no two

queens on the same row, column, or diagonal.
–In these cases, we can use local search algorithms that keep a single current
state, try to improve it.
• The general idea of local search is to start with a complete configuration
and to make modifications to improve its quality
• Implementation:
–Consider all states laid out on the surface of a landscape
–It moves around the landscape trying to find the optimal solutions
• Search algorithms: Hill climbing, Simulated annealing, ….
 Hill–climbing Algorithm
• The idea here is that we don’t keep track of our path,
we don’t check for repeated states;
–It just keep track of the current state
–it looks around current position & moves uphill or downhill.
• Hill climbing is best suited to problems where the
heuristic improves the closer it gets to the solution;
–it works poorly where there are sharp drop-offs. It assumes
that local improvement will lead to global improvement.

• Implementation:
–Evaluate the successors and chose the best state that leads
us closer to the goal (according to the heuristic estimate), and
continue from there
 Hill climbing algorithm
Function hill_climbing(problem) returns a solution state
variable: current and next (nodes)
current = initial state
while current ≠ NULL do
next = a highest value successor of current
if (value (next) < value (current)) then
return current
current = next
end loop
end function
 Properties
• If an accurate heuristic measure is available in the
domain, and if there are no local maxima, the
algorithm is
–fast to reach to goal state
• it just looks a head to determine if any successor is better than the
current state
–takes less memory
• space requirement is constant since current state is the only node
stored

• Incomplete and not optimal

–Since it didn’t remember previously generated nodes, it may
get back to the node that is already examined and hence
become infinite
• No special implementation data structure required
since hill climbing discards old nodes
–In case the path followed doesn’t lead to goal state, there is
no way to backtrack or to jump to an alternate path, since no
information is remembered.
 Hill – climbing…
Example: The 8-puzzle problem
Rearrange the titles until the final goal is achieved.
For evaluation consider the number of mismatched
tiles.

2 8 3 1 2 3
1 6 4 8 4
7 5 7 6 5
 Hill – climbing…
• Traveling Sales person problem: Given a set of cities, with
costs of getting from one to the other, find a minimum cost
route that goes through each of them exactly once.
• Consider the following example: Salesperson lives in city A. He has to
visit all the 5 cities, A to E, and must return home afterwards. Goal is
circle from A to A with minimal cost. Find shortest path for travel
minimize cost and/or time of travel
 Problems with hill climbing
• May lead to local maxima – once it reaches to local maxima, it
can’t go to global max. It caught by small foothills.
• Walking in the Plateau - situations where all of the local steps
are indistinguishable; all of them have the same value.
• ridges traversal - there is a direction we would like to move, but
none of the allowed steps actually takes us in that direction.
 Possible Improvements
• Random-restart (Simulated Annealing)
–To avoid the problems of hill-climbing, we need to mix some
randomness with the greedy uphill steps such that
• incase the algorithm stuck on a local maximum, we could allow the
search to take some down hill steps to escape the local maximum
• The basic idea is that, instead of always going up the
sharpest incline, pick a random direction.
–If the move improves the situation; i.e. if it goes uphill, go
there and execute it;
–otherwise, go downhill with probability which depends on
how bad the move is (the probability decreases
exponentially with the badness of the move.
 Simulated Annealing vs. Hill climbing
Ball on terrain example: The ball is initially placed at a random position on the
terrain. From the current position, the ball should be fired such that it can only
move one step left or right. What algorithm should we follow for the ball to
finally settle at the lowest point on the terrain?
Initial position
of the ball Simulated Annealing explores
more. Chooses this move with a
small probability (Hill Climbing)

Greedy Algorithm
gets stuck here!
Locally Optimum
Solution.

Upon a large no. of iterations,

SA converges to this solution.
 Heuristic Search Methods
Name Time Space Optimal Complete
Comp. Comp.

Greedy O(bm) O(bm) No No

A* O(bd) O(bd) yes optimal if h'(n) yes

never overestimates
Hill bm O(1) No No, it needs a way of
Climbing escaping from local
minima
Simulated unbou O(1) Yes will find optimal
Annealing nded solution if cooling is
slow enough and as
time goes to infinity

b is the branching factor; d is the depth of solution; m is the maximum

depth of the search tree; N the number of nodes
 Constraint satisfaction problem (CSP)
• It can be solved by iterative improvement method
–First assign values to all variables
–Then apply modifications operators to move the configuration
towards the goal
• Modification operators simply assign a different value
to a variable
–In choosing a new value for a variable, the most obvious
heuristic is to select the value that results in the minimum
number of conflicts with other variables - the min-conflicts
heuristic or max-conflicts heuristic

• Algorithms that solves CSPs problems in this fashion

are called heuristic repair methods
Example: the 8-queens problem
 The 8-queens problem formulation
• Place 8 queens on a board so that no queen is
attacking another queen (see next slide for detail).
• Initial State:
– 8 queens placed randomly on the board, one per column.
• Successor function:
– States that obtained by moving one queen to a new
location in its column.
• Heuristic/objective function: The number of pairs of
attacking queens.
– Use min-conflict heuristic to move attacked queen
column-wise.
– Use max-conflict to move the most attacked queen
– Break any ties randomly.
• Cost function: each move costs 1.
 8-Queens Example 1
1 2
2 2
2 2
1 1 Solution found
3 3 in 4 steps using
2 2 min-conflicts
heuristic
2 3
1 1
1 1
3 2
2 3
3 3
1 0
2 2
1 1
2 1
 Games Playing
as Search Problem
 Tic-Tac-Toe Problem
• Given n x n matrix, the object of the game is to
make three of your symbol in a row, column or
diagonal
– This is a win state for the game.
• One player is designated as player X and makes
the first play by marking an X into any of the n x n
open squares of the board.
– Think a 3 x 3 square board where the player can put an
X in any of the 9 open places
• The second player, "O", then follows suit by
marking an O into any of the other open squares
that remain.
• This continues back-and-forth until one player wins
the game or players fill all squares on the board
without establishing a winner. This is a draw.
Partial Game Tree for Tic-Tac-Toe Problem
 How to play a game
•A way to play a game is to:
–Consider all the legal moves you can make
–Compute the new position resulting from each move
–Evaluate each resulting position and determine which is best
–Make that move
–Wait for your opponent to move and repeat
•Key problems are:
–Representing the “board”
–Generating all legal next boards
–Evaluating a position
•For real problems, the search tree is too big to make it
possible to reach the terminal states
–Example: Chess: ~10120 nodes. How many nodes to consider
in a Tic-Tac-Toe game with 3 x 3 board?
 Adversarial Search
•It is used in game playing since one player's
attempts to maximize their fitness (win) is opposed
by another player.
•The search tree in adversarial games such as tic-tac-
toe consist of alternating levels where the moving
(MAX) player tries to maximize fitness and then the
opposing (MIN) player tries to minimize it.
•To find the best move
–The system first generates all possible legal moves, and
applies them to the current board.
–Evaluate each of the positions/states and determine the
best one to move.
–In a game like tic-tac-toe this process is repeated for each
possible move until the game is won, lost, or drawn.
 Typical case
•2-person game: two players alternate moves
e.g. chess playing, TIC-TAC-TOE, foot ball, …
•Zero-sum game: one player’s loss is the other’s gain
–The zero-sum assumption allows us to use a single evaluation function
to describe the goodness of a board with respect to both players.
–f(n) > 0: position n good for A and bad for B
–f(n) < 0: position n bad for A and good for B
–f(n) near 0: position n is a neutral position
–f(n) = +infinity: win for A
–f(n) = -infinity: win for B

•Perfect information: both players have access to complete

information about the state of the game. No information is
hidden from either player.
–Board configuration is known completely to both players at all times.

Examples: Tic-Tac-Toe and Chess are a perfect information

game.
–Playing cards is not. Cards held by one are not known to others.
 Game Tree Search
•Problem spaces for typical games are represented as
trees
•Game tree represents possible moves by both
players given an initial configuration.
–Each node represents a (board) configuration. Ex. Each
node marked by a letter (A, B etc.)
–Root node represents initial configuration
–Children of a node n indicate possible configurations after
the player makes a move from node n
• Ex. B, C, D are children of A. They are possible configurations after
player makes a move from configuration A.
–In case of alternating moves between the players,
alternating levels in a tree have alternating moves.
• Each level of the tree has nodes that are all MAX or all MIN; nodes at
level i are of the opposite kind from those at level i+1.
 Example Game tree
•Game between players X and Y. Let us analyze from X's
perspective by looking ahead of two moves.
• H is a winning state for X (marked by plus infinity). If H reached, X wins.
No further moves.
• E, M are losing states for X (marked by minus infinity). If E or M reached,
X loses. No further moves.

Question: Where should X move from A? Should X move to B, C, or D?

what is the optimal play ?
 How to go searching?
•For many games, we can look the progress of the
game based on the player’s position on the board.
–We can use heuristics to evaluate these board positions and
judge how good (a chance of winning) any one of the next
moves can be.
–Evaluation is done on leaf nodes
• +inf stands for sure win,
• -inf for sure loss.
• Other numbers stand for intermediate values

•To obtain values of non-leaf nodes, analyze alternate

levels in game tree as maximizing and minimizing
levels.
•Two techniques:
– Min-max algorithm
–Alpha and Beta pruning
 Min-max Algorithm
• It is a method in decision theory for minimizing the
maximum possible loss.
– Alternatively, it can be thought of as maximizing the
minimum gain (max-min).
– It started from two player zero-sum game theory,
considering the case where players take alternate
moves.
• The Min-max algorithm helps find the best move,
by working backwards from the end of the game.
– At each step it assumes that player A is trying to
maximize the chances of A winning,
– On the next turn player B is trying to minimize the
chances of A winning (i.e., to maximize B's own chances
of winning).
 Min-max procedure
• Create start node as a MAX node with current board
configuration
• Expand nodes down to some depth of look ahead in
the game
–The min-max algorithm proceeds with depth-first search
down to the terminal states at the bottom of the tree.
• Apply the evaluation function at each of the leaf
nodes
• “Back up” values for each of the non-leaf nodes until
a value is computed for the root node
–At MIN nodes, the backed-up value is the minimum of the
values associated with its children.
–At MAX nodes, the backed up value is the maximum of the
values associated with its children.
 Example
• Lets consider perfect two players game
•Two players MAX and MIN take turn (with MAX playing first)
• Score tells whether a terminal state is a win, loss, or draw (for MAX)
•Perfect knowledge of states, no uncertainty in successor
function

This is the optimal

play
 Min-max algorithm
function min-max(node, depth)
if node is a root node or depth = MaxDepth then
return the heuristic value of node
if MIN turn then
minimum = +∞
for each child of node do
minimum = min(minimum, min-max(child, depth+1))
return minimum
else
maximum = -∞
for each child of node do
maximum = max(maximum, min-max(child,
depth+1))
return maximum
end min-max
Use Min-max to find optimal play
 Comments
• A complete depth-first search exploration of the
game tree is made by min-max algorithm.
– In a tree of maximum depth m with b legal moves
at any point results in complexity on the order of bm.
– This exponential complexity means that min-max by
itself is not practical for real games.

• Is there a way to improve the performance of Min-

Max algorithm without affecting the result?
 Exercise: Identify the optimal play path?
MAX

MIN

MAX

MIN

MAX

MIN
 Alpha-Beta Pruning
• We can improve on the performance of the min-
max algorithm through alpha-beta pruning

• Basic idea: “If you have an idea that is surely bad,

don't take the time to see how truly awful it is.”

• Alpha Beta pruning reduces the complexity by

pruning parts of the tree that won't influence
decision making
– With this we don't have to search the complete tree.
 Alpha-Beta Pruning

• We don’t need to compute the value at this node

–No matter what it is, it can’t affect the value of the root node
 Alpha-Beta Procedure
• Two values are used by this procedure.
–Alpha - the highest-value found so far at any choice-point along
the path for Max
–Beta - the lowest-value found so far at any choice-point along the
path for Min
• Traverse the search tree in depth-first order and update
alpha and beta values for each search node as the
search progresses
–At each MAX node n, alpha(n) = maximum value found so far
–At each MIN node n, beta(n) = minimum value found so far
–The alpha values start at -infinity and only increase, while beta
values start at +infinity and only decrease.

• Prune branches that can not change the final decision

–Stop searching when
alpha(i) ≥ beta(n) for some MAX node ancestor i of n.
Example: mark pruned paths with ‘X’

XX
 Exercise: mark pruned paths with ‘X’
MAX

MIN

MAX

MIN

MAX

MIN
 Effectiveness of alpha-beta
• Alpha-beta is guaranteed to compute the same
value for the root node as computed by min-max,
with less or equal computation
• Worst case: no pruning, examining bm leaf nodes,
where each node has b children and a m-depth
search is performed
• Best case is when each player’s best move is the
first alternative generated
– Best case: examine only bm/2 leaf nodes.
– The result shows that you can search twice as
deep as min-max.
• In Deep Blue (chess program), they found
empirically that alpha-beta pruning meant that the
average branching factor at each node was about 6
instead of about 35!