1 Problem Solving

1.1 Introduction
A goal and a set of means for achieving the goal, is called a problem; in other words, a problem is a collection of
information that an agent will use to decide what to do. The process of exploring the various action sequences that
may lead to a solution to the problem, and then choosing the best one, is called search. We consider in this section
how a problem-solving agent can decide what to do by systematically considering the outcomes of various
sequences of actions that it might take.

Figure 1: Simplified roadmap of Romania (adapted from Russel & Norvig (1995))

We illustrate the processes involved by considering the following problem (see map in Figure 1). Assume the
agent represents a holiday maker currently in the city of Arad, Romania. The agent has a ticket to fly out of the
city of Bucharest the next day. The ticket is non-refundable, and the agent’s visa is about to expire. If the agent
misses the flight of tomorrow, then the next available seat on a plane will only be in about 2 months time because
this is the peak of the travel season. Because of the seriousness of the current situation, the agent should adopt the
goal of driving to Bucharest so that it can catch the flight of tomorrow. Such a goal helps the agent by limiting its
search space (i.e., the various things the agent has to take into consideration before arriving to a solution). For
example, the agent can out rightly reject all actions that will prevent it from reaching Bucharest on time. Goal
formulation based on the current situation, is the first step in problem solving.
What is a goal? Consider there to be a number of world states. Goal states are a set of world states that satisfy the
goal. Agent actions cause transitions between the world states. Some of these transitions cause the agent to reach
goal states. The job of the agent is thus to find those transitions that lead to goal states.
The next task for the agent is problem formulation. What sorts of actions should the agent consider? Should it
consider the possible transitions in world states that result from moving an inch at a time, or should the agent
consider the transitions that cause it to drive from one city to the next? Obviously, the first of these options is
inappropriate for this problem, as the amount of computations and memory requirements for such fine-grained
processing is beyond the capabilities of any computer. Problem formulation is the process of deciding what
actions and states to consider. In this example, our agent will consider actions at the level of driving from one city
to another.

Page 1 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

The third task for the agent is to search for a solution. Search is the process of looking for a sequence of actions
that reaches the goal. A search algorithm takes a problem as input and returns a solution in the form of an action
sequence. Our agent has adopted the goal of driving to Bucharest, and is now considering which town to drive to
from its current location, Araz. As we can see from the map, none of the three roads from Arad leads directly to
Bucharest. If the agent does not know the geography of Romania, or has no access to a map, then it cannot know
which of the three roads is best to follow to take it closest to its goal. In such a case, the agent can only make a
random choice. But if the agent has the map in its memory, then the agent can determine which states it might get
into if it followed particular actions. The agent can use this information to find subsequent stages through each of
the three towns neighbouring Arad, and so on, until it determines a path to Bucharest.
Once a solution is found, the execution phase carries out the recommended solution.

1.2 Formulating Problems

1.2.1 Problem Types
Assume a vacuum-cleaner world described as follows:
• The world comprises just two locations
• Each location may or may not contain dirt
• The agent may only be at one location or the other
• The agent has three possible actions (Move Left, Move Right, Suck up dirt)
There are eight possible world states as illustrated in Figure 2. An action causes a transition from one state to
another. The goal of the problem is to clean up all the dirt, i.e., the goal is equivalent to the state set {7, 8} (we
assume that sucking of the dirt is 100% effective).

Figure 2: The eight possible states of a simple vacuum cleaner world

Page 2 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

First, assume that the environment is accessible, i.e., the agent’s
sensors give it enough information to tell it exactly what state it is
in. Then, the agent can calculate exactly which state it will be in Percept Sequence Action
after any sequence of actions. This is the simplest case and is called [A, Clean] Right
a single-state problem. For example, given the percept sequences
on the left of the table on the right, the agent will be expected to [A, Dirty] Suck
take the corresponding actions on the right column of the table. If [B, Clean] Left
the current state of the agent is 5 for example, then the action
sequence [right, suck] will take the agent to goal state 8. [B, Dirty] Suck
[A, Clean][A, Clean] Right
Assume next that the agent knows all the effects of its actions but
has only limited access to its environment. In the extreme case [A, Clean][A, Dirty] Suck
where the agent has no sensors, all it knows is that its initial state is ... ...
one of the 8 states. In such a case, the agent must reason about its
environment. This is known as a multi-state problem. For example, [A, Clean][A, Clean][A, Clean] Right
the agent may calculate that the action Right will always take it to [A, Clean][A, Clean][A, Dirty] Suck
one of the states {2, 4, 6, 8}. Or, it may discover that the action
sequence [Right, Suck, Left, Suck] will always lead to a goal state.
Next consider the case where the Suck action sometimes deposits dirt on the floor, but only if there is no dirt there
already. For example, if the agent is in State 5 and it sucks, it will reach one of the states {1, 5}.
Suppose that the agent has a position sensor and a local dirt sensor, i.e., the agent is capable of determining its
current position and whether or not that position has dirt, but incapable of telling if some other position has dirt.
Hence if the agent determines from the sensors that it is in the left cell and that that cell is dirty, then the agent is
in one of the states {1, 3}. Suppose the agent formulates the action sequence [Suck, Right, Suck]. The first action
(Suck) would lead to one of {5, 7}. The second action (Right) would lead to one of {6, 8} and the third action to
{8, 6} (success, failure). Clearly, there is no way to guarantee the correct solution. The last action ought to have
been Suck only if there is dirt there, that is, the solution requires sensing during execution. The agent must now
calculate a tree of actions, not just a single action sequence. This problem is called a contingency problem because
each branch of the tree deals with a possible contingency (i.e., unforeseen event) that may arise. By the way, this
is the same reason most people keep their eyes open while walking. Contingency problems require much more
complex search algorithms than single-state and multiple-state problems.
Our last example concerns an agent that has no information about the effects of its actions (not unlike being lost in
a strange country with no map to guide you). The agent must experiment, gradually discovering what its actions
do, and what states exist. The agent survives in such an environment by learning a map of the environment. This
kind of problem is referred to as an exploration problem.

1.2.2 Well-Defined Problems and Solutions

We start by considering the information needed to define a single-state problem. Then we show that to deal with
multiple-state problems, only minor changes are required to the single-state problem. The example in Section
1.3.1.2 will concretise the discussion in this section.
Single-State Problems
Initial State. This is the state that the agent knows itself to be in. The goal of the agent is to move from the initial
state to a goal state.
Operator. This term is used to denote the description of an action in terms of which state will be reached by
carrying out the action in a particular state.
Successor Function. This is an alternative formulation to Operator. Given a particular state x, successor function
S(x) returns the set of states reachable from x by any single action.
State Space. The state space of a problem is the set of all states reachable from the initial state by any sequence of
actions.
Path. A path in the state space is any sequence of actions leading from one state to another

Page 3 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

Goal Test. This is a test that the agent can apply to a state description to determine if it is a goal state.
Path Cost. A path cost function assigns a cost (the path cost) to each path. This cost is equal to the sum of the
costs of the individual actions along the path.
Solution. A search algorithm takes as input instances of the initial state, operator set, goal test and path cost
function, and outputs a solution, i.e., a path from the initial state to a state that satisfies the goal test.
Multi-State Problems
A multi-state problem consists of an initial state set (not just a single state) as in single state problems; a set of
operators specifying for each action the set of states reached from any given state; and a goal test and path cost
function. When an operator is applied to a state set, the result obtained is the union of the results that are obtained
by applying the operator to each state in the set.
A path in multi-state problems connects state sets (not simply states), and a solution is a path that leads to a set of
goal states. Finally, the state space in single-state problems is replaced by a state set space.

1.2.3 Measuring Problem-Solving Performance

There are at least 3 ways to measure the effectiveness of a search.
• Does it find a solution?
• Is the path cost acceptable? Path cost is the online cost associated with individual actions in the path
• What is the search cost associated with the time and memory required to find a solution? The search cost,
also called offline cost, is the cost involved before the agent interacts with the environment.
In the route-finding example in Section 1.1 for example, the path cost might be the number of kilometres travelled
coupled with a cost of wear and tear of the car on different road surfaces. In a static environment, the search cost
will be zero; if the traveller must get to Bucharest urgently, the environment is semi-dynamic because deliberating
longer for the best solution becomes a costly proposition and the search cost varies with the computation time.

1.2.4 Choosing States and Actions

There are usually too many details that can be included when the states and actions involved in problem solving
are defined. Whilst these details do not lead to a wrong solution, they add considerably to the computation costs
(time and memory). The art of problem solving is deciding what goes into the description of states and operators,
and what is left out. The process of removing detail from a representation is known as abstraction.
Consider again the example in Section 1.1 (travelling to Bucharest). The state of the world includes many things
like travelling companions, how fast the vehicle is going, what is on the radio, etc. Because these considerations
are irrelevant to the problem of finding the best route to Bucharest, they are left out.

1.3 Example Problems

We consider a few toy and real world problem examples. Toy problems are intended to illustrate or exercise
various problem-solving methods. They can be given a concise and exact description and are usually used to
compare the performance of algorithms. Real-world problems on the other hand, tend to be less precise in their
description. They are usually more difficult, and people do care about their solutions.

1.3.1 Toy Problems

1.3.1.1 8-Puzzle
An instance of this problem is shown in Figure 3. The puzzle comprises a 3 × 3 board with eight numbered tiles
and a blank space. A tile adjacent to the blank space can slide into the space. The aim is to reach the goal state on
the right of the figure.

Page 4 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

 Figure 3: Instance of 8-puzzle problem

This problem can be greatly simplified by noting that it is easier to use operators such as: the blank space changes
position with the tile to its left (see Figure 4), instead of using operators such as: move the 4 tile into the blank
space. The reason for this is that there are fewer of the former type of operator, and so the state space for the
problem would be much more manageable.

Figure 4: Illustrating movement of blank

1.3.1.2 Vacuum World

Figure 5 shows the state space of the single-state case of the simplified vacuum world that was introduced in
Section 1.2.1. The diagram shows the eight states of the problem as well as the actions and transitions that take
place when an action is taken in each state.
Figure 6 shows the state set space for the vacuum robot for the multiple-state case. The start state set (top centre)
is the set of all states because the agent has no sensors. A solution to the problem is any sequence leading from the
start state set to a set of states with no dirt.

Page 5 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

 Figure 5: Diagram of simplified vacuum state space. Arcs denote actions. L = move left, R = move right

Page 6 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

Figure 6: State set space for the multi-state case of the simple vacuum world problem. Each dashed box encloses a set of states. At
any given point the agent is within a state set but does not know which state of that set it is in. The initial state set (complete
ignorance) is the top centre box. Actions are represented by labelled arcs.

1.3.1.3 Missionaries and Cannibals

Three missionaries and three cannibals come to a river and find a boat that holds two. If the cannibals ever
outnumber the missionaries on either bank, the missionaries will be eaten.
How shall they cross? Note that the boat cannot cross the river by itself with nobody on board.
In this problem, a state may be represented as a triple comprising the numbers of boats, missionaries, and
cannibals on the starting bank of the river. Alternatively, we could use M to represent a missionary, C to represent
a cannibal, and b to represent a boat. The initial state is 133, and the goal state is 000. The possible operators are to
take either one missionary, one cannibal, two missionaries, two cannibals, or one of each across the in the boat.
The path cost is the number of crossings the boat makes. One solution to the problem is given by the sequence
(133, 022, 132, 030, 131, 011, 122, 020, 103, 010, 102, 000), illustrated further below.

Page 7 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

One solution, where the three numbers represent the number of missionaries, cannibals, and boats on the near
bank, is as follows:

1.3.1.4 Towers of Hanoi

The Towers of Hanoi game is played with three pegs, and a pile of disks of different sizes. The game begins with
the disks stacked on one peg as shown below. The aim of the game is to move the stack of disks from Peg A to
Peg C while obeying the following rules:
• Only one disk can be moved at a time
• A larger disk can never go on top of a smaller one.

The state space for this problem in the case where there are three disks is given in Figure 7.

Page 8 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

 Figure 7: State space for Tower of Hanoi problem with 3 disks

1.3.2 Real World Problem Examples

1.3.2.1 Route Finding

Route finding is defined in terms of locations and transitions along arcs between them. Application areas include:
routing in computer networks, automated travel advisory systems and airline travel planning.

1.3.2.2 Touring and Travelling Salesperson Problems

Consider the problem of visiting every city in Figure 1 at least once, starting and ending in a city, say Bucharest.
This is an instance of the touring problem. Here, the agent not only keeps track of the current city, but also of the
set of cities that the agent has visited. The initial state for this problem would be In Bucharest; visited {Bucharest}.
An intermediate state may be In Sibiu; visited {Bucharest, Fagaras, Sibiu}. The goal test would check if the agent
is in Bucharest and has visited all the cities.
The travelling salesperson problem (TSP) is a touring problem in which each city must be visited exactly once
with the objective of finding out if there exists a tour that costs less than some constant. Applications range from
planning trips for travelling salespersons to planning movements of automatic circuit board drills.

1.3.2.3 Robot Navigation

Robot navigation is a generalization of the route-finding problem (think of the robot the US sent to explore the
planet Mars). Here, the robot is not restricted to a discrete set of routes; rather, it can move in continuous space
with an infinite set of possible actions and states. On a flat surface, the space is two-dimensional; but with legs,
arms, and obstacles to avoid, the space is multidimensional.

Page 9 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

1.4 Search
Finding the solution to a problem involves a search through the state space. This can be done by maintaining and
extending a set of partial solution sequences.
Consider the route finding problem from Arad to Bucharest (Figure 1). We first test if the first state (Arad) is the
goal state. It is not, so we generate other states by applying the operators to expand the current state. In this
example, three new states Sibiu, Timisoara, and Zerind are generated.
We now have to choose one of the new states and put the others aside for later, in case the first choice does not
lead to a solution. Suppose we choose Sibiu. We test to see if it is the goal state. It is not, so we expand it to get
Arad, Fagaras, Oradea, and Rimnicu Vilcea. We can at this stage choose any of these four states for expansion, or
go back to consider Timisoara and Zerind. We continue choosing, testing, and expanding until a solution is found,
or until no more states can be expanded. The choice of which state to expand first is determined by the search
strategy. Search strategies can be evaluated in terms of the following criteria:
Completeness: Is the strategy guaranteed to find a solution when there is one?
Time complexity: How long does it take to find a solution?
Space complexity: How much memory does it need to perform the search?
Optimality: Does the strategy find the highest quality solution when there are several different solutions?
The search process can be thought of as building a search tree superimposed on the state space. The root of the
search tree is the initial state and leaf nodes correspond to states that have no successors either because they have
not been expanded yet or because they generated an empty set when they were expanded. For the route-finding
problem example we are considering, there is an infinite number of paths in the set space (example, circular
paths), and the tree has an infinite number of nodes. Figure 8 shows some of the expansions in the search tree for
route-finding from Arad to Bucharest.

Figure 8: Partial search tree for route finding from Arad to Bucharest

How can the nodes of the search tree be represented? One way is to use the following five components:
• The state in the state space to which the node corresponds
• The node in the search tree that generated this node (i.e., the parent node)
• The operator that was applied to generate the node
• The number of nodes on the path from the root to this node, i.e., the depth of the node
• The path cost of the path from the initial node to this node
The collection of nodes that are waiting to be expanded is called the fringe or frontier. This collection of nodes
can be stored in a queue; the search strategy is a function that selects the next node to be expanded from this
collection of nodes.

Page 10 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

1.4.1 Uninformed Search Strategies
Uninformed search, (also called blind search, or brute-force search) means that there is no information about the
number of steps or the path cost from the current state to the goal. This is in contrast to informed or heuristic
search, which means that the search function has some information that may help, but is not guaranteed, to guide
it towards the goal state. Uninformed search strategies are distinguished by the order in which nodes are
expanded.

1.4.1.1 Breadth-first Search

Using this search strategy which is illustrated in Figure 9 for a binary tree, the root node is expanded first, then all
the nodes generated by the root node are expanded next, and then their successors, and so on. In general, all the
nodes at depth d are expanded before all the nodes at depth d + 1. The strategy can be implemented by putting all
the generated nodes at the end of a queue data structure (i.e., using a FIFO queue).

Figure 9: Breadth-first search after 0, 1, 2, 3 node expansions

Breadth-first search is guaranteed to find a solution if it exists (i.e., the algorithm is complete). If several solutions
exist, the shallowest ones are found first. If the path cost is a non-decreasing function of the depth of the node, the
solution is optimal.
Breadth-first search has some very serious
drawbacks though, related to the time and
space it takes to complete the search.
Consider for example, a situation with a
branching factor of b, i.e., where every
state can be expanded to yield b new
states. So, b nodes are generated at the first
level, each of the b nodes generates b other
nodes at the second level for a total of b 2
nodes generated at the second level, b3
nodes generated at the third level, etc. This
gives a maximum of: 1 + b + b 2 +…+bd
nodes expanded for a tree of path height d.
The time complexity of the algorithm is
therefore exponential (i.e., O(bd)). The Figure 10: Illustrating time and space complexities of breadth-first search
space complexity is similarly O(bd)
because all the leaf nodes must be maintained in memory at the same time. Assuming a branching factor of 10 and
that it takes an average of 1 ms to expand one node, and that each node requires 100 bytes of storage, Figure 10
demonstrates the explosion in time and space complexity of the algorithm.
We notice from Figure 10 that execution time is a major problem with this algorithm for solutions that require a
search tree with a large depth. And as bad as execution time is, memory requirement is an even bigger problem.
While many people can wait 18 minutes for an important solution, those of them with low-end computers do not
have sufficient RAM to run the algorithm. And while some users can even wait up to 31 hours to get a really
important solution, very few users can have the luxury of a computer with 11 GB of RAM.

Page 11 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

Example

Figure 11: Breadth-first search example, and corresponding FIFO queue evolution
Order of generation of queue: ABCDEFGHIJKLMNOPQRSTUV
Order of expansion of nodes: ABCDEFGHIJKLMNOPQRSTUV

1.4.1.2 Depth-first Search

This search strategy always expands one of the nodes at the deepest level of the tree, and only when the search hits
a non-goal node with no expansion does the search go back and expand nodes at shallower levels. The strategy can
be implemented by putting all the generated nodes at the front of a stack (i.e., using a LIFO data structure). Depth-
first search is illustrated in Figure 12.

Figure 12: Depth-first search trees for left subtree of a binary tree rooted at Node 1

The memory requirements of depth-first search are modest. The algorithm only need store a single path from the
root to a leaf node, along with the remaining unexpanded sibling nodes for each node on the path. If the branching
factor is b and the maximum depth is m, depth-first search requires storage of bm nodes (cf. minimum b d required
for breadth-first search). Using the same assumptions as in Figure 11, we see that for a depth of 12, the space
requirement for depth-first search is 12 kilobytes (cf. 111 terabytes for breadth-first search!).
The time complexity of depth-first search for a maximum depth of m is O(b m) in the worst case. But the algorithm
can be faster than breadth-first search in problems that have many solutions because there is a good chance of

Page 12 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

finding a solution after exploring only a small portion of the space. Breadth-first search would still have to search
all paths of length d-1 before considering any of length d.
The problem with depth-first search is related to state spaces with very deep and perhaps infinite search trees. In
such cases, the algorithm can get stuck in the wrong path if an unlucky choice is made of the next node to expand.
So depth-first search is not complete. It is also not optimal because a solution can be found deep down the tree
before another one at a shallower level if the path to the former was considered first.

Example

Figure 13: Depth-first search example, and corresponding LIFO queue evolution
Order of generation of stack: ABCDEFKLMGHINORSTVJPQU
Order of expansion of nodes: ABEKLFMCGHNORSVTIDJPQU

1.4.1.3 Depth-limited Search

This search strategy avoids the problems of depth-first search by imposing a cut-off on the maximum depth of a
path. In the Map of Romania with 20 cities for example, a maximum path length of 19 can be imposed. With this
cut-off value, the algorithm is complete but still not optimal. If the cut-off is too small, the algorithm will not be
complete. Space and time complexities are similar to depth-first search.

1.4.1.4 Iterative Deepening Search

The difficulty in depth-limited search is deciding an appropriate cut-off limit. If we study the map example more
carefully, we notice that any city can be reached from any other city in at most 9 steps. This is known as the
diameter of the state space, and gives us a better depth limit, and more efficient search. The problem here though,
is that it is not easy to determine this limit before the problem is solved. Iterative deepening search overcomes this
problem by trying all possible depth limits starting with 0, then 1, then 2, etc.
Iterative deepening combines the benefits of breadth-first search and depth-first search. It is complete and optimal
like breadth-first search, and at the same time, has only slightly higher memory requirements than depth-first
search.

Page 13 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

 Figure 14: Illustrating the iterative deepening search strategy (4 iterations)

The order of expansion of nodes is illustrated in Figure 14. The order of expansion of states is similar to breadth-
first, except that some states are expanded multiple times. Expanding some states multiple times may seem too
wasteful, but as we illustrate below, that is not really the case.
Total # of expansions in breadth-first search to depth d with branching factor b is
1 + b + b2 + b3 +…+ bd-2 + bd-1 + bd
If b = 10 and d = 5, this gives 1 + 10 + 100 + 1,000 + 10,000 + 100,000 = 111,111
In iterative deepening search, the nodes at the bottom level are expanded once, those at the next level up expanded
twice, etc., up to the root node which is expanded d + 1 times. The total number of expansions is thus
(d + 1)1 + (d)b + (d – 1)b2 +…+ 3bd-2 + 2bd-1 + 1bd
If b = 10 and d = 5, this gives 6 + 50 + 400 + 3,000 + 20,000 + 100,000 = 123,456, only about 11% more
expansions than breadth-first or depth-first search.
The time complexity of iterative deepening is still O(b d) as in the strategies we saw before, and the space
complexity O(bd) as in depth-first search.

1.4.1.5 Uniform Cost Search

Uniform cost search improves on breadth-first search by attempting to find the least-cost solution; the node on the
fringe that is expanded is always the one with the lowest-cost. Uniform cost search finds the cheapest solution
provided the cost of a path never decreases as we go along the path. The algorithm is illustrated in Figure 15. The
problem is to go from S to G. Node S is expanded first yielding the paths to A, B, C. Node A is expanded next
because its cost is cheapest. This results to a path SAG to the goal at a cost of 11. Because this cost is greater than
an existing path, viz., SB, whose cost is 5, the path SAG is put on the queue below path SB, and not recognized
(yet) as the optimal path. Path SB is expanded next, and this leads to path SBG with a path cost of 10. This is the
new cheapest path remaining in the queue, and because it leads to the goal state, it is returned as the solution.

Page 14 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

 Figure 15: Illustrating uniform cost search. (a) state space showing cost of each operator; (b) Progression of search. Each
node is labelled with the cost. At the next step, the goal node with cost = 10 will be selected

1.4.2 Informed Search Methods

The uninformed search strategies which find solutions by systematically generating new states and testing them
against the goal state are usually inefficient in the sense that several unnecessary searches are made. Informed
search strategies improve efficiency by using problem-specific knowledge to find a solution.

1.4.2.1 Best-First Search Strategies

Best-first search strategies use an evaluation function to arrange the nodes to expand next in a queue, such that the
node that the function determines to be the most desirable to expand next is at the front of the queue. The
evaluation function focuses the search by estimating the cost of the path from a state to the closest goal state.
Many times the evaluation function works well; sometimes though, it may lead the search astray. We consider two
best-first search strategies: greedy search tries to expand the node that is closest to the goal, while A* search tries
to expand the node on the least-cost solution path.

1.4.2.2 Greedy Search

Greedy search minimizes the estimated cost to reach a goal by first expanding the node whose state is judged to be
closest to the goal state. A heuristic function h is used to estimate the cost to the goal.
h(n) = estimated cost of the cheapest path from the state at node n to a goal state.
The function h can be any function provided h(n) = 0 if n is a goal state. To illustrate greedy search, we consider
again the map of Romania, repeated in Figure 16, but this time, with the straight line distances from each town to
Bucharest shown. We use the straight line distance heuristic1, i.e.,
hSLD(n) = straight line distance between n and the goal location (Bucharest in this example).
Note that hSLD is only useful because a road from A to B usually tends to head in the straight line direction between
the two points.

1
The word heuristic means to find or to discover. A heuristic for a problem is a process that may solve a given problem but offers no
guarantees of doing so. In the area of search algorithms, it refers to a function that provides an estimate of the cost of a solution.

Page 15 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

 Figure 16: Map of Romania with road distances and straight-line distances to Bucharest

Figure 17 shows the stages of the algorithm from Arad to Bucharest. As before, the Arad node is expanded to
Sibiu, Timisoara, and Zerind. The next node to be expanded is Sibiu, since it has the shortest straight line distance
to the goal. This is followed by Fagaras, which leads to Bucharest.

Figure 17: Stages in a greedy search from Arad to Bucharest using the hSLD heuristic function

In this example, the algorithm leads to a minimal search cost as the goal state is reached without ever expanding a
node that is not on the solution path. Notice however that the solution is not optimal as a lower cost path: Sibiu,
Rimnicu Vilcea, Pitesi, Bucharest, can be found (418 km vs. 450 km). This path was not found because Fagaras
has a shorter straight line distance to Bucharest than Rimnicu Vilcea. The algorithm prefers to take the biggest bite
towards the goal without worrying about whether it will be the best in the long run, hence the name greedy.

Page 16 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

Greedy search tends to find a solution quickly. It is susceptible to false starts though. For example, moving from
Iasi to Fagaras, the algorithm first considers Neamt, which is a dead end. The algorithm is incomplete and is not
optimal, as it may also get bogged down an infinite path if repeated states are not detected.
The worst case time complexity is O(bm), where m is the maximum depth of the search space. The space
complexity is also O(bm) as all the nodes are retained in memory.

1.4.2.3 A* Search
As we saw in Section 1.4.1.5, uniform cost search minimizes the cost g(n) of the path so far. It is also complete
and optimal, but it can be very inefficient because of the large number of nodes that may need to be expanded to
determine their associated costs. We also saw in Section 1.4.2.2 that greedy search can cut the estimated cost h(n)
to a goal significantly, but is unfortunately neither complete nor optimal.
A* search aims at combining the strategies used in greedy search and uniform cost search to capitalize on the
advantages of each by simply combining their evaluation functions: f(n) = g(n) + h(n).
Because g(n) is the path cost from the start node to node n, and h(n) is the estimated cost of the cheapest path from
node n to the goal, f(n) effectively is an estimate of the cost of the cheapest solution through node n to the goal.
It can be proven that if function h never overestimates the cost to reach the goal, A* is complete and optimal. In
such a case, h is an admissible heuristics, and like all admissible heuristics, is optimistic in the sense that it
assumes that the cost of solving the problem is actually less than it actually is. If h is admissible, then f(n) never
overestimates the actual cost of the best solution through n.
Figure 18 illustrates the behaviour of the A* algorithm. We note that along any path from the root, the f-cost never
decreases. This means that the f exhibits monotonicity.

Page 17 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

Figure 18: Stages in an A* search for Bucharest. Nodes are labelled with f=g+h. The h values are the straight-line distances to
Bucharest taken from Figure 16

1.5 Introduction to Game Playing

In game playing, we plan ahead in a world that includes a hostile agent. Games have exercised human intellectual
faculties for a long time, perhaps in part because they offer competition without the need to muster up two armies
and going to war to determine which side is stronger. The kinds of games that have traditionally attracted the
attention of AI researchers are those whose states are easy to represent, and agents are restricted to a small number
of well defined actions.
Normally in game playing, the environment is fully accessible (i.e., the agent can perceive everything that needs to
be known about its environment), and the rules are simple. Hence, the game states can be completely represented
on a computer, and so all that the player needs to do is proceed along the best path to the solution (i.e., do a search
in the search space). But the presence of an opponent makes the decision process much more complicated because
the opponent introduces uncertainty, since one never knows what move the opponent would make. Then there is
the added difficulty that games usually have a large branching factor (about 35 or more), and each player makes
several moves, perhaps 50 or more (i.e., depth of 50 or more). This leads to a search tree for the two players with
about 35100 nodes – although only about 1040 of them may represent legal positions.
With so many nodes to consider, there is not enough time to compute the consequences of the different moves that
may be made. Hence, the player has to make a best guess based on past experience, and act before s/he knows the
best action to take. And the player must act within a time limit to avoid penalties that would cause them to lose the
game.

Page 18 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

1.5.1 Perfect Decisions in Two-Person Games
We consider a board game with two players (MIN and MAX). MAX moves first, then MIN, and so on, until the
game is over. At the end of the game, points are awarded to the winning player (or deducted from the losing
player). A game is a kind of search problem with the following components:
• Initial state. Includes the board position and an indication of whose turn it is to move
• A set of operators. Defines the legal moves that a player can make
• A terminal test. Determines when the game is over; states where the game has ended are called terminal
states
• A utility function. Gives a numeric value for the outcome of a game, e.g., win=+1, loss=-1, draw=0 in some
games, or a much wider range of numbers in some other games
Because MAX cannot simply search for the sequence of moves that leads to a terminal state that is a winner (since
MIN has something to say about it), MAX must find a strategy (comprising the correct move for MAX for every
possible move by MIN) that will lead to a winning terminal state, irrespective of what MIN does.
Figure 19 shows part of the search tree for the game Tic-Tac-Toe. MAX starts by placing an x in one of the nine
squares, then MIN places a o in one of the remaining squares, and the two keep taking turns until either the game
ends in a draw, or the winner achieves a row or column or diagonal of three squares with either x or o.
We note that the search tree of even a game as simple as Tic-Tac-Toe is too large to be represented on the page

Page 19 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

Figure 19: A partial search tree for the game Tic-Tac-Toe. The top node is the initial state, and MAX moves first, placing an x in
some square. Part of the search tree is shown, giving alternative moves by MIN (o) and MAX, until we eventually reach terminal
states, which can be assigned utilities according to the rules of the game

Figure 20 shows the search tree of a much more trivial game than Tic-Tac-Toe. The possible moves for MAX are
A1, A2, A3, and the possible replies for MIN are A11, A12, A13, etc. This simple game ends after one move each
by MAX and MIN, i.e., the game is one move deep, consisting of two half-moves or two ply. The utilities
computed at the terminal states of the game range from 2 to 14.

Figure 20: A 2-ply game tree generated by minimax algorithm. The D nodes are moves by MAX, and the Ñ nodes moves by MIN.
Terminal nodes show utility value for MAX computed by the rules of the game (i.e., utility function); the utilities of the other nodes
are computed by the minimax algorithm from the utilities of their successors. MAX’s best move is A 1, and MIN’s best reply is A11

Page 20 Denis L. Nkweteyim CSC 405 – Artificial Intelligence

The minmax algorithm is designed to determine the optimal strategy for MAX, and thus decide the best possible
first move for MAX.
Steps of the algorithm
• Generate the whole tree
• Apply the utility function to each terminal state to get its value
• Use the utility of the terminal states to determine the utility of the nodes one level higher up the search
tree. Example
◦ Consider the left-most three leaf nodes
◦ In the Ñ node above them, the best MIN can do is choose A11, since that would lead to the minimum
outcome for MAX. Hence, we assign a utility value of 3 to that node under the assumption that MIN
will do the right thing.
◦ The same argument holds for the other two D nodes
• At the top level D node, the utility value stated is 3 to say that MAX will do the best thing by making a
move that will lead to the best possible outcome. This is known as the minmax decision.

Page 21 Denis L. Nkweteyim CSC 405 – Artificial Intelligence