Topic - 6 (Game Playing).ppt

CSE 412: Artificial Intelligence
Department of CSE
Daffodil International University
Topic – 5: Game Playing

Case Studies: Playing Grandmaster Chess
Game Playing as Search
Optimal Decisions in Games
 Greedy search algorithm
 Minimax algorithm
Alpha-Beta Pruning
Topic Contents

• The real success of AI in game-playing was achieved much
later after many years’ effort.
• It has been shown that this search based approach works
extremely well.
• In 1996 IBM Deep Blue beat Gary Kasparov for the first time.
and in 1997 an upgraded version won an entire match against
the same opponent.
Case Studies: Playing
Grandmaster Chess
3

4
Case Studies: Playing
Grandmaster Chess…
Kasparov vs. Deep Blue, May 1997
• 6 game full-regulation match sponsored by ACM
• Kasparov lost the match 1 wins to 2 wins and 3 tie
• This was a historic achievement for computer chess
being the first time a computer became
the best chess player on the planet.
• Note that Deep Blue plays by “brute force” (i.e. raw
power from computer speed and memory). It uses
relatively little that is similar to human intuition and
cleverness.

5
Game Playing and AI
Why would game playing be a good problem for AI
research?
– game playing is non-trivial
• players need “human-like” intelligence
• games can be very complex (e.g. chess, go)
• requires decision making within limited time
– game playing can be usefully viewed as a search problem in
a space defined by a fixed set of rules
• Nodes are either white or black corresponding to reflect the
adversaries’ turns.
• The tree of possible moves can be searched for favourable positions.

6
Game Playing and AI…
Why would game playing be a good problem for AI
research?
– games often are:
• well-defined and repeatable
• easy to represent
• fully observable and limited environments
– can directly compare humans and computers

7
Game Playing as Search
• Consider two-player, turn-taking, board
games
– e.g., tic-tac-toe, checkers, chess
• Representing these as search problem:
– states: board configurations
– edges: legal moves
– initial state:start board configuration
– goal state: winning/terminal board configuration

8
Game Playing as Search:
Game Tree
X X X
X
…
O
X O
X
O
X
O
X
…
How can this be handled?
What's the new aspect
to the search problem?
There’s an opponent
that we cannot control!
X
O
X
X O
X
X O
X
O
X X
…
O
X
X

9
Game Playing as Search:
Complexity
• Assume the opponent’s moves can be
predicted given the computer's moves.
• How complex would search be in this case?
– worst case: O(bd) ; b is branching factor, d is depth
– Tic-Tac-Toe: ~5 legal moves, 9 moves max game
• 59 = 1,953,125 states
– Chess: ~35 legal moves, ~100 moves per game
• 35100 ~10154 states, but only ~1040 legal states
Common games produce enormous search trees.

10
Greedy Search Game Playing
A utility function maps each terminal state of the
board to a numeric value corresponding to the
value of that state to the computer.
– positive for winning, > + means better for computer
– negative for losing, > - means better for opponent
– zero for a draw
– typical values (lost to win):
• -infinity to +infinity
• -1.0 to +1.0

12
E
D
B C E
3
D
2
B
-5
C
9
• Expand each branch to the terminal states
• Evaluate the utility of each terminal state
• Choose the move that results in the board
configuration with the maximum value
M N O
K L
F G H I J
computer's
possible moves
opponent's
possible moves
board evaluation from computer's perspective
A
terminal states
A
9
M
1
N
3
O
2
K
0
L
2
F
-7
G
-5
H
3
I
9
J
-6

13
Assuming a reasonable search space,
what's the problem with greedy search?
It ignores what the opponent might do!
e.g. Computer chooses C. Opponent
chooses J and defeats computer.
M
1
N
3
O
2
K
0
L
2
F
-7
G
-5
H
3
I
9
J
-6
E
3
D
2
B
-5
C
9
computer's
possible moves
opponent's
possible moves
board evaluation from computer's perspective
A
9
terminal states

14
Minimax: Idea
• Assuming the worst (i.e. opponent plays
optimally):
given there are two plays till the terminal states
– If high utility numbers favor the computer
Computer should choose which moves?
maximizing moves
– If low utility numbers favor the opponent
Smart opponent chooses which moves?
minimizing moves

15
E
D
B C
A
• The computer assumes after it moves the
opponent will choose the minimizing move.
E
1
D
0
B
-7
C
-6
A
1
M
1
N
3
O
2
K
0
L
2
F
-7
G
-5
H
3
I
9
J
-6
computer's
possible moves
opponent's
possible moves
board evaluation from computer's perspective terminal states
Minimax: Idea
• It chooses its best move considering
both its move and opponent’s best move.

16
• Explore the game tree to the terminal states
• Evaluate the utility of the terminal states
• Computer chooses the move to put the board
in the best configuration for it assuming
the opponent makes best moves on her
turns:
– start at the leaves
– assign value to the parent node as follows
• use minimum of children when opponent’s moves
• use maximum of children when computer's moves
Minimax: Passing Values up
Game Tree

17
E
D
0
B C
R
0
N
4 O P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
F G
-5
H
3
I
8 J L
2
W
-3
X
-5
A
Deeper Game Trees
• Minimax can be generalized for > 2 moves
• Values backed up in minimax way
terminal states
O
-5
K
5
M
-7
F
4
J
9
E
-7
B
-5
C
3
A
3
opponent
min
computer
max
opponent
min
computer max

18
Minimax: Direct Algorithm
For each move by the computer:
1. Perform depth-first search to a terminal state
2. Evaluate each terminal state
3. Propagate upwards the minimax values
if opponent's move minimum value of children backed up
if computer's move maximum value of children backed up
4. choose move with the maximum of minimax values of
children
Note:
• minimax values gradually propagate upwards as DFS
proceeds: i.e., minimax values propagate up
in “left-to-right” fashion
• minimax values for sub-tree backed up “as we go”,
so only O(bd) nodes need to be kept in memory at any
time

19
Minimax: Algorithm Complexity
Assume all terminal states are at depth d
Space complexity?
depth-first search, so O(bd)
Time complexity?
given branching factor b, so O(bd)
Time complexity is a major problem!
Computer typically only has a
finite amount of time to make a move.

20
Minimax: Algorithm Complexity
• Direct minimax algorithm is impractical in practice
– instead do depth-limited search to ply (depth) m
What’s the problem for stopping at any ply?
– evaluation defined only for terminal states
– we need to know the value of non-terminal states
Static board evaluator (SBE) function uses heuristics
to estimate the value of non-terminal states.
It was first proposed by Shannon in his paper,
Programming a computer for playing chess, in 1950.

21
Minimax:
Static Board Evaluator (SBE)
A static board evaluation function estimates how
good a board configuration is for the computer.
– it reflects the computer’s chances of winning from that
state
– it must be easy to calculate from the board configuration
• For Example, Chess:
SBE = α * materialBalance + β * centerControl + γ * …
material balance = Value of white pieces - Value of black pieces
(pawn = 1, rook = 5, queen = 9, etc).

22
Minimax:
• How to design good static board evaluator functions?
First, the evaluation function should order the
terminal states in the same way as the true utility
function; otherwise, an agent using it might select
suboptimal moves even if it can see ahead all the
way to the end of the game.
Second, the computation must not take too long!
 Third, for nonterminal states, the evaluation
function should be strongly correlated with the actual
chances of winning.

• Evaluation function is a heuristic function, and it is where the
domain experts’ knowledge resides.
• Example of an evaluation function for Tic-Tac-Toe:
f(n) = [# of 3-lengths open for me] - [# of 3-lengths open for you],
where a 3-length is a complete row, column, or diagonal.
• Alan Turing’s function for chess
– f(n) = w(n)/b(n), where w(n) = sum of the point value of white’s pieces
and b(n) is sum for black.
• Most evaluation functions are specified as a weighted sum of
position features:
f(n) = w1*feat1(n) + w2*feat2(n) + ... + wn*featk(n)
• Example features for chess are piece count, piece placement,
squares controlled, etc.
• Deep Blue has about 6,000 features in its evaluation function.
Minimax:

24
int minimax (Node s, int depth, int limit) {
if (isTerminal(s) || depth == limit) //base case
return(staticEvaluation(s));
else {
Vector v = new Vector();
//do minimax on successors of s and save their values
while (s.hasMoreSuccessors())
v.addElement(minimax(s.getNextSuccessor(),
depth+1, limit));
if (isComputersTurn(s))
return maxOf(v); //computer's move returns max of
kids
else
return minOf(v); //opponent's move returns min of
kids
}
}
Minimax: Algorithm with SBE

25
• The same as direct minimax, except
– only goes to depth m
– estimates non-terminal states using SBE function
• How would this algorithm perform at chess?
– if could look ahead ~4 pairs of moves (i.e. 8 ply)
would be consistently beaten by average players
– if could look ahead ~8 pairs as done in typical
pc, is as good as human master
Minimax: Algorithm with SBE

26
• Can't minimax search to the end of the
game.
– if could, then choosing move is easy
• SBE isn't perfect at estimating.
– if it was, just choose best move without
searching
Recap

27
Alpha-Beta Pruning Idea
• Some of the branches of the game tree won't be
taken if playing against a smart opponent.
• Use pruning to ignore those branches.
• While doing DFS of game tree, keep track of:
– alpha at maximizing levels (computer’s move)
• highest SBE value seen so far (initialize to -infinity)
• is lower bound on state's evaluation
– beta at minimizing levels (opponent’s move)
• lowest SBE value seen so far (initialize to +infinity)
• is higher bound on state's evaluation

28
Alpha-Beta Pruning Idea
• Beta cutoff pruning occurs when maximizing
if child’s alpha ≥ parent's beta
Why stop expanding children?
opponent won't allow computer to take this move
• Alpha cutoff pruning occurs when minimizing
if parent's alpha ≥ child’s beta
Why stop expanding children?
computer has a better move than this

29
A
A
A
α=-
O
W
-3
B
N
4
F G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
Alpha-Beta Search Example
minimax(A,0,4)
max
Call
Stack
A
alpha initialized to -infinity
Expand A? Yes since there are successors, no cutoff test for root

30
B
B
B
β=+
O
W
-3
N
4
F G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
Expand B? Yes since A’s alpha >= B’s beta is false, no alpha cutoff
minimax(B,1,4) beta initialized to +infinity

31
F
F
F
α=-
O
W
-3
B
β=+
N
4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
F
Expand F? Yes since F’s alpha >= B’s beta is false, no beta cutoff
minimax(F,2,4) alpha initialized to -infinity

32
O
W
-3
B
β=+
N
4
F
α=-
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
N
4
B
min
max
F
green: terminal state
N
minimax(N,3,4) evaluate and return SBE value

33
F
α=-
F
α=4
O
W
-3
B
β=+
N
4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
F
Keep expanding F? Yes since F’s alpha >= B’s beta is false, no beta cutoff
alpha = 4, since 4 >= -infinity (maximizing)
back to
minimax(F,2,4)

34
O
O
O
β=+
W
-3
B
β=+
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
F
O
min
Expand O? Yes since F’s alpha >= O’s beta is false, no alpha cutoff
minimax(O,3,4) beta initialized to +infinity

35
blue: non-terminal state (depth limit)
O
β=+
W
-3
B
β=+
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
F
O
W
-3
min
W
minimax(W,4,4) evaluate and return SBE value

36
O
β=+
O
β=-3
W
-3
B
β=+
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
F
O
min
Keep expanding O?
beta = -3, since -3 <= +infinity (minimizing)
back to
minimax(O,3,4)
No since F’s alpha >= O’s beta is true: alpha cutoff

37
red: pruned state
O
β=-3
W
-3
B
β=+
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
Why?
Smart opponent will choose W or worse, thus O's upper bound is –3.
Computer already has better move at N.
max
Call
Stack
A
B
min
max
F
O
min
X
-5

38
O
β=-3
W
-3
B
β=+
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
F
min
X
-5
back to
minimax(F,2,4)
alpha doesn’t change, since -3 < 4 (maximizing)
Keep expanding F? No since no more successors for F

39
B
β=+
B
β=4
back to
minimax(B,1,4)
O
β=-3
W
-3
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
min
X
-5
beta = 4, since 4 <= +infinity (minimizing)
Keep expanding B? Yes since A’s alpha >= B’s beta is false, no alpha cutoff

40
O
β=-3
W
-3
B
β=4
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
min
X
-5
G
G
-5
minimax(G,2,4)
evaluate and return SBE value

41
B
β=4
B
β=-5
O
β=-3
W
-3
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=-
max
Call
Stack
A
B
min
max
min
X
-5
back to
minimax(B,1,4)
beta = -5, since -5 <= 4 (minimizing)
Keep expanding B? No since no more successors for B

42
A
α=
A
α=-5
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
max
Call
Stack
A
min
max
min
X
-5
back to
minimax(A,0,4)
alpha = -5, since -5 >= -infinity (maximizing)
Keep expanding A? Yes since there are more successors, no cutoff test

43
C
C
C
β=+
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
minimax(C,1,4) beta initialized to +infinity
Expand C? Yes since A’s alpha >= C’s beta is false, no alpha cutoff

44
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=+
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=
minimax(H,2,4)
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
H
3
H

45
C
β=+
C
β=3
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
back to
minimax(C,1,4)
beta = 3, since 3 <= +infinity (minimizing)
Keep expanding C? Yes since A’s alpha >= C’s beta is false, no alpha cutoff

46
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=
minimax(I,2,4)
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
I
8
I

47
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8 J L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
back to
minimax(C,1,4)
beta doesn’t change, since 8 > 3 (minimizing)
Keep expanding C? Yes since A’s alpha >= C’s beta is false, no alpha cutoff

48
J
J
J
α=-
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
L
2
A
α=
minimax(J,2,4)
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
J
alpha initialized to -infinity
Expand J? Yes since J’s alpha >= C’s beta is false, no beta cutoff

49
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=-
L
2
A
α=
minimax(P,3,4)
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
J
P
P
9

50
J
α=-
J
α=9
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
J
back to
minimax(J,2,4)
alpha = 9, since 9 >= -infinity (maximizing)
Keep expanding J?
Q
-6
R
0
red: pruned states
No since J’s alpha >= C’s beta is true: beta cutoff

51
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=9
L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
J
Why?
Computer will choose P or better, thus J's lower bound is 9.
Smart opponent won’t let computer take move to J
(since opponent already has better move at H).
red: pruned states

52
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=9
L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=-5
C
back to
minimax(C,1,4)
beta doesn’t change, since 9 > 3 (minimizing)
Keep expanding C? No since no more successors for C

53
A
α=-5
A
α=3
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=9
L
2
max
Call
Stack
A
min
max
min
X
-5
back to
minimax(A,0,4)
alpha = 3, since 3 >= -5 (maximizing)

54
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=9
L
2
A
α=
minimax(D,1,4)
max
Call
Stack
A
min
max
min
X
-5
A
α=3
D
D
0

55
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=9
L
2
A
α=
max
Call
Stack
A
min
max
min
X
-5
A
α=3
back to
minimax(A,0,4)
alpha doesn’t change, since 0 < 3 (maximizing)

56
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K M
H
3
I
8
J
α=9
L
2
A
α=
How does the algorithm finish searching the tree?
max
Call
Stack
A
min
max
min
X
-5
A
α=3

57
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
β=2
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K
α=5 M
H
3
I
8
J
α=9
L
2
A
α=
Stop Expanding E since A's alpha >= E's beta is true: alpha cutoff
max
Call
Stack
A
min
max
min
X
-5
A
α=3
Why?
Smart opponent will choose L or worse, thus E's upper bound is 2.
Computer already has better move at C.

58
green: terminal states, red: pruned states
blue: non-terminal state (depth limit)
O
β=-3
W
-3
B
β=-5
N
4
F
α=4
G
-5
X
-5
E
β=2
D
0
C
β=3
R
0
P
9
Q
-6
S
3
T
5
U
-7
V
-9
K
α=5 M
H
3
I
8
J
α=9
L
2
A
α=
Result: Computer chooses move to C.
max
Call
Stack
A
min
max
min
X
-5
A
α=3

59
Alpha-Beta Effectiveness
Effectiveness depends on the order
in which successors are examined.
What ordering gives more effective pruning?
More effective if best successors are examined first.
• Best Case: each player’s best move is left-most
• Worst Case: ordered so that no pruning occurs
– no improvement over exhaustive search
• In practice, performance is closer
to best case rather than worst case.

60
If opponent’s best move where first
more pruning would result:
E
β=2
S
3
T
5
U
-7
V
-9
K
α=5 M
L
2
A
α=3
E
β=2
S
3
T
5
U
-7
V
-9
K M
L
2
A
α=3

61
• In practice often get O(b(d/2)) rather than O(bd)
– same as having a branching factor of sqrt(b)
recall (sqrt(b))d = b(d/2)
• For Example: chess
– goes from b ~ 35 to b ~ 6
– permits much deeper search for the same time
– makes computer chess competitive with humans

63
Other Issues: The Horizon
Effect
• Sometimes disaster lurks just
beyond search depth
– e.g. computer captures queen,
but a few moves later the opponent checkmates
• The computer has a limited horizon, it cannot
see that this significant event could happen
How do you avoid catastrophic losses
due to “short-sightedness”?
– quiescence search
– secondary search

64
Other Issues: The Horizon
Effect
• Quiescence Search
– when SBE value frequently changing,
look deeper than limit
– looking for point when game quiets down
• Secondary Search
1. find best move looking to depth d
2. look k steps beyond to verify that it still looks
good
3. if it doesn't, repeat step 2 for next best move

Topic - 6 (Game Playing).ppt

More Related Content

Similar to Topic - 6 (Game Playing).ppt(20)

Recently uploaded(20)

Topic - 6 (Game Playing).ppt