Artificial Intelligence

Lecture 7

Bicol University
1st Semester 2021-2022
Constraint Satisfaction Problem
(CSP)
Constraint satisfaction problems
 What is a CSP?
Finite set of variables V1, V2, …, Vn
Finite set of constraints C1, C2, …, Cm
Non-empty domain of possible values for each variable
DV1, DV2, … DVn
Each constraint Ci limits the values that variables can take,
e.g. V1 ≠ V2

 A state is defined as an assignment of values to

some or all variables.

 Consistent assignment: assignment does not violate

the constraints.
Constraint satisfaction problems

 An assignment is complete when every value is

mentioned.
 A solution to a CSP is a complete assignment that
satisfies all constraints.
 Some CSPs require a solution that maximizes an
objective function.

 Applications: Scheduling the time of observations on

the Hubble Space Telescope, Floor planning, Map
coloring, Cryptography
CSP example: map coloring
CSP example: map coloring

 Variables: WA, NT, Q, NSW, V, SA, T

 Domains: Di={red,green,blue}
 Constraints:adjacent regions must have different colors.
 E.g. WA NT (if the language allows this)
 E.g. (WA,NT) {(red,green),(red,blue),(green,red),…}
CSP example: map coloring

 Solutions are assignments satisfying all constraints, e.g.

{WA=red,NT=green,Q=red,NSW=green,V=red,SA=blue,T=gre
en}
Constraint graph

 CSP benefits
●
Standard representation pattern
●
Generic goal and successor functions
●
Generic heuristics (no domain specific
expertise).

 Constraint graph = nodes are variables, edges show

constraints.
 Graph can be used to simplify search.
 e.g. Tasmania is an independent subproblem.
Varieties of CSPs
 Discrete variables
Finite domains; size d 
O(dn) complete assignments.
- E.g. Boolean CSPs, include. Boolean satisfiability (NP-
complete).
Infinite domains (integers, strings, etc.)
- E.g. job scheduling, variables are start/end days for each job
- Need a constraint language e.g StartJob1 +5 ≤ StartJob3.
- Linear constraints solvable, nonlinear undecidable.
 Continuous variables
e.g. start/end times for Hubble Telescope observations.
Linear constraints solvable in poly time by LP methods.
Varieties of constraints

 Unary constraints involve a single variable.

e.g. SA green
 Binary constraints involve pairs of variables.
e.g. SA WA
 Higher-order constraints involve 3 or more variables.
e.g. cryptharithmetic column constraints.

 Preference (soft constraints) e.g. red is better than

green often representable by a cost for each
variable assignment constrained optimization
problems.
Idea 1: CSP as a standard search
problem

 A CSP can be easily expressed as a

standard search problem.

 Incremental formulation
Initial State: the empty assignment {}.
Successor function: Assign value to unassigned variable
provided that there is no conflict.
Goal test: the current assignment is complete.
Path cost: as constant cost for every step.
CSP as a standard search problem

 This is the same for all CSP’s !!!

 Solution is found at depth n (if there are n variables).
Hence depth first search can be used.
 Path is irrelevant, so complete state representation
can also be used.
 Branching factor b at the top level is nd.
 b=(n-l)d at depth l, hence n!dn leaves (only dn
complete assignments).
Commutativity

 CSPs are commutative.

The order of any given set of actions has no effect on the
outcome.
Example: choose colors for Australian territories one at
a time
- [WA=red then NT=green] same as [NT=green then WA=red]
- All CSP search algorithms consider a single variable
assignment at a time there are dn leaves.
Backtracking search

 Cfr. Depth-first search

 Chooses values for one variable at a time

and backtracks when a variable has no legal
values left to assign.

 Uninformed algorithm
No good general performance
Backtracking search

function BACKTRACKING-SEARCH(csp) return a solution or failure

return RECURSIVE-BACKTRACKING({} , csp)

function RECURSIVE-BACKTRACKING(assignment, csp) return a solution or failure

if assignment is complete then return assignment
var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)
for each value in ORDER-DOMAIN-VALUES(var, assignment, csp) do
if value is consistent with assignment according to CONSTRAINTS[csp] then
add {var=value} to assignment
result RECURSIVE-BACTRACKING(assignment, csp)
if result failure then return result
remove {var=value} from assignment
return failure
Backtracking example
Backtracking example
Backtracking example
Backtracking example
Idea 2: Improving backtracking efficiency

 General-purpose methods & heuristics can

give huge gains in speed, on average
 Heuristics:
Q: Which variable should be assigned next?
1. Most constrained variable
2. Most constraining variable

Q: In what order should that variable’s values be tried?

3. Least constraining value

Q: Can we detect inevitable failure early?

4. Forward checking
Heuristic 1: Most constrained variable

var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)

 a.k.a. Minimum remaining values (MRV)

 Rule: choose variable with the fewest legal moves
 Which variable shall we try first?
Heuristic 2: Most constraining variable

 a.k.a. degree heuristic

 Rule: select variable that is involved in the largest number of
constraints on other unassigned variables.
 Degree heuristic is very useful as a tie breaker.
 In what order should its values be tried?
Heuristic 3: Least constraining value

 Least constraining value heuristic

 Rule: given a variable choose the least constraing value i.e.
the one that leaves the maximum flexibility for subsequent
variable assignments.
Heuristic 4: Forward checking

 Can we detect inevitable failure early?

And avoid it later?
 Forward checking idea: keep track of remaining legal values
for unassigned variables.
 Terminate search when any variable has no legal values.
Forward checking

 Assign {WA=red}
 Effects on other variables connected by constraints with WA
NT can no longer be red
SA can no longer be red
Forward checking

 Assign {Q=green}
 Effects on other variables connected by constraints with WA
NT can no longer be green
NSW can no longer be green
SA can no longer be green

 MRV heuristic will automatically select NT and SA next, why?

Forward checking

 If V is assigned blue
 Effects on other variables connected by constraints with WA
SA is empty
NSW can no longer be blue
 FC has detected that partial assignment is inconsistent with the
constraints and backtracking can occur.

 A Step toward AC-3: The most efficient algorithm

Example: 4-Queens Problem
Place 4 Queens on a chess board of 4x4 such
that no two queens reside in the same row,
column or diagonal.
1 2 3 4
• Variables: ?
• Domains: ?
1
• Constraints:?
2
3
4

[4-Queens slides copied from B.J. Dorr CMSC 421 course on AI]
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

[4-Queens slides copied from B.J. Dorr CMSC 421 course on AI]
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
Backtrack!!!
{ , , , } { ,2,3, }
Example: 4-Queens Problem
Picking up a little later after two steps of
backtracking....

X1 X2
1 2 3 4 { ,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }
Towards Constraint propagation

 Solving CSPs with combination of heuristics plus forward

checking is more efficient than either approach alone.
 FC checking propagates information from assigned to
unassigned variables but does not provide detection for all
failures.
NT and SA cannot be blue!
 Constraint propagation repeatedly enforces constraints locally
Idea 3 (big idea): Inference in CSPs
CSP solvers combine search and inference
– Search

●
assigning a value to a variable
– Constraint propagation (inference)
●
Eliminates possible values for a variable if the value
would violate local consistency
– Can do inference first, or intertwine it with search
●
You’ll investigate this in the Sudoku homework
 Local consistency
• Node consistency: satisfies unary constraints (This is trivial!)
• Arc consistency : satisfies binary constraints
—X i is arc-consistent w.r.t. X j if for every value v in D i , there is
some value w in D j that satisfies the binary constraint on the arc
between X i and X j .
Example: Sudoku

•Variables: 81 slots

•Domains =
{1,2,3,4,5,6,7,8,9}

•Constraints:
•27 not-equal

Constraint
propagation
23
426

• Each row, column and major block must be alldifferent

• “Well posed” if it has unique solution
Arc consistency

 X-> Y is consistent iff

for every value x of X there is some allowed y
 SA ->NSW is consistent iff
SA=blue and NSW=red
Arc consistency

 X-> Y is consistent iff

for every value x of X there is some allowed y
 NSW -> SA is consistent iff
NSW=red and SA=blue
NSW=blue and SA=???
Arc can be made consistent by removing blue from NSW
Arc consistency

 Arc can be made consistent by removing blue from NSW

 RECHECK neighbours !!
Remove red from V
Arc consistency

 Arc can be made consistent by removing blue from NSW

 RECHECK neighbours !!
Remove red from V
 Arc consistency detects failure earlier than FC
 Can be run as a preprocessor or after each assignment.
Repeated until no inconsistency remains

 Arc consistency does not detect all inconsistencies:

Partial assignment {WA=red, NSW=red} is inconsistent
Arc consistency

 Ideally, if no solutions, would remove all values from the domain.

 Isn’t always effective
Arc consistency algorithm
function AC-3(csp) return the CSP, possibly with reduced domains
inputs: csp, a binary csp with variables {X1, X2, …, Xn}
local variables: queue, a queue of arcs initially the arcs in csp

while queue is not empty do

(Xi, Xj) REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then
for each Xk in NEIGHBORS[Xi ] do
add (Xk, Xi) to queue

function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a value

removed false
for each x in DOMAIN[Xi] do
if no value y in DOMAIN[Xj] allows (x,y) to satisfy the constraints between Xi and Xj
then delete x from DOMAIN[Xi]; removed true
return removed
 CSP using ARC CONSISTENCY
(AC-3 Algorithm)
Three employees in a small company, Alice, Rob and Ian
must contact their respective customers as quickly as possible.
The company has one telephone, one fax and one computer (for
email), with independent lines. Alice needs to contact two
customers: one who only has a telephone and the second one
has both a telephone and a fax. Rob must contact a customer
who has also has a telephone and a fax. Ian's customer can be
reached by fax or by computer. Suppose that Alice contacts her
customers one after the other while, during the same time, Rob
and Ian are communicating with their customers.
A) Model each problem as a CSP by specifying clearly the
variables, their domains and constraints.
B) Draw the constraint graph for each CSP.
C) Apply AC-3 to the problem.
 CSP using ARC CONSISTENCY
(AC-3 Algorithm)
CSP FORMULATION:
CONSTRAINT GRAPH
Variables: {A1, A2, R, I} where
A1: Alice's 1st customer
A2: Alice's 2nd customer
R: Rob's customer
I: Ian's customer
Domain: A1 = {T} ,
A2 & R = {T, F} ,
I = {F, C} where
T: telephone
F: fax
C: computer
Constraints:
(1) Alldiff(A1, R, I)
(2) Alldiff(A2, R, I)
CSP using ARC CONSISTENCY
(AC-3 Algorithm)
CSP using ARC CONSISTENCY
(AC-3 Algorithm)
CSP using ARC CONSISTENCY
(AC-3 Algorithm)
Local search for CSP

 Use complete-state representation

 For CSPs
allow states with unsatisfied constraints
operators reassign variable values
 Variable selection: randomly select any conflicted
variable
 Value selection: min-conflicts heuristic
Select new value that results in a minimum number of conflicts with the
other variables
Local search for CSP

function MIN-CONFLICTS(csp, max_steps) return solution or failure

inputs: csp, a constraint satisfaction problem
max_steps, the number of steps allowed before giving up

current  an initial complete assignment for csp

for i = 1 to max_steps do
if current is a solution for csp then return current
var a randomly chosen, conflicted variable from VARIABLES[csp]
value the value v for var that minimizes CONFLICTS(var,v,current,csp)
set var = value in current
return faiilure
Min-conflicts example 1

h=5 h=3 h=1

 Use of min-conflicts heuristic in hill-climbing.
Min-conflicts example 2

 A two-step solution for an 8-queens problem using min-

conflicts heuristic.
 At each stage a queen is chosen for reassignment in its
column.
 The algorithm moves the queen to the min-conflict square
breaking ties randomly.
Problem structure

 How can the problem structure help to find a solution quickly?

 Subproblem identification is important:
Coloring Tasmania and mainland are independent subproblems
Identifiable as connected components of constrained graph.
 Improves performance
Problem structure

 Suppose each problem has c variables out of a total of n.

 Worst case solution cost is O(n/c dc), i.e. linear in n
Instead of O(d n), exponential in n
 E.g. n= 80, c= 20, d=2
280 = 4 billion years at 1 million nodes/sec.
4 * 220= .4 second at 1 million nodes/sec
Tree-structured CSPs

 Theorem: if the constraint graph has no loops then CSP

can be solved in O(nd 2) time
 Compare difference with general CSP, where worst case
is O(d n)
Tree-structured CSPs

 In most cases subproblems of a CSP are connected as a tree

 Any tree-structured CSP can be solved in time linear in the number
of variables.
Choose a variable as root, order variables from root to leaves such that
every node’s parent precedes it in the ordering.
For j from n down to 2, apply REMOVE-INCONSISTENT-
VALUES(Parent(Xj),Xj)
For j from 1 to n assign Xj consistently with Parent(Xj )
Nearly tree-structured CSPs

 Can more general constraint graphs be reduced to

trees?
 Two approaches:
Remove certain nodes
Collapse certain nodes
Nearly tree-structured CSPs

 Idea: assign values to some variables so that the remaining

variables form a tree.
 Assume that we assign {SA=x} cycle cutset(a set of variables
whose removal results in a tree)
And remove any values from the other variables that are inconsistent.
The selected value for SA could be the wrong one so we have to try all of them
Nearly tree-structured CSPs

 This approach is worthwhile if cycle cutset is small.

 Finding the smallest cycle cutset is NP-hard
Approximation algorithms exist
 This approach is called cutset conditioning.
Nearly tree-structured CSPs

 Tree decomposition of the

constraint graph in a set of
connected subproblems.
 Each subproblem is solved
independently
 Resulting solutions are
combined.
 Necessary requirements:
Every variable appears in at least one of
the subproblems.
If two variables are connected in the
original problem, they must appear
together in at least one subproblem.
If a variable appears in two
subproblems, it must appear in each
node on the path.
Summary

X Red
green
blue
X: red
X • Search
backtracking,
Y Z Red Red
variable/value Y: blue
green green heuristics

Y
blue blue

Z •Inference Z: green
Coloring Consistency
Constraint Graph
Problem enforcement,
forward checking

Variables
Problem Values CSP
Statement Algorithm Solution
Constraints

CSP Representation
Example: cryptharithmetic
Example: cryptharithmetic

CSP Representation:
Example: cryptharithmetic
Get solution by applying backtracking search with forward checking and
heuristics(MRV, LCV, degree heuristic).

Constraints:
(1) Alldiff(T, W, O, F, U, R)
(2) O + O = R + 10 * X1
(3) X1 + W + W = U + 10 * X2
(4) X2 + T + T = O + 10 * X3
(5) X3 = F
Example: cryptharithmetic
Get solution by applying backtracking search with forward checking and
heuristics.

Constraints:
(1) Alldiff(T, W, O, F, U, R)
(2) O + O = R + 10 * X1
(3) X1 + W + W = U + 10 * X2
(4) X2 + T + T = O + 10 * X3
(5) X3 = F
Example: cryptharithmetic
Get solution by applying backtracking search with forward checking and
heuristics.

Constraints:
(1) Alldiff(T, W, O, F, U, R)
(2) O + O = R + 10 * X1
(3) X1 + W + W = U + 10 * X2
(4) X2 + T + T = O + 10 * X3
(5) X3 = F
Example: cryptharithmetic
Get solution by applying backtracking search with forward checking and
heuristics.

Constraints:
(1) Alldiff(T, W, O, F, U, R)
(2) O + O = R + 10 * X1
(3) X1 + W + W = U + 10 * X2
(4) X2 + T + T = O + 10 * X3
(5) X3 = F
Example: cryptharithmetic
Get solution by applying backtracking search with forward checking and
heuristics.
Example: cryptharithmetic
SOLUTION:
Summary

 CSPs are a special kind of search problem: states defined by

values of a fixed set of variables, goal test defined by
constraints on variable values
 Backtracking=depth-first search with one variable assigned
per node
 Variable ordering and value selection heuristics help
significantly
 Forward checking prevents assignments that lead to failure.
 Constraint propagation does additional work to constrain
values and detect inconsistencies.
 The CSP representation allows analysis of problem structure.
 Tree structured CSPs can be solved in linear time.
 Iterative min-conflicts is usually effective in practice.
End