Lecture 5 - Constraint Satisfaction Problems

Introduction to Artificial Intelligence
Constraint Satisfaction Problems
University of Technology and Applied Sciences

Computing and Information Sciences
Outline
1. Definition of CSPs
2. CSP Examples
3. Backtracking Search
4. Domain Update
5. Dynamic Ordering
1
Defining Constraint Satisfaction Problems
• A constraint satisfaction problem (CSP) consists of three components:

◦ 𝑋 is a set of variables, {𝑋1 , … , 𝑋𝑛 }
◦ 𝐷 is a set of domains, {𝐷1 , … , 𝐷𝑛 }, one for each variable.
◦ 𝐶 is a set of constraints that specify the allowable combination of values.
Solving CSPs
CSPs deal with assignment of values to variables.
• A solution to a CSP is a consistent, complete assignment.

• A complete assignment is one in which every variable is assigned a value.
• A consistent assignment is one in which no constraints are violated.
• A partial assignment is one that leaves some variables unassigned.
• A partial solution is a partial assignment that is consistent.
2
CSPs vs Search Problems
Standard search problem: CSP:
• A state is a “black box” – any old data • A state is a factored representation –
support goal test, successor, cost. defined by variables 𝑋𝑖 with values from
• The order of the sequence of actions to domain 𝐷𝑖 .
reach the goal is important. • A goal test is a set of constraints
specifying allowable values for variables.
• We don’t often care about how do we get
to the goal, as long as we reach the goal
in a reasonable amount of time
Outline
2. CSP Examples
4. Domain Update
5. Dynamic Ordering
3
Example: Map Coloring
• Variables: 𝑊𝐴, 𝑁𝑇, 𝑄, 𝑁𝑆𝑊, 𝑉, 𝑆𝐴, 𝑇 Northern

• Domains: 𝐷𝑖 = {𝑟𝑒𝑑, 𝑔𝑟𝑒𝑒𝑛, 𝑏𝑙𝑢𝑒} Western
Territory
Queensland
Australia
• Constraints: adjacent regions must have
South
different colors Australia
◦ E.g., 𝑊𝐴 ≠ 𝑁𝑇, or New South Wales
◦ 𝑊𝐴, 𝑁𝑇 ∈ { 𝑟𝑒𝑑, 𝑔𝑟𝑒𝑒𝑛 , 𝑟𝑒𝑑, 𝑏𝑙𝑢𝑒 , … }

Victoria
Tasmania
Exercise
Find a solution to the Australia’s color Northern

Territory
mapping problem, i.e., color each province Western
Australia
Queensland
with either red, green or blue so that no

South
two neighboring provinces have the same Australia
color. New South Wales
Victoria
Tasmania
4
Map Coloring Solution
Northern
Territory
Western Queensland
Australia
South
Australia
New South Wales
Victoria
Tasmania
Solutions are assignments satisfying all constraints, e.g.,

{𝑊𝐴 = 𝑟𝑒𝑑, 𝑁𝑇 = 𝑔𝑟𝑒𝑒𝑛, 𝑄 = 𝑟𝑒𝑑, 𝑁𝑆𝑊 = 𝑔𝑟𝑒𝑒𝑛, 𝑉 = 𝑟𝑒𝑑, 𝑆𝐴 = 𝑏𝑙𝑢𝑒, 𝑇 = 𝑔𝑟𝑒𝑒𝑛}
Constraint Graph
NT
• Binary CSP: each constraint relates at Q
most two variables. WA
• Constrain graph: nodes are variables,
arcs show constraints. SA NSW
• General purpose CSP algorithms can

use the graph structure to speed up V
search. E.g. Tasmania is an independent
sub-problem!
T
10
10
5
Varieties of Variables
• Discrete variables:
◦ Finite domains:
− Assume 𝑛 variable, 𝑑 values, the number of complete assignment is 𝑂(𝑑 𝑛 ).
− E.g., map coloring, 8-queens, …
◦ Infinite domains (integers, strings, etc.)
− Need a constraint language
− E.g., job scheduling 𝑆𝑡𝑎𝑟𝑡𝐽𝑜𝑏1 + 5 ≤ 𝑆𝑡𝑎𝑟𝑡𝐽𝑜𝑏2
• Continuous variables:
◦ Common in operations research: e.g., energy optimization in a production facility.
◦ Linear programming (LP) methods.
11
11
Varieties of Constraints
• Unary constraints involve a single variable:

◦ E.g., 𝑆𝐴 ≠ 𝑔𝑟𝑒𝑒𝑛
• Binary constraints involve pairs of variables,
◦ E.g., 𝑆𝐴 ! ≠ 𝑊𝐴
• High-order constraints involve 3 or more variables,
◦ E.g., cryptarithmetic column constraints
• Preferences (soft constraints), e.g., 𝑟𝑒𝑑 is better than 𝑔𝑟𝑒𝑒𝑛
◦ Often representable by cost for each variable assignment
◦ Constrained optimization problems.
12
12
6
Example: Cryptarithmetic
Find digit-letter mapping that
solves the equation.
• Variables: 𝐹 𝑇 𝑈 𝑊 𝑅 𝑂 𝑋1 𝑋2 𝑋3
X3 X2 X1
T WO
• Domains: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Constraints
Hypergraph + T WO • Constraints:
F O U R ◦ 𝐴𝑙𝑙𝑑𝑖𝑓𝑓 (𝐹, 𝑇, 𝑈, 𝑊, 𝑅, 𝑂)
◦ 𝑂 + 𝑂 = 𝑅 + 10 . 𝑋1
◦ 𝑋1 + 𝑊 + 𝑊 = 𝑈 + 10 . 𝑋2
F T U W R O ◦ 𝑋2 + 𝑇 + 𝑇 = 𝑂 + 10 . 𝑋3
◦ 𝑋3 = 𝐹
X3 X2 X1
13
13
Real-World CSPs
• Assignment problems
◦ E.g., who teaches what
• Timetabling problems
◦ E.g., which class is offered when and where
• Hardware configurations
• Datacenter layout planning
• Factory scheduling
• Logistics and supply chain
• More example of such CSPs: https://www.csplib.org/
14
14
7
Exercise
Formulate the following cryptarithmetic as a CSP. Draw the constraints hyper graph.
X3 X2 X1
D I G
+ D I G
H O L E
15
15
Exercise: Solution
Formulate the following cryptarithmetic as a CSP. Draw the constraints hyper graph.
X3 X2 X1
D I G
+ D I G H O D L I E G
H O L E
Variables: 𝐻 𝑂 𝐷 𝐿 𝐼 𝐸 𝐺 𝑋1 𝑋2 𝑋3
Domains: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
Constraints:
• 𝐴𝑙𝑙𝑑𝑖𝑓𝑓 (𝐻, 𝑂, 𝐷, 𝐿, 𝐼, 𝐸, 𝐺)
X3 X2 X1
• 𝐺 + 𝐺 = 𝐸 + 10 . 𝑋1
• 𝑋1 + 𝐼 + 𝐼 = 𝐿 + 10 . 𝑋2
• 𝑋2 + 𝐷 + 𝐷 = 𝑂 + 10 . 𝑋3
• 𝑋3 = 𝐻
16
16
8
Outline
2. CSP Examples
4. Domain Update
5. Dynamic Ordering
17
17
Solving CSPs: Uninformed Search Approach
• Let us start with the straightforward, naive approach, then fix it.
• States are defined by values assigned so far, i.e., partial assignments.
• Solving CSP as a search problem:
◦ Initial state: the empty assignment {}
◦ Successor function: assign a value to an unassigned variable that does not conflict
with current assignment
− Fail if no legal values (not fixable!)
◦ Goal test: the current assignment is complete
• Every solution appears at depth 𝑛, where 𝑛 is the number of variables
◦ Same as games, use DFS
• Path is irrelevant, we care about the solution at the end.
18
18
9
Backtracking Search
Backtracking search (BTS) is a basic uninformed search for CSPs. It uses DFS such
that:
1. Assign one variable at a time: assignments are commutative, e.g., (𝑊𝐴 = 𝑟𝑒𝑑, 𝑁𝑇 =
𝑔𝑟𝑒𝑒𝑛) is the same as (𝑁𝑇 = 𝑔𝑟𝑒𝑒𝑛, 𝑊𝐴 = 𝑟𝑒𝑑).
2. Check constraints on the go: consider values that don’t conflict with previous
assignment.
3. If an assignment fails, backtrack to upper level, and assign other value (expensive).
19
19
BTS Example
NT
Q
WA
Use the BTS to color the shown map. Pick
variable according to the following order: SA NSW
𝑊𝐴, 𝑁𝑇, 𝑆𝐴, 𝑄, 𝑉, 𝑁𝑆𝑊, and 𝑇. The order of
values is 𝑅, 𝐺, and 𝐵.
V
20
20
10
BTS Example
NT
Q
WA
SA NSW
Initialize variables and domains. Nothing is assigned yet.
21
21
BTS Example
NT
Q
WA
SA NSW
Assign R to WA. Assignment consistent. Recurse.
22
22
11
BTS Example
NT
Q
WA
SA NSW
Assign R to NT. Assignment inconsistent. Continue next value.
23
23
BTS Example
NT
Q
WA
SA NSW
Assign G to NT. Assignment consistent. Recurse.
24
24
12
BTS Example
NT
Q
WA
SA NSW
Assign R to SA. Assignment inconsistent. Continue next value.
25
25
BTS Example
NT
Q
WA
SA NSW
Assign G to SA. Assignment inconsistent. Continue next value.
26
26
13
BTS Example
NT
Q
WA
SA NSW
Assign B to SA. Assignment consistent. Recurse.
27
27
BTS Example
NT
Q
WA
SA NSW
Assign R to Q. Assignment consistent. Recurse.
28
28
14
BTS Example
NT
Q
WA
SA NSW
Assign R to V. Assignment consistent. Recurse.
29
29
BTS Example
NT
Q
WA
SA NSW
Assign R to NSW. Assignment inconsistent. Continue next value.
30
30
15
BTS Example
NT
Q
WA
SA NSW
Assign G to NSW. Assignment consistent. Recurse.
31
31
BTS Example
NT
Q
WA
SA NSW
Assign R to T. Assignment consistent. Recurse.
32
32
16
BTS Example
NT
Q
WA
SA NSW
Assignment complete. Return ☺
33
33
BTS Solution Tree
NT
Q
WA
SA NSW
34
34
17
BTS Exercise
NT
Q
WA
Use the BTS to color the shown map. Pick
variable according to the following order: SA NSW
𝑊𝐴, 𝑁𝑇, 𝑄, 𝑁𝑆𝑊, 𝑉, 𝑆𝐴, and 𝑇. The order of
values is 𝑅, 𝐺, and 𝐵.
V
35
35
Outline
2. CSP Examples
4. Domain Update
5. Dynamic Ordering
36
36
18
Improving BTS
• BTS can fail to assign value to a variable. Hence need to backtrack.

◦ Expensive in large or complex CSPs
• Can we detect inevitable failure early?

◦ Identify potential conflicts before they arise.
◦ Help prune unnecessary explorations of doomed paths.
• The order of picking variables and values makes a difference

◦ How to prioritize?
37
37
Improving BTS
Heuristics are back! We will look at two approaches:

1. Domain Update: detect inevitable failure early.
◦ Forward checking (FC)
◦ Arc consistency (AC3 Algorithm) - see the book
◦ Also called constraint propagation.
2. Dynamic Ordering:
◦ Which variables should be assigned next? Most Constrained Variable (MCV)
heuristic.
◦ In what order should it’s values be tried? Least Constraining Value (LCV).
38
38
19
Backtracking Search Algorithm
def backtracking_search(csp):
return backtrack({}, csp)
def backtrack(assignment, csp):

if is_complete(assignment, csp): # If the assignment is complete
return assignment # return the assignment
var = select_unassigned_variable(assignment, csp) # Select next unassigned variable
for value in order_domain_values(var, assignment, csp): # Iterate over each value

if is_consistent(var, value, assignment, csp): # value is consistent
assignment[var] = value # assign it
result = backtrack(assignment.copy(), csp) # recurse
if result != None: # If result is not a None,

return result # return it
# remove value form it from the assignment

del assignment[var]
return None
39
39
Domain Update
• Key idea: Forward checking provides one-step lookahead.
• After assigning a variable 𝑋𝑖 , eliminate inconsistent values from the
domains of 𝑋𝑖 ’s neighbors.
• If any domain becomes empty, don’t recurse, continue with next values.
NT NT
Q Q
Assign Q
value B
WA WA
SA NSW SA NSW
V V
T T
40
40
20
BTS+FC Example
NT
Q
WA
Use the BTS+FC to color the shown map.
Pick variable according to the following SA NSW
order: 𝑊𝐴, 𝑁𝑇, 𝑆𝐴, 𝑄, 𝑉, 𝑁𝑆𝑊, and 𝑇. The
order of values is 𝑅, 𝐺, and 𝐵.
V
41
41
Backtracking Search (With Forward Checking)
NT
Q
WA
SA NSW
42
42
21
NT
Q
WA
SA NSW
Assign R to WA. Assignment consistent. Update NT, SA

domains. Recurse.
43
43
NT
Q
WA
SA NSW
Assign G to NT. Assignment consistent. Update SA, Q domains.

Recurse.
44
44
22
NT
Q
WA
SA NSW
Assign B to SA. Assignment consistent. Update Q, NSW, V

domains. Recurse.
45
45
NT
Q
WA
SA NSW
Assign R to Q. Assignment consistent. Update NSW domain.

Recurse.
46
46
23
NT
Q
WA
SA NSW
Assign R to V. Assignment consistent. Update NSW domain.

Recurse.
47
47
NT
Q
WA
SA NSW
Assign G to NSW. Assignment consistent. Recurse.
48
48
24
NT
Q
WA
SA NSW
49
49
NT
Q
WA
SA NSW
50
50
25
BTS +FC Solution Tree
NT
Q
WA
SA NSW
51
51
BTS+FC Exercise
NT
Q
Use the BTS with Forward Checking (FC) WA

to color the shown map. Pick variable SA NSW
according to the following order:
𝑊𝐴, 𝑁𝑇, 𝑄, 𝑁𝑆𝑊, 𝑉, 𝑆𝐴, and 𝑇. The order of
values is 𝑅, 𝐺, and 𝐵. V
52
52
26
Outline
2. CSP Examples
4. Domain Update
5. Dynamic Ordering
53
53
Variable and Value Ordering
• The algorithms we considered so far require the order in which variables or values
picked to be specified.
• Choosing the right order of variables (and values) can noticeably improve the efficiency
of the CSP algorithm.
• Dynamic ordering: let the algorithm decide the next variable or value to be picked for
consideration.
• To implement dynamic ordering, we use two heuristics:
◦ Most constraint variable (MCV) for variable ordering
− Also called Minimum Remaining Value (MRV).
◦ Least constraining value (LCV) for value ordering.
• Note, these heuristics are useful iff domain update is used. Why?
54
54
27
Most Constrained Variable (MCV)
• Key Idea: Choose the variable with the fewest legal values in its domain.
NT
Q
Which variable should WA

be assigned first? SA NSW
55
55
BTS + Variable Ordering Example
NT
Q
WA
Use the BTS, with variable ordering to
color the shown map. Pick variables SA NSW
according to the MCV heuristic. Assign WA
first. The order of values is 𝑅, 𝐺, and 𝐵.
V
56
56
28
Backtracking Search (With Variable Ordering)
NT
Q
WA
SA NSW
57
57
NT
Q
WA
SA NSW

domains. Recurse.
58
58
29
NT
Q
WA
SA NSW

Recurse.
59
59
NT
Q
WA
SA NSW
Assign B to SA. Assignment consistent. Update Q, NSW, V

domains. Recurse.
60
60
30
NT
Q
WA
SA NSW
Assign R to Q. Assignment consistent. Update NSW domain.

Recurse.
61
61
NT
Q
WA
SA NSW
Assign G to NSW. Assignment consistent. Update V domain.

Recurse.
62
62
31
NT
Q
WA
SA NSW
63
63
NT
Q
WA
SA NSW
64
64
32
NT
Q
WA
SA NSW
65
65
BTS + Variable Ordering, Exercise
NT
Q
WA
Use the BTS, with variable ordering to
color the shown map. Pick variables SA NSW
according to the MCV heuristic. Assign SA
first. The order of values is 𝑅, 𝐺, and 𝐵.
V
66
66
33
Least Constraining Value (LCV)
NT
Q
WA
SA NSW
What value should we assign to Q?
67
67
NT
Q
WA
Assigning R to Q leaves 2 + 2 + 2 = 6 SA NSW

consistent values.
V
68
68
34
NT
Q
WA
Assigning B to Q leaves 1 + 1 + 2 = 4 SA NSW

consistent values.
V
Which value is better for

Q? R or B? Why? T
69
69
Key Idea: Given a variable 𝑋𝑖 , pick the value that rules out the
fewest choices for the neighboring variables. So, this heuristic
allows you to pick the value which gives maximum flexibility for
subsequent variable assignment.
70
70
35
BTS + Value Ordering Example
NT
Q
Use the BTS, with variable ordering to WA

color the shown map. Pick the variables SA NSW
according to the following order: WA, NT,
SA, NSW, Q, V, and T. The order of values
should be according to the LCV heuristic. V
71
71
Backtracking Search (With Value Ordering)
NT
Q
WA
SA NSW
Next variable: WA
V
Consistent values:
R=4, G=4, B=4
72
72
36
NT
Q
WA
SA NSW
Next variable: NT
V
Consistent values:
G=3, B=3

domains. Recurse.
73
73
NT
Q
WA
SA NSW
Next variable: SA
V
Consistent values:
B=5

Recurse.
74
74
37
NT
Q
WA
SA NSW
Next variable: NSW
V
Consistent values:
R=1, G=2
Assign B to SA. Assignment consistent. Update Q, NSW,V

domains. Recurse.
75
75
NT
Q
WA
SA NSW
Next variable: Q
V
Consistent values:
R=0
Assign G to NSW. Assignment consistent. Update Q, V domains.

Recurse.
76
76
38
NT
Q
WA
SA NSW
Next variable: V
V
Consistent values:
R=0
Assign R to Q. Assignment consistent. Recurse.
77
77
NT
Q
WA
SA NSW
Next variable: T
V
Consistent values:
R=0, G=0, B=0
78
78
39
NT
Q
WA
SA NSW
79
79
NT
Q
WA
SA NSW
80
80
40
Homework
NT
Q
WA
Use the BTS with forward checking (FC).
Pick the variables according to the MCV SA NSW
heuristic. The order of the values should
be to the LCV heuristic.
V
81
81
Recommended Reading
For more on the topics covered in this lecture please refer to the following
sources:
• Russell-Norvig Book (Russell & Norvig, 2020): Sections 5.1 – 5.3.
82
82
41
References
Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach

(4th Edition). Pearson. http://aima.cs.berkeley.edu/
83
83
42

Lecture 5 - Constraint Satisfaction Problems

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture 5 - Constraint Satisfaction Problems

Uploaded by

Copyright:

Available Formats

Introduction to Artificial Intelligence

Constraint Satisfaction Problems

University of Technology and Applied Sciences

• A constraint satisfaction problem (CSP) consists of three components:

CSPs deal with assignment of values to variables.

• A solution to a CSP is a consistent, complete assignment.

• Variables: 𝑊𝐴, 𝑁𝑇, 𝑄, 𝑁𝑆𝑊, 𝑉, 𝑆𝐴, 𝑇 Northern

◦ 𝑊𝐴, 𝑁𝑇 ∈ { 𝑟𝑒𝑑, 𝑔𝑟𝑒𝑒𝑛 , 𝑟𝑒𝑑, 𝑏𝑙𝑢𝑒 , … }

Find a solution to the Australia’s color Northern

with either red, green or blue so that no

Solutions are assignments satisfying all constraints, e.g.,

• General purpose CSP algorithms can

• Unary constraints involve a single variable:

Solving CSPs: Uninformed Search Approach

Initialize variables and domains. Nothing is assigned yet.

Assign R to WA. Assignment consistent. Recurse.

Assign R to NT. Assignment inconsistent. Continue next value.

Assign G to NT. Assignment consistent. Recurse.

Assign R to SA. Assignment inconsistent. Continue next value.

Assign G to SA. Assignment inconsistent. Continue next value.

Assign B to SA. Assignment consistent. Recurse.

Assign R to Q. Assignment consistent. Recurse.

Assign R to V. Assignment consistent. Recurse.

Assign R to NSW. Assignment inconsistent. Continue next value.

Assign G to NSW. Assignment consistent. Recurse.

Assign R to T. Assignment consistent. Recurse.

Assignment complete. Return ☺

BTS Solution Tree

• BTS can fail to assign value to a variable. Hence need to backtrack.

• Can we detect inevitable failure early?

• The order of picking variables and values makes a difference

Heuristics are back! We will look at two approaches:

def backtrack(assignment, csp):

var = select_unassigned_variable(assignment, csp) # Select next unassigned variable

for value in order_domain_values(var, assignment, csp): # Iterate over each value

if result != None: # If result is not a None,

# remove value form it from the assignment

Backtracking Search (With Forward Checking)

Initialize variables and domains. Nothing is assigned yet.

Assign R to WA. Assignment consistent. Update NT, SA

Backtracking Search (With Forward Checking)

Assign G to NT. Assignment consistent. Update SA, Q domains.

Assign B to SA. Assignment consistent. Update Q, NSW, V

Backtracking Search (With Forward Checking)

Assign R to Q. Assignment consistent. Update NSW domain.

Assign R to V. Assignment consistent. Update NSW domain.

Backtracking Search (With Forward Checking)

Assign G to NSW. Assignment consistent. Recurse.

Assign R to T. Assignment consistent. Recurse.

Backtracking Search (With Forward Checking)

Assignment complete. Return ☺

Use the BTS with Forward Checking (FC) WA

Variable and Value Ordering

Which variable should WA

BTS + Variable Ordering Example

Initialize variables and domains. Nothing is assigned yet.

Backtracking Search (With Variable Ordering)

Assign R to WA. Assignment consistent. Update NT, SA

Assign G to NT. Assignment consistent. Update SA, Q domains.

Backtracking Search (With Variable Ordering)

Assign B to SA. Assignment consistent. Update Q, NSW, V

Assign R to Q. Assignment consistent. Update NSW domain.

Backtracking Search (With Variable Ordering)

Assign G to NSW. Assignment consistent. Update V domain.