CONSTRAINT SATISFACTION PROBLEM (CSP)

In this lesson, the focus is on solving problems efficiently using a factored representation for
each state.
This approach involves representing a state as a set of variables, each with a specific value.
The goal is to find values for these variables that satisfy all given constraints, forming a
constraint satisfaction problem (CSP).
Key Concepts of CSPs:
1. Factored Representation:
o Each state in a problem is described by a set of variables.
o Each variable has a range of possible values.
2. Constraints:
o Conditions that variables must satisfy.
o A solution is achieved when all variables have values that satisfy these
constraints.
3. CSP Search Algorithms:
o Use the structured nature of states to solve problems.
o Apply general-purpose heuristics rather than problem-specific ones.
o Aim to reduce the search space by identifying and eliminating variable/value
combinations that violate constraints.

Example of CSPs

1. Sudoku:
o Variables: Cells in the grid.
o Values: Numbers 1-9.
o Constraints: Each number must appear exactly once in each row, column, and 3x3
subgrid.
2. Map Coloring:
o Variables: Regions on the map.
o Values: Colors.
o Constraints: Adjacent regions must have different colors.
Constraint Satisfaction Problems (CSPs) have their roots in the field of artificial intelligence and
operations research.
They emerged as a way to formalize and solve a variety of combinatorial problems where the
goal is to find a state that meets a set of constraints.
CSPs are applicable in many areas, including scheduling, planning, resource allocation, and
spatial reasoning.
A CSP can be formally defined by three main components:
1. Variables:
o A finite set of variables X={X1,X2,…,Xn}
2. Domains:
o Each variable Xi has a domain Di which is the set of possible values that Xi can
take.
D={D1,D2,…,Dn}.
3. Constraints:
o A finite set of constraints C={C1,C2,…,Cm} that restrict the values the variables
can simultaneously take.
o Each constraint Ck involves a subset of variables and specifies allowable
combinations of values for those variables.
What is the Objective?
The objective in a CSP is to find an assignment of values to all variables such that all the
constraints are satisfied.
An assignment is a function A:X→D where A(Xi)∈Di for all Xi∈X and every constraint in C is
satisfied under this assignment.
Example:
To illustrate the formal definition of a CSP.
Let us use the problem of coloring a map with three regions: A, B, and C.
The goal is to color each region such that no two adjacent regions have the same color, using the
colors Red, Green, and Blue.
Variables
The variables represent the regions that need to be colored. X={A,B,C}
Domains
Each variable (region) can be colored with one of the three colors. DA=DB=DC={Red, Green,
Blue}
D={DA, DB, DC}
Constraints
The constraints specify that adjacent regions must not share the same color.
C={CAB, CBC, CCA}
• CAB: A and B must have different colors.
• CBC: B and C must have different colors.
• CCA: C and A must have different colors.

Objective

The objective is to find a color assignment for each region such that all constraints are satisfied.

Example: Coloring the map of Australia.

This is a classic problem in constraint satisfaction problems (CSPs) known as the map-coloring
problem. To frame it as a CSP, let us break down the problem into its components: variables,
domains, and constraints.
Variables

The variables represent the regions of Australia:

X={WA, NT, Q, NSW, V, SA, T}

Domains

The domain for each variable (region) is the set of colors available:

Di={red, green, blue}

Constraints

The constraints ensure that no neighboring regions share the same color.

The neighboring relations can be expressed as:

C={SA≠WA,SA≠NT,SA≠Q,SA≠NSW,SA≠V,WA≠NT,NT≠Q,Q≠NSW,NSW≠V}

Each constraint can be detailed as follows.

For example, the constraint SA ≠ WA can be represented by:

{(red, green),(red, blue),(green, red),(green, blue),(blue, red),(blue, green)}

Example Solution

A possible solution to this problem is to assign colors to each region in such a way that no
neighboring regions share the same color.

For instance: {WA=red, NT=green, Q=red, NSW=green, V=red, SA=blue, T=red}

Note that: T (Tasmania) doesn't border any other region in this problem, so its color is
independent.

By using the constraint graph, you can better conceptualize the relationships between the
variables in the CSP and approach the problem with a clearer strategy.

Qn: What is the best node to start with (heuristically), class?

 HEURISTIC APPROACH IN CSP
Heuristics in Constraint Satisfaction Problems (CSPs) are strategies used to improve the
efficiency of the search process by making intelligent choices about which variables to assign
and which values to try.
These heuristics are not guaranteed to find the optimal solution, but they significantly reduce the
search space and time, making it easier to solve complex problems.

1. MRV
The Minimum Remaining Values (MRV) heuristic, also known as the "most constrained
variable" heuristic, selects the variable with the fewest legal values remaining in its domain.
By choosing the most constrained variable, MRV aims to reduce the branching factor of the
search tree as early as possible, which facilitates the quick detection of failures.
For example, in a Sudoku puzzle, if a cell has only two possible numbers, it is more strategic to
assign a value to that cell first rather than a cell with five possible numbers, as this minimizes
potential conflicts and narrows down the search space more effectively.

2. Degree Heuristic
The Degree Heuristic selects the variable involved in the largest number of constraints with other
unassigned variables.
By choosing the variable that interacts with the most other variables, the degree heuristic aims to
maximize the impact of the variable assignments on the remaining unassigned variables.
For example, in a map-coloring problem, a region that borders the most other regions would be
selected first for coloring.

3. LCV
The Least Constraining Value (LCV) heuristic, once a variable is chosen, picks the value that
imposes the fewest constraints on the remaining unassigned variables.
LCV tries to keep future options open by choosing values that leave the most flexibility for
subsequent variable assignments.
For instance, in scheduling problems, assigning a time slot that leaves the most other time slots
available for future assignments helps maintain flexibility.
 4. Forward Checking
Forward Checking, after assigning a value to a variable, immediately eliminates values from the
domains of neighboring variables that are inconsistent with the assignment.
By proactively reducing the domains of future variables, forward checking prevents dead-end
paths earlier, which reduces the need for backtracking.
For example, in a logic puzzle, if assigning a particular value to one variable eliminates all
possible values for a neighboring variable, forward checking helps detect this conflict early.

5. Constraint Propagation
Constraint Propagation techniques like arc consistency (AC-3 algorithm) enforce local
consistency by ensuring that for every pair of variables, every value of one variable has a
consistent value in the other variable.
By maintaining a higher level of consistency throughout the search, constraint propagation
reduces the need for extensive backtracking.
For example, in a crossword puzzle, if one word's placement restricts the letters in intersecting
words, constraint propagation helps adjust those intersecting variables to maintain consistency.

Combining Heuristics
Heuristics are often combined to leverage their strengths.
For example, using MRV to choose the next variable and then applying LCV to choose the value
for that variable can be more effective than using either heuristic alone.
Benefits of Using Heuristics in CSPs
• Efficiency: Heuristics reduce the size of the search space and the number of variable
assignments needed.
• Effectiveness: They help quickly detect conflicts and dead ends, reducing the need for
backtracking.
• Scalability: Heuristics enable the solving of larger and more complex CSPs that would
be infeasible to solve with brute-force methods.

Benefits of CSPs

1. Generality

CSPs provide a general framework that can be applied to a wide variety of problems across
different domains.

Whether it's scheduling, planning, resource allocation, or spatial reasoning, the same CSP
techniques can be used without needing problem-specific modifications.

2. Structured Problem Representation

By using variables, domains, and constraints, CSPs offer a clear and structured way to represent
problems.

This structured representation simplifies the problem-solving process and makes it easier to
understand and manage the problem's components.
3. Reusability of Techniques

The algorithms and heuristics developed for CSPs are reusable across different problems.

Techniques like backtracking, forward checking, and constraint propagation can be applied to
many types of CSPs, saving time and effort in developing new solutions.

4. Efficiency

CSP algorithms, especially those incorporating heuristics and constraint propagation, are
designed to efficiently prune the search space.

This efficiency allows CSPs to solve problems that would be infeasible to address with brute-
force methods.

5. Scalability

CSPs can handle large and complex problems by breaking them down into smaller, more
manageable components.

The use of heuristics and constraint propagation techniques helps scale solutions to problems
with many variables and constraints.

6. Flexibility

CSPs can easily incorporate additional constraints and modify existing ones, providing flexibility
in problem formulation.

This adaptability is useful in dynamic environments where problem conditions may change over
time.

7. Optimality

For certain types of CSPs, especially those that can be solved with exact algorithms, the
solutions found are guaranteed to be optimal.

This is crucial in applications where the best possible solution is required.

8. Parallelism

CSPs can often be solved using parallel processing techniques, where different parts of the
search space are explored simultaneously.

This parallelism can significantly speed up the problem-solving process.

9. Automation

The structured nature of CSPs allows for automation in problem-solving.

CSP solvers can automatically find solutions without extensive human intervention, making
them suitable for applications requiring autonomous decision-making.

10. Theoretical Foundations

CSPs are backed by strong theoretical foundations from fields like graph theory, logic, and
optimization.

These foundations provide a robust basis for developing efficient and effective algorithms.
 CSP METHODS

CSPs can be solved using a variety of methods, each with its own strengths and applicable
scenarios.

1. Backtracking Search

Backtracking is a depth-first search algorithm that incrementally builds candidates for solutions
and abandons a candidate (i.e., backtracks) as soon as it determines that the candidate cannot
possibly lead to a valid solution.

• Basic Algorithm:
o Start with an empty assignment.
o Choose an unassigned variable.
o Assign a value to the variable that doesn't violate any constraints.
o Recursively attempt to assign values to the remaining variables.
o If a constraint is violated, backtrack to the previous assignment and try the next
value.
• Advantages: Simple and easy to implement.
• Disadvantages: Can be inefficient due to the large search space.

2. Forward Checking

Forward checking is an enhancement of the backtracking algorithm.

It reduces the search space by keeping track of remaining legal values for unassigned variables
and terminating early when any variable has no legal values left.

• Algorithm:
o After assigning a value to a variable, remove incompatible values from the
domains of all neighboring unassigned variables.
o If any variable’s domain becomes empty, backtrack immediately.
• Advantages: Helps prevent the exploration of paths that are guaranteed to fail.
• Disadvantages: Still can be inefficient for large problems.
3. Constraint Propagation

Constraint propagation methods further reduce the search space by enforcing local consistency
conditions. Common techniques include arc consistency, path consistency, and k-consistency.

• Arc Consistency (AC-3 algorithm):

o For every pair of variables connected by a constraint, ensure that for every value
of one variable, there is a consistent value in the other variable.
o Repeat until no more values can be removed.
• Advantages: More powerful than forward checking in reducing the search space.
• Disadvantages: Can be computationally expensive.

4. Heuristic Search

Heuristics are used to make the search process more efficient by intelligently selecting variables
and values.

• Minimum Remaining Values (MRV): Select the variable with the fewest legal values.
• Degree Heuristic: Select the variable that is involved in the largest number of
constraints.
• Least Constraining Value (LCV): Choose the value that leaves the most options open
for other variables.
• Advantages: Significantly improves the efficiency of the search.
• Disadvantages: The effectiveness depends on the problem and the heuristics used.

5. Local Search

Local search algorithms start with an initial assignment and iteratively make small changes to
improve the assignment.

• Min-Conflicts Algorithm:
o Start with a complete, but possibly inconsistent assignment.
o Select a variable involved in a conflict and assign it a value that minimizes the
number of conflicts.
o Repeat until a solution is found or a maximum number of steps is reached.
• Advantages: Can quickly find solutions for large problems.
• Disadvantages: May get stuck in local minima, requiring restarts or additional
techniques.
6. Global Search Algorithms

These algorithms, such as Genetic Algorithms (GA) and Simulated Annealing (SA), use global
optimization techniques to explore the search space.

• Genetic Algorithms:
o Use evolutionary techniques to evolve a population of candidate solutions.
o Apply selection, crossover, and mutation to generate new candidates.
• Simulated Annealing:
o Mimic the annealing process in metallurgy to explore the search space.
o Gradually reduce the "temperature" to decrease the likelihood of accepting worse
solutions.
• Advantages: Can escape local minima and explore the search space globally.
• Disadvantages: Often require careful tuning of parameters.

7. Hybrid Methods

Combining different techniques can leverage the strengths of multiple methods.

For example, a hybrid approach might use constraint propagation to reduce the search space and
then apply backtracking or local search to find a solution.

• Advantages: Can be more robust and efficient than single methods.

• Disadvantages: Complexity in implementation and tuning.
EXERCISE:
1. How many solutions are there for the map-coloring problem in Figure 6.1? How many
solutions if four colors are allowed? Two colors?
2. Explain why it is a good heuristic to choose the variable that is most constrained but the
value that is least constraining in a CSP search.
3. Consider the following logic puzzle: In five houses, each with a different color, live five
persons of different nationalities, each of whom prefers a different brand of candy, a
different drink, and a different pet. Given the following facts, the questions to answer are
“Where does the zebra live, and in which house do they drink water?”
The Englishman lives in the red house.
The Spaniard owns the dog.
The Norwegian lives in the first house on the left.
The green house is immediately to the right of the ivory house.
The man who eats Hershey bars lives in the house next to the man with the fox.
Kit Kats are eaten in the yellow house.
The Norwegian lives next to the blue house.
The Smarties eater owns snails.
The Snickers eater drinks orange juice.
The Ukrainian drinks tea.
The Japanese eats Milky Ways.
Kit Kats are eaten in a house next to the house where the horse is kept.
Coffee is drunk in the green house.
Milk is drunk in the middle house.
Discuss different representations of this problem as a CSP. Why would one prefer one
representation over another?