How should feasibility be handled by Genetic Algorithms on Constraint Combinatorial Optimization Problems?

The case of the valued n-queens problem (extended version)

Vladimir Estivill-Castro Department of Computer Science and Software Engineering University of Newcastle University Drive, Callaghan NSW 2308 Australia vlad@cs.newcastle.edu.au Rodolfo Torres-Vel zquez a Department of Computer Science and Software Engineering University of Newcastle University Drive, Callaghan NSW 2308 Australia rodolfo@cs.newcastle.edu.au

Abstract- We show that a Genetic Algorithm based on the Baldwin effect successfully attacks Constraint Combinatorial Optimization Problems (CCOPs) for permutations. Deterministic mechanisms for efcient construction of feasible solutions form the basis of our approach. We illustrate our proposal with the valued n-queens problem, a typical CCOP. In this case, we obtain high quality solutions with execution times several orders of magnitude lower than the most recent previous approach.

1 Introduction
Constraint Satisfaction Problems (CSPs) have long been studied, mainly because many problems arising in Operation Research can be represented as CSPs [Smith, 1995]. Moreover, if a problem can be modeled using variables, values (domains for variables) and constraints, then constraint-satisfaction techniques can be applied to solve it [Tsang, 1999]. The general CSP is known to be NP-complete [Eiben and Ruttkay, 1997]. Additionally, when the search seeks an optimum feasible solution, the CSP is named a Constraint Optimization Problem [Paredis, 1993]. Constraint problems pose an important question to the eld of Genetic Algorithms (GAs): How should the concept of feasibility be handled?. In this respect, several proposals have been suggested, from the numerical perspective [Michalewicz, 1995] to the combinatorial perspective [Eiben and Ruttkay, 1997]. Two well known results from the eld of GAs are relevant to this question. First, the improvement of the genotype, usually by a local search mechanism [Ulder et al., 1991], has been recognized as an efcient way of coping with hard combinatorial optimization problems. Second, for many combinatorial optimization problems it is relatively easy to transform an infeasible genotype into a feasible by a xing process [Michalewicz, 1995]. These results are a window into our proposal. Genetic algorithms based on the Baldwin effect (also called Baldwinian approach) use the result of the improvement to the genotype without modifying the genotype. We introduce the Baldwinian approach to effectively deal with feasibility on the class of Constraint Combinatorial Optimization Problems (CCOPs) that admit formulation as permutation prob-

lems. As a distinctive feature, our proposal includes a constructive deterministic stage to repair (without backtracking) infeasible genotypes. The n-queens problem [Tsang, 1999] and the valued n-queens problem [Paredis, 1993, Ruiz-Andino et al., 2000] are cited frequently as prototypical CSPs and CCOPs, respectively. It has been shown that, for the n-queens problem, GAs are not as effective as a local-search mechanism based on conict minimization [Sosi and Gu, 1994, Hynek, 2000b] (the c trade-off on time requirement to nd a solution favors the latter). On the other hand, a recent approach [Ruiz-Andino et al., 2000] for the valued n-queens problem piles up high execution times; with CPU requirements increasing exponentially with the size of the problem. In this paper we efciently solve the valued n-queens problem, as a proof-of-concept example. For this problem we obtain good quality solutions while requiring several orders of magnitude less execution time than the most recent previous approach [Ruiz-Andino et al., 2000]. Moreover, because we apply conventional order-based GAs, our approach is easier to implement than others. Although this paper illustrates our approach with the valued n-queens problem we indicate how our techniques extend in a natural fashion to a more general CCOP framework. To detail our algorithm, we shall review the Baldwin effect. Section 2 provides the denition for Baldwinian algorithms, presents a detailed map of techniques for solving CSP and CCOPs, and locates our contribution in this horizon. Section 3 provides the details of or approach in generic form. Section 4 instantiates our methods to the valued n-queens problem. A summary of experimental results using our methods for this problem are presented in Section 5. We conclude with a discussion in Section 6.

2 Preliminaries
2.1 The Baldwin Effect The most common form of hybrid-GA uses local search [Ulder et al., 1991] to improve tness on genotypes produced by genetic operators. There are two general approaches for dealing with the resulting improvements: Baldwinian and Lamar-

Lamarckian Approach Genotype Baldwinian Approach

Phenotype Fitness Phenotype

b

Search space S d

Genotype Fitness Phenotype

Infeasible search space

Population
Feasible search space

Figure 1: Lamarckian and Baldwinian. Two approaches to handle the results of genotype improving. ckian. The characteristic of Baldwinian is not to bring back genotypes improved by local search to the evolutionary cycle. As a result, without altering the genotype, the Baldwinian approach changes the tness of the individual and modies the tness landscape [Whitley et al., 1994]. If genotypes are modied by hybridization with local search, then the approach is Lamarckian. Thus, Lamarckian approaches encode the resulting improvements onto the genotype processed by the genetic algorithm. Figure 1 describes both approaches. It becomes important to clarify the meaning of the terms genotype and phenotype because the distinction between Baldwinian and Lamarckian is concerned with the inclusion or exclusion of structural change to the genotype [Whitley et al., 1994], while both assess tness (or use local search) on the phenotype. Genotype means the internally coded inheritable information of a living organism. Phenotype means anything that is part of the observable structure, function or behavior of a living organism. While the spirit of these denitions imported from Biological Theory assumes that to every genotype there is one phenotype, the application of Genetic Algorithms to Constraint Problems faces the challenge that genotypes may be infeasible, and therefore, they may lack a phenotype. We make the observation that this implies two subtypes of (Baldwinian or Lamarckian) approaches. One that repairs genotypes for feasibility and one that explores a neighborhood of feasible phenotypes. The later is as originally applied [Whitley et al., 1994, Turney, 1996]. Using the same form of local search, and a set of test functions, experimental evidence has shown that Baldwinian algorithms nd better optima than a Lamarckian algorithms [Whitley et al., 1994]. However, in the experiments, the former are slower than the latter. Thus, this result left open the need to evaluate the effectiveness of Baldwinian algorithms in more complex domains. The application of the Baldwinian approach to problems arising in Machine Learning [Turney, 1996] shows that Baldwinian is a good strategy for such class of problems. Recently, experiments with a dynamic environment, where each individuals tness is decided by interactions with others and there is no explicit solution through generations, show that the Baldwinian approach pro-

Figure 2: An example of CCOP search space S . Points a; b; c are in the feasible search space F and points d; e in the infeasible search space S n F . vides outstanding performance [Arita and Suzuki, 2000]. We introduce here the Baldwin effect to handle feasibility in the complex domain of CCOPs. We experimentally illustrate its benets in the valued n-queens problem. 2.2 Combinatorial Optimization for Permutations In a Constraint Satisfaction Problem (CSP), the aim is to nd values for a (nite) set of variables satisfying a (nite) set of constraints [Tsang, 1999]. More formally [Eiben and Ruttkay, 1997], we consider a set X of n variables xi , where each variable xi may assume values vj from a given domain

Di fvj j
=

j ki g jDi j ki :
; =

Denition 1 (FSS) A Free Search Space is the Cartesian Product S = D1 Di Dn . Denition 2 (CSP) A Constraint Satisfaction Problem is a pair hS; i where S is a FSS and is a formula (Boolean functions on S ). A solution of a CSP is an s 2 S with (s) = true. Denition 3 (COP) A Constraint Optimization Problem is a triple hS; f; i, where S is a FSS, f is a (real valued) objective function on S and is a formula (Boolean function on S ). A solution of a COP is an s 2 S with (s) = true and an optimal f -value. Denition 4 (CCOP) A Constraint Combinatorial Optimization Problem is a COP where S is nite (typically because each Di is nite). Note we have added, to the original denition, the adjective Combinatorial to emphasize the fact that we are not dealing with constraint numerical optimization problems [Michalewicz, 1995]. Denition 5 (FeSS) For CSPs and CCOPs we call the feasibility condition and the set F = fs 2 S j (s) = trueg will be called the Feasible Search Space. The subclass of CCOPs (and CSPs) that can be formulated as permutation problems require solutions that can be thought of as assigning a permutation of the values to the variables [Smith, 2000]. More formally:

Denition 6 (PP) A Constraint Problem (CSP or CCOP) admits formulation as a Permutation Problem, provided that: D1 = : : : = Di = : : : = Dn , and to each variable xi a different value vj is assigned. In this case, the search space S consist of integer permutations = h 1 ; : : : ; n i. The n-queens problem is the best-known example of a permutation problem, although practical examples also occur in scheduling and allocation problems. 2.3 Solving Techniques In solving a CSP we may have particular interest in one of several types of answers [Smith, 1995]: 1. If no solution exist, we want proof that such is the case. 2. When a solution exist, we may want just one (no matter which one). 3. When a solution exist, we may want all solutions (a description of them in case there are exponentially many). 4. When we have an objective function, we want the solution that is optimal according to it (we recall that, when the problem at hand is combinatorial, we are dealing with a Constraint Combinatorial Optimization Problem). There are two basic strategies for solving CSPs [Tsang, 1999]: systematic search and repair. Systematic search tests all values for each variable. Examples of this strategy are the methods based on search trees (branch and bound, for example). At each step they assign a value to the current variable, and make sure that no constraint is violated. If this is not possible, they backtrack to revise previously assigned values. This process ends when a valid value has been assigned to every variable (or an exhaustive proof concludes no feasible assignment exist). These methods are able to obtain the four kinds of answers. Note, however, that if the CSP has n variables each with m possible values, the depth in the search tree is n and up to m branches are created from each node. Hence, the tree has up to mn leaf nodes, in consequence these methods are by nature exponential in time (even if the number of constraints is bounded by n and we assume evaluating a constraint or the objective function takes constant time). Repair strategies assign, at the same time, values to every variable, then repair such initial assignment until no constraint is violated. In contrast with systematic search, they are just able to obtain answers of type 2 (nd a feasible assignment). For answers of type 4, repair strategies, can not guarantee optimality but can typically provide acceptable solutions. Since a conventional GA does not perform a systematic search, it is considered as a repair strategy when dealing with CSPs or CCOPs.

2.4 GAs for Solving COPs and CCOPs CSPs and CCOPs have the common reputation of being GAhard [Eiben and Ruttkay, 1997]. Most of these have a feasible search space F that it is not easy to characterize (otherwise, such characterization is the type 3 answer). Typically it is also difcult to dene an operator !1 such that !1 (a) produces a feasible assignment, given an element a from the feasible search space F . It then results more difcult to dene !2 (a; b) to produce a feasible solution for a; b 2 F . Opt t erators !1 where the Markov chain !1 (a) = !1?1 (! (a)) samples F result in the application of Simulated Annealing, while operators !2 (a; b) may constitute themselves into the crossover operators and operators !1 (a) into mutation operators for a Genetic Algorithms (refer to Figure 2). The rst line of attack is naturally a genotype representation that enforces a large number of the constraints (minimizes the size of the set S n F of points of the search space that do not satisfy the constraints). In fact, if the encoding could represent exactly and uniformly the search space of feasible solutions F , the problem would become GA-easy (there would be no issue about genetic operators producing infeasible offsprings). Thus, a more computational denition of genotype is the structure that is the input and output of genetic operators and is maintained as representation of individuals in the simulated evolution cycle of the GA. Since S n F is typically non-empty, the usual next step is the transformation of feasibility from a qualitative concept into a quantitative concept. In the quantitative approach a function p : S ! <+ is applied to obtain the feasibility value of a given genotype. In the context of GAs, this function is called penalty function [Richardson et al., 1989] and attempts to measure how far d 2 S n F is from F (typically, a feasible genotype has no penalty, i.e. p(a) = 0 if a 2 F ). Approaches based on penalty functions differ in how the penalty function is designed and applied to infeasible genotypes [Michalewicz, 1995]. The evaluation function e, used by the GA to compute the tness of individuals in the population, is a combination of the penalty function p and the objective function f [Michalewicz, 1995]. Once the problem has been transformed (to handle feasibility as a quantitative concept), the basic GA mechanism can be enhanced in two ways to encourage feasible solutions: 1. Using operators that try to create children that are less infeasible than the parents [Paredis, 1993, Michalewicz, 1995, Ruiz-Andino et al., 2000]; that is, nd ! ( ; ) such that p(! (a; b)) p(a) and p(!(a; b)) p(b). 2. Fixing individuals if they are not feasible [Eiben et al., 1998]; that is nd ! ( ) such that p(! (a)) = 0. However, these two approaches can be consider very similar. Namely, if ( ; ) is the genetic operator in the GA plan that produces infeasible genotype, and we have a xing operator !( ) from 2 above, then !( ( ; )) is the genetic operator that creates children that are less infeasible as required by 1.

Also, a drawback of both of these approaches is that the attempt to land close to F although in S n F (rst case) or map S n F to F (case 2) could affect properties necessary of genetic operators (like assortment [Radcliffe, 1994]). Other problems of the rst case, are that general operators [Paredis, 1993, Ruiz-Andino et al., 2000] do not guarantee to produce only feasible solutions (the output of their operators may contain infeasible assignments, even if the input is feasible). Also, specialized operators are conceivable only for severely restricted contexts [Michalewicz, 1995]. In the second case, the challenge is the design of a xing mechanism. In fact, the approach is applied as a Lamarckian approach that modies genotypes. Lamarckianism enjoys particular popularity in the evolutionary computation community, because it has been relatively easy to x an infeasible genotype for some classical combinatorial optimization problems (e.g., traveling salesman problem, knapsack problem, set covering problem) [Michalewicz, 1995]. Unfortunately, for most CSPs and CCOPs such design is a really hard task. However, it is viable to build mechanisms that create partially feasible solutions [Eiben et al., 1998]. Our approach belongs to this second kind of enhancements, but in Baldwinian style, so we avoid the assortment problem. With a Baldwinian approach we do not need to change the original crossover and mutation operators.

Genotype Constructor

Phenotype Learner

Fitness-phenotype

Population

Figure 3: Our Baldwinian approach comprises two stages: Constructor and Learner. The tness-phenotype, returned by the Learner, is assigned to the original genotype. A remarkably additional benet of this approach is that it makes possible to incorporate problem specic knowledge by the use of a different version of constructor and/or learner. We use the word constructor to emphasize that the goal of this stage is moving a genotype d 2 S n F into a genotype closer to F . The generic operator constructor applied to a genotype d 2 S n F reduces the value of the penalty function; thus, p(d) p(constructor(d)): The name learner is applied to the local search that improves the objective function, since in Baldwinian or Lamarckian, it is assumed that it is the living phenotype who discovers and learns behaviors that make it tter. Our contribution is that, against previous applications of the Baldwinian approach, in the case of CCOPs, the constructor is crucial and perhaps more relevant than the learner. First, because without it, the search is costly, consuming resources on evaluating infeasible solutions d 2 S n F that are far from feasible solutions. The effort of the GA is really spent on trying to learn what is feasible, rather on what is best among the feasible, and tness evaluation is costly. Second, because it is easier to dene the constructor operator than the learner operator. Recall the contexts of CCOPs is such that it is hard to nd an operator ! : F ! F (that is, !(a) 2 F; 8a 2 F ). Thus it is typically difcult to dene the local search around the phenotype, and if found, it is because it involves a constructor, or it is again CPU-costly. 3.1 The Constructor Stage Since we are dealing with CCOPs that admit formulation as permutation problems, and we are interested in using a conventional GA framework, we represent the genotype as an integer permutation. Each position in the permutation corresponds to each variable. The content of each position is the value assigned to that variable. That is, S is the space of n! permutations. The constructor : S ! S then is an operator that reduces

3 Our Baldwinian Approach

The Baldwinian approach proposes to assess a genotype after the improvements from local search, but not to incorporate them into the genotype. When feasibility is not an issue, the obvious candidate to guide the improvement is the objective function [Whitley et al., 1994]. However, for Constraint Optimization Problems we additionally have the penalty function. The most common approach utilizes the combination of the objective function and the penalty function [Michalewicz, 1995]. Our thesis is that there are efciency reasons to guide the improvement in stages. Our Baldwinian approach operates in two stages. During the rst stage, which we called constructor, we set up an almost feasible phenotype from the genotype. In the second stage, named here learner, we improve this phenotype using the objective function. The purpose of the rst stage is producing feasible solutions in an efcient and effective way. In the second stage (learner) we may apply local search in the landscape of the evaluation function in order to compute phenotype tness. As prescribed by the Baldwinian approach, we assign the resulting tness value to the original genotype without affecting the genotype. Figure 3 displays the stages that our proposal comprises. The rationale behind our proposal is to integrate an effective and efcient mechanism, that produces almost feasible solutions, into a general search mechanism (the GA) that is particularly effective in dealing with hard optimization problems.

infeasibility with respect to p, a penalty function. Which penalty function p? If the feasibility criteria is decomposable, then it is typically possible to consider partial assignments (and partial solutions) [Paredis, 1993, Eiben and Ruttkay, 1997]. What does decomposable mean? For CCOPs, decomposable means we can separate constraints. More formally [Eiben and Ruttkay, 1997]. A partial assignment sp is an assignment of values in D to variables in X where some variables may not be assigned a value. Denition 7 (PS) A Constraint Problem (CSP or CCOP) is is given by a set of constraints C = decomposable if fc1 ; : : : ; cm g where all partial assignments sp can be checked for feasibility on those variables that have values. A partial assignment is a partial solution if all constraints involving variables with values are satised. Any assignment of values to variables is a partial assignment, and at least correspond to the empty partial solution. With a separable CCOP we can check separately the constraints involving variables and build sp . Namely, if all the constraints involving a variable are satised with its current value, we consider this variable assigned. However, if one of the constraints is violated we consider this variable to have its value not assigned. Then, we can count a ratio of constraints violated in an infeasible (total or partial) assignment and the penalty function is any monotone function p on the number of variables without specied variables. Alternatively, the penalty function can be a monotone function on the number of constraints not satised. Note than, in the resulting sp not all of the n variables are given values, only a fraction of them. Also constraints involving variables with assigned values are satisfactorily satised. In this case, the constructor is a local search that increases the number of constraints satised or increases the number of variables assigned a value that satises more constraints. These are efcient procedures, for which we mention two examples next. The rst example is a mechanism that iteratively attempts to complete a partially dened permutation [Hynek, 2000b]. We will refer to it as iterative prex extension for future reference and we now explain the algorithm. Let h 1 ; 2 ; : : : ; k i be a partially dened assignment (k < n), that encodes that the value of variable i is i for i = 1; : : : ; k ; but the value of variable i is not dened for k < i n. It is partial in the sense that all constraints involving the rst k variables are satised, and is a valid permutation in that f 1 ; 2 ; : : : ; k g f1; : : :; ng (and not two variables have been assigned a repeated value). The iteration is initialized by looking at a permutation (a genotype) and extracting the largest prex that constitutes a partial solution. The remaining genes of the genotype are kept in order in an initial proposals list. The body of the iteration attempts to extend the current partial solution by determining values for the (k + 1)-th variable and beyond. The appended values must be different from all previous values and each must satisfy the constraints of the corresponding variable. A scan is performed of the proposals

Partial assignment

Genotype

Extraction

Extension

proposals list

rejects list

Figure 4: The constructor stage extracts from the genotype the largest prex that constitutes a partial solution, the rest of the genotype is stored in a proposals list. The partial solution is extended with the scanned value in the proposal list if it satises the constraints, otherwise the value is appended to the rejects list. At the end of this stage we have set up an almost feasible phenotype. list, appending the next value in the proposals list to the partial solution if the corresponding constraints are satised, or otherwise appending it in a rejects list. When the proposals list is exhausted, the rejects list replaces it, and the body of the iteration is repeated. An empty rejects list or an iteration that does not reduce the proposals list (does not increase the partial assignment) signal termination of the procedure. In the latter case for termination, the rejects list could be appended to the partial solution to reconstruct a phenotype. Clearly, this constructor complies with what we have required. It will reduce the number of non-satised constraints. It typically is fast, since in the body of the iterations, constraints are checked for one variable. Although it can potentially require O(n2 ) constraint checks, we show in the case of study that: 1) it typically requires the same order ( O(n)) of effort than evaluating the objective function once; 2) and more importantly even if this may be more costly than evaluating the objective function, the savings in reducing evaluation of infeasible solutions compensate the effort. This is an important point in of our ideas. Our approach is even be more benecial when the evaluation of the objective function is more costly. Despite of the fact that the number of constraint checks performed by the constructor is potentially O(n2 ), it is worth the effort. We reduce the waste of resources on evaluating infeasible solutions. Our evolutionary search while not restricted to the feasible region, it is focused on it and its neighborhood. Figure 4 displays the rst type of constructor. Our second example of a constructor is based on using hill-climbing on the space of permutations with objective function the penalty function p. In generic terms, this is based on considering a family T of reversible transformations and the search space as a discrete graph. Two permutations are adjacent if there is a transformation in T that takes one into the other. Then, the current genotype is a node in a graph, and its

neighborhood is computed by applying the transformations to it. The penalty function p is evaluated in the neighbors and the one that results in a smaller number of unsatised constraints is adopted as the new permutation. The hill-climber stops when no neighbor suggests an improvement. A common example of a family of transformations is all pair-wise swaps. Unfortunately, this results in a quadratic number of neighbors for every node in the search space, so we can add the specics of a CCOP to make the constructor more efcient. For example, using the notion of a partial solution described before, it should be clear that swapping the values of two variables that already satisfy their constraints is more likely to increase the value of p, so we may consider swap only variables whose values violate constraints. Using domain knowledge of the valued n-queens problem this is the type of constructor proposed in [Sosi and Gu, 1994]. We c refer to this mechanism as restricted swap. Note that the constructor is not concerned with optimality of the objective function, their main interest is in producing feasible phenotypes instead. 3.2 The Learner Stage It is only after feasible phenotypes have been encouraged that we make use of the objective function. The learner applies an evaluation function e to obtain a tness value and uses local search in the landscape of e. For CCOPS, we advocate that this may be rather costly. Usually, for unconstrained problems, the evaluation function and the optimization function are the same. However, our evaluation function is a combination of a penalty function and the objective function. This is simply because the constructor may have not reduced p all the way down to zero. Also, since the objective function may have been dened only on F and not on S , some adjustments may be necessary. Different ways of combining p with f has been proposed [Richardson et al., 1989, Paredis, 1993, Michalewicz, 1995, Eiben and Ruttkay, 1997, Eiben et al., 1998, Ruiz-Andino et al., 2000], and several taxonomies of p has been suggested [Michalewicz, 1995, Eiben et al., 1998]. Unfortunately, the factors [Michalewicz, 1995] that inuence the design of e are problem dependent. In Section 4.2 we explain more in detail our evaluation function. Finally, since we are using the Baldwinian approach, the tness value is assigned to the original genotype (the input of the constructor).

25 16 20 18 1 27 23 21

16 9 15 27 5 12 1 7

20 15 16 31 30 18 23 26

18 27 31 5 16 29 10 12

1 5 30 16 6 30 25 23

27 12 18 29 30 9 22 26

23 1 23 10 25 22 26 30

21 7 26 12 23 26 30 0

n=8 optimum value=190

Figure 5: An example of a valued 8-queens problem and its optimum value. et al., 1992]. It is well known that for this problem there are many solutions when n > 3 (discounting the trivial case of one solution when n = 1). However, no closed-form formula has been found to compute the total number of solutions for the general case (experimental results show that the number of solutions grows exponentially as the size of the problem increases) [Erbas et al., 1992]. Moreover this problem is a kind of the maximal coverage problem and it has many practical scientic and engineering applications, amongst them: VLSI testing, communication systems, computer task scheduling, computer resource management, optical parallel processing, and data compression [Sosi and Gu, 1994]. c The n-queens problem is easily solved (in observed polynomial time) by repairing methods (specically by a localsearch mechanism [Sosi and Gu, 1994]), when just one soc lution is required. GAs are ineffective for this type of solution [Hynek, 2000b]. However, if our purpose is to obtain all solutions, a systematic search must be applied at the cost of exponential time. Former results for the n-queens problem, using GAs, apply mutation as the predominant operator [Crawford, 1992, Hynek, 2000a], or ad hoc asexual operators [Eiben et al., 1995]. However, they [Homaifar et al., 1992, Eiben et al., 1995, Hynek, 2000a] do not outperform local-search mechanisms and incur in a prohibitive computational effort [Crawford, 1992, Homaifar et al., 1992]. The valued n-queens problem is an extension of the n-queens problem. In this case each square of the chess board is weighted. The purpose is to nd a solution to the n-queens problem that maximizes the sum of the weighted squares occupied by the queens. If the global optimum is required, this problem must be solved by a systematic search, provided that the problem is small. Otherwise, with large problems, it can get extremely difcult to nd a solution [Smith, 1995]. Figure 5 shows an example of a valued 8-queens problem and its optimum value.

4 A case study. The valued n-queens problem

The n-queens problem consist of placing n queens on n different squares on a chess board with n rows and n columns, satisfying the constraint that no two queens can threaten each other (following chess rules that establish that a queen can threaten any other pieces on the same row, column or diagonal) [Tsang, 1999]. This problem has been studied for more than a century and there are many proposals to solve it [Erbas

For the valued n-queens problem specialized representation and genetic operators, based on constraint-solving techniques incurs in a high cost (execution time), that shows rapid growth when the size of the problem increases [Ruiz-Andino et al., 2000]. In this section we apply our proposed Baldwinian approach to deal with this problem. 4.1 Genotype Representation Letting i 2 f1; : : :; ng denote the column position of the queen in the ith row we ensure that the genotype h 1 ; : : : ; n i has no two queens on the same row. If all the i are distinct integers in f1; : : : ; ng then no two queens share a column [Sosi and Gu, 1994]. Thus, this representation now c popular [Crawford, 1992, Homaifar et al., 1992, Eiben et al., 1995, Hynek, 2000a] ensures all row and column constraints are satised but diagonal conicts may exist. This orderbased representation means that the GA search space (with size n!) consist of integer permutations (some of which are not feasible). 4.2 Fitness Evaluation Before we proceed to describe the mechanism of obtaining the tness value, it is convenient to mention how we deal with diagonal conicts. Diagonal conict are used to compute penalty functions. To calculate the number of diagonal conicts we apply the method proposed in [Crawford, 1992, Sosi and Gu, 1994]. This method allows the identication c of diagonal collisions in O(n) time by the construction of a pair of arrays, the positive and the negative array. These arrays have size 2n ? 1. Each time a queen is placed on row i and column i , a counter is incremented on the n + i ? i position of the negative array to account for this diagonal of negative slop. Also on the i + i ? 1 position of the positive array. Figure 6 displays the arrays (positive and negative) after a queen has been placed on a chess board with size 4 4. A non conictive placement of the n queens is achieved if the counter on each position of each array is 0 or 1. During our genetic search, a genotype is evaluated by the Baldwinian approach proposed in the previous section. The constructor is a combination of the local search mechanisms for nding a feasible solution described in Section 3.1. The particular constructor we have designed for the valued n-queens problem corrects an almost feasible solutions for the n-queens problem. It rst applies the iterative prex extension to a partial solution, and if it does not succeed it then continues with the restricted swap also described in that section. It should be mentioned that experimental evidence suggests that the iterative prex extension produces solutions for the n-queens problem with a very low number of diagonal conicts regardless of the problem size [Hynek, 2000b]. We emphasize that this is a very efcient local search since after placing a queen in location (i; i ), the number

n+i-j=4 - [0 0 0 1 0 0 0] j=2

i+j-1=3 + [0 0 1 0 0 0 0]

i=2

n=4

Figure 6: Positive and negative arrays after a queen has been placed on a chess board (row 2, column 2) with size 4 4. of diagonal conicts (dc) on each arrays is updated easily. It results easy to test if the new placement has enlarged the current partial solution for both, the iterative prex extension and the restricted swap search for a feasible solution. At the end of this constructor, with a serialization of two local searches for feasibility, we obtain a feasible solution for the n-queens problem with high probability. While we have no mathematical proof, our experiments show that the number of times the composed constructor we have described fails to produce a feasible solution is extremely low and decreases rapidly as the size of the problem grows. This is not surprising since we are using the two fastest and most effective strategies known to nd feasible solutions to the n-queens problem. For this particular case we use a hill-climber mutation operator as learner. We will give now a brief description of this operator. Although it is an unusual point of view, classical sorting algorithms can be conceptualized as localsearch mechanisms, where the local information provided by each comparison is enough to decide, in a deterministic way, the search direction. The sorting algorithm works as a map for the search space of n! permutations. SORT is a hill climber which follows the path traced by a sorting algorithm. SORT has been applied to the Error-Correcting Graph Isomorphism problem, which can be modeled as a permutation search problem, with remarkable success [Estivill-Castro and Torres-Vel zquez, 2001]. a We nally arrive to the evaluation of the tness function. Evaluation will typically just apply the evaluation function. Our implementation computes the tness of this phenotype as follows. A chess board can be represented as a matrix M with n n cells (squares) and the weights w associated i; j n. with each board location as w(i; j ), where 1 The permutation of values (the location of queens) is given by = h 1 ; : : : ; i ; : : : ; n i. The function to optimize is dened by

) =

n X w i; i : i=1
( )

Nevertheless, the possibility exist for an infeasible solution to reach this stage. In this case f only takes valid location of queens into account and a slightly different penalty function is applied. That is, f is evaluated in the partial solutions which is the largest subsequence of (the partial assignment with largest number of variables assigned). While during the constructor stage the penalty function was the length of the largest prex that causes no constraint conict, the penalty function we apply at this stage is based on computing the effort r of repairing the genotype [Michalewicz, 1995]. To obtain r, an initial value of 1 is increased by one each time the iterative prex extension mechanism adds an element to the rejects list and each time a swap is performed by the restricted swap mechanism. This penalty function is not applied to feasible solutions. The tness value is the given by

Parameter Representation Scheme Population Size Number of Generations Selection Scaling Crossover Mutation Crossover Probability Mutation Probability

Baldwin-GA Integer Permutation 20 100 Roulette Wheel Linear Scaling PMX SORT 0.8 0.2

Table 1: Baldwin-GA parameters.

Quality of Solutions Baldwin-GA 1

fitness f r :
= ( )

0.8

This tness value is assigned to the genotype that was the input to the constructor without applying the structural changes it may have been suggested by the constructor itself. 4.3 Genetic Operators Our proposal, named here Baldwin-GA, uses the operators and parameters reported in Table 1. Because PMX [Goldberg and Lingle, 1985], Roulette Wheel Selection and Linear Scaling [Goldberg, 1989] are well known in the GA community, we will skip details here. Hybridization with a hillclimber at the level of the tness landscape is achieved by the use of a mutation operator SORT [Estivill-Castro and TorresVel zquez, 2001], which has been briey described in Seca tion 4.2.

Correctness

0.6

0.4

0.2

0 4 6 8 10 12 Problem Size 14 16 18 20

Figure 7: Baldwin-GA produces good quality solutions for problem sizes between 4 and 20. 5.1 Execution Time and Quality of Solutions To compare the quality of our solutions with the optimal values obtained from methods that guarantee optimality, we contrasted with the well-known B&B method. B&B is the usual optimization method applied by constraint programming languages [Ruiz-Andino et al., 2000]. Because this method is exponential, we only report results with size n in the integer interval 4; : : : ; 20]. Quality is measured as a competitive ratio. We obtain an optimal value QB&B applying B&B , as well as the best value QBGA (after ten runs) applying Baldwin-GA. Q ?QBGA Then, we compute Quality = 1 ? B&B . This QB&B is the competitiveness ratio of the multi-start Baldwin-GA. The value of the competitive ratio is in the real interval 0; 1] (larger values means better quality). Figure 7 displays quality values obtained for each board size. The x-axis represents the problem size and the y -axis the quality obtained with Baldwin-GA. The lowest quality value (0.859) occurs when n = 19. We conclude that our Baldwin-GA produces good quality solutions along these sizes. The competitive ratios that indicate quality are also reported in Table 2 along with CPU-time requirements. Table 2 displays the average time over ten runs for Baldwin-GA. The contrast with the execution time required by the B&B algorithm is drastic. Even if we charge 10 times the average time to account for the multi-

5 Experimental Results
The benchmark of Ruiz-Andino et al. assigns integer values (weights) in the interval 0; 31] to each cell mi;j of the matrix M , independently and uniformly (each integer entry mi;j is chosen with uniform probability 1=32 and independently of other entries). In the construction of our test cases we apply this method. We compare Baldwin-GA with a branch and bound (B&B ) algorithm in terms of cost and quality of solutions. Implementation of Baldwin-GA is based on the software GAlib genetic algorithm package written by M. Wall at the Massachusetts Institute of Technology. The implementation of the B&B algorithm is totally our own. We rst show results in the trade-off of quality versus resource requirements (most importantly, CPU-time). We next show results that display the high success rate of the constructor stage in nding feasible solutions. This is reported as the proportion of feasible solutions produced by constructor over the number of its invocations. Last, we will report the benets obtained when using SORT mutation.

start, the Baldwin-GA is about 10 thousand times faster by the time the problem size is n = 20. In particular, the cost of our algorithm is less than one second for the tested problem sizes.

Proportion of Feasible Solutions Baldwin-GA 1

n
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

0.8

B&B : Time in seconds 0 0 0 0 0 0.01 0.03 0.25 0.53 1.93 2.47 17.63 56.9 84.37 1392.57 7073.9 21117.9

BaldwinGA: Time in seconds 0.024 0.029 0.041 0.048 0.054 0.067 0.082 0.097 0.112 0.137 0.159 0.172 0.195 0.215 0.245 0.261 0.29

BaldwinGA: Quality of solutions 1 1 1 1 1 1 1 1 1 0.958 0.943 0.870 0.913 0.989 0.874 0.859 0.879

Proportion

0.6

0.4

0.2

0 50 100 150 200 250 300 350 400 450 500 Problem Size

Figure 8: The proportion of feasible solutions produced by a Baldwin-GA increases as the problem size grows.
Benefits of using SORT Mutation 1400 Baldwin-GA-SORT Baldwin-GA-No-SORT

1200

1000 Fitness

800

Table 2: Comparison of a B&B algorithm with a Baldwin-GA in terms of execution time, using different sizes of the valued n-queens problem. We also report the quality of the solutions found by the Baldwin-GA.

600

400

15

20

25

30

35

40 45 Problem Size

50

55

60

65

5.2 Proportion of Feasible Solutions To assess the quality of the composed constructor we ran our algorithm three times for each size n 2 f50; 100; 150; : : :; 500g. Each time, we record the number phf of feasible phenotypes and the number phu of infeasible phenotypes produced by the invocations to the constructor. ph We compute the proportion phf +f u . Figure 8 displays the ph proportion of feasible phenotypes produced by Baldwin-GA for tested problem sizes. The x?axis represents the problem size n and the y -axis the proportion of feasible phenotypes. when a Baldwin-GA is applied. While for n < 200 the proportion of feasible solutions is lower than 0.5, starting with n >= 200 this proportion increases consistently and it almost reaches 1 by the time the problem size is 500. These results show that our constructor is effective in producing feasible solutions. 5.3 Benets of SORT Mutation We present results for four different sizes: 16, 32, 48 and 64. Ten runs were carried out for each size, using different random seeds. We compare two versions of our Baldwin-GA. The rst one uses SORT mutation [Estivill-Castro and TorresVel zquez, 2001], the second one applies random swapping. a

Figure 9: SORT mutation provides benets statistically signicant, even more when the problem size grows. The latter swaps two randomly selected variable assignments. The swap is carried out with a given mutation probability (dened as a parameter of the GA). This is a simple randomized swap mutation. A plot of the results are displayed in Figure 9. The x-axis is for the problem size. The y -axis represents the tness value (computed as described in Section 4.2). The plotted lines interpolate the average tness values over the 10 runs. The dotted line corresponds to results for Baldwin-GA without SORT mutation (labeled in the gure as Baldwin-GA-No-SORT), while the solid line corresponds to Baldwin-GA instantiated with SORT mutation (labeled as Baldwin-GA-SORT). This gure illustrates that a Baldwin-GA with SORT mutation performs much better. The improvement is statistically signicant across the entire range tested of problem sizes. We conclude that SORT mutation is consistently better.

6 Discussion
An important point derived from this work is the remarkably performance obtained when a Genetic Algorithm based on

the Baldwin effect is applied to the valued n-queens problem, a typical representative of Constraint Combinatorial Optimization Problems. Secondly, our approach facilitates the inclusion of problem specic knowledge by the use of different versions of constructor, and/or learners; while it benets from the use of a general and well-known framework (the order-based GA). Our approach is not a purely local search neither a simple mapping from genotype to phenotype. It includes a deterministic stage (the constructor) that does not perform backtracking. Although the constructor does not guarantee to deliver a feasible solution, it does achieve this with a high rate of success. This is more effective than guaranteeing the delivery of feasible solutions through backtracking or extensive random sampling. The diversity of the population in the GA guarantees that a feasible solution exists within the current population. Thus, even though we do not guarantee to create the whole phenotype space, its high success in producing feasible phenotypes keeps the problem of infeasibility under control. Actually, as a result of our evolutionary search, for the tested problem sizes, we always could obtain a feasible solution. It is unclear if Ruiz-Andino et al. approach can report results beyond partial solutions. We already mentioned our belief that one of the reasons for the better performance is the effectiveness of the constructor in producing feasible solutions. We also believe that another important reason is the separation of the feasibility criteria in a penalty function p from the objective function f to guide the constructor. The constructor is based on feasibility conditions alone. One of the difculties in a GA that is presented with a tness landscape that is the combination of feasibility and tness in a combined evaluation function e(p; f ) is its high dependency on the particular problem at hand. For example, it is conceivable to deal with an instance of the valued 4-queens problem as depicted in Figure 10. For this problem, no feasible solution s produces a better objective function value f (s) than any infeasible solution. Thus, a GA using e(p; f ) would have very serious difculties in compensating the values r(sp ) obtained for partial solutions sp with the penalty function p in order to nd a feasible solution. This is certainly an important reason to consider the constructor as proposed here. Finally, in literature there has been interest also in methods that reply with a partial solutions sp with a reasonable f (sp ) value despite feasible solutions exists and although sp can not be completed to a feasible solution. The interest for this comes because satisfying all the constraints may be impossible and it would take too long to verify this. For example, the problem at hand may have no solution and we are unaware of this (verifying this requires exponential time). Thus, it may be satisfactory to supply a partial solution rather than spend the time to check that no feasible solution exist. This may imply that the issue of feasibility has less relevance. In any case, our methods have a natural extension to this type of scenarios. For example, in the valued n-queens problem we may

20

14

17

19

23

0 n=4

Figure 10: An instance of the valued 4-queens problem where no feasible solution is better than any infeasible solution. prefer a method that places rapidly k < n of the n queens with no attacks rapidly and obtains a high f value even if it is impossible to place the remaining queens and retain feasibility. The constructor can potentially increase the value of f by extending the partial solution of k queens.

Acknowledgment
The second author was supported by a fellowship from the National Council of Science and Technology (CONACYT, M xico), grant number 135385. e

Bibliography
T. Arita and R. Suzuki. Interactions between learning and evolution: The outstanding strategy generated by the baldwin effect. In Proceedings of Articial Life VII, pages 196 205. MIT Press, 2000. K. D. Crawford. Solving the n-queens problem using genetic algorithms. In Proceedings ACM/SIGAPP, Symposium on Applied Computing, Volume 2, Technological Challenges of the 1990s, pages 10391047, 1992. A. Eiben, P.-E. Rau , and Z. Ruttkay. Ga-easy and gae hard constraint satisfaction problems. In Proceedings of the ECAI-94, Workshop on Constraint Processing, Lecture Notes in Computer Science 923, pages 267283. SpringerVerlag, 1995. A. E. Eiben and Z. Ruttkay. Handbook of Evolutionary Computation, chapter Constraint Satisfaction Problems. IOP Publishing Ltd. and Oxford University Press, 1997. A. E. Eiben, J. van der Hauw, and J. van Hemert. Graph coloring with adaptive evolutionary algorithms. Journal of Heuristics, 4(1):2546, 1998.

C. Erbas, S. Sarkeshik, and M. M. Tanik. Different perspectives of the n-queens problem. In ACM Annual Computer Science Conference. Proceedings of the 1992 ACM Annual Conference on Communications, pages 99108, 1992. V. Estivill-Castro and R. Torres-Vel zquez. Classical sorta ing embedded in genetic algorithms for improved permutation search. In Congress on Evolutionary Computation CEC2001. IEEE, 2001. D. E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley Publishing Company, 1989. D. E. Goldberg and R. J. Lingle. Alleles, loci, and the traveling salesman problem. In J. Grefenstette, editor, Proceedings of an International Conference on Genetic Algorithms and their Applications, pages 154159. Lawrence Erlbaum Associates, 1985. A. Homaifar, J. Turner, and S. Ali. The n-queens problem and genetic algorithms. In Proceedings IEEE Southeast Conference, volume 1, pages 262267, 1992. J. Hynek. An improved genetic algorithm for the n-queens problem. In Proceedings of the International Conference on Articial Intelligence IC-AI2000, pages 517522, Las Vegas NV, 2000a. J. Hynek. The n-queens problem revisited. In Proceedings of the ICSC. Symposia on Intelligent Systems and Applications (ISA2000. ICSC Academic Press, 2000b. Z. Michalewicz. A survey of constraint handling techniques in evolutionary computation methods. In Proceedings of the 4th Annual Conference on Evolutionary Programming, pages 135155. MIT Press, 1995. J. Paredis. Genetic state-space search for constrained optimization problems. In Proceedings of the 13th International Joint Conference on Articial Intelligence, volume 2, pages 967972. Morgan Kaufmann, 1993. N. J. Radcliffe. The algebra of genetic algorithms. Annals of Mathematics and Articial Intelligence, 10:339 384, 1994. J. T. Richardson, M. R. Palmer, G. Liepins, and M. Hilliard. Some guidelines for genetic algorithms with penalty functions. In J. D. Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, pages 191 197. Morgan Kaufmann Publishers, Inc., 1989. A. Ruiz-Andino, L. Araujo, F. S enz, and J. Ruz. A hya brid evolutionary approach for solving constrained optimization problems over nite domains. IEEE Transactions on Evolutionary Computation, 4(4):353372, November 2000.

B. M. Smith. A tutorial on constraint programming. Research Report 95.14, University of Leeds, School of Computer Studies, April 1995. B. M. Smith. Modelling a permutation problem. Research Report 2000.18, University of Leeds. School of Computer Studies, June 2000. Presented at the ECAI 2000, Workshop on Modelling and Solving Problems with Constraints. R. Sosi and J. Gu. Efcient local search with conict minc imization: A case study of the n-queens problem. IEEE Transactions on Knowledge and Data Engineering, 6(5): 661668, October 1994. E. Tsang. A glimpse of constraint satisfaction. Articial Intelligence Review, 13:215227, 1999. P. Turney. How to shift bias: Lessons from the baldwin effect. Evolutionary Computation, 4(3):271295, 1996. N. L. J. Ulder, E. H. L. Aarts, H.-J. Bandelt, P. J. M. van Laarhoven, and E. Pesch. Genetic local search algorithms for the travelling salesman problem. In First International Conference on Parallel Problem Solving from Nature, pages 109116. Springer Verlag, 1991. D. Whitley, V. S. Gordon, and K. Mathias. Lamarckian evolution, the baldwin effect and function optimization. In Y. Davidor, H. Schwefel, and R. Maenner, editors, International Conference on Evolutionary Computation, Parallel Problem Solving from Nature III, pages 615. Springer Verlag, 1994.