Using The Meta-Raps Approach To Solve Combinatorial Problems

Gail W. DePuy, Ph.D. Department of Industrial Engineering University of Louisville Louisville, KY 40292 USA Gary E. Whitehouse, Ph.D., P.E. and Reinaldo J. Moraga, M.Sc. Industrial Engineering and Management Systems Department University of Central Florida Orlando, FL 32816-2450 USA Abstract
This paper introduces an interesting meta-heuristic called Meta-RaPS (Meta-heuristic for Randomized Priority Search) for solving combinatorial problems . Meta-RaPS incorporates randomness within priority rules to construct a feasible solution at each iteration. In addition, Meta-RaPS includes improvement heuristics for enhancing the feasible solution already obtained. Applications discussed include: Traveling Salesman Problem, Resource Constrained Project Scheduling Problem, and Knapsack Problem.

Keywords
Meta-Heuristic, TSP, RCPSP

1.0 Introduction
Meta-RaPS (Meta-heuristic for Randomized Priority Search) is the result of research conducted at the University of Louisville and the University of Central Florida on the application of a modified COMSOAL approach to solve several types of combinatorial problems. COMSOAL is a computer heuristic originally reported by Arcus as a solution approach to the assembly line balancing problem [2]. Despite the fact that each modified version of COMSOAL conserves in essence the underlying Arcus's ideas, in practice, the versions discussed in [8] [9] [24] differ considerably from the original COMSOAL. This led to the development of "Meta-RAPS". The philosophy behind Meta-RaPS considers the following three basic ideas that have been supported throughout this research work: The incorporation of randomness in a particular heuristic may dramatically improve the solution quality. Random combinations of heuristics may lead to better results than each heuristic individually. The recent and ongoing increase in computer speed and capacity encourages the use of practical although computer-intensive methods for solving combinatorial problems. The application of Meta-RaPS to any combinatorial problem as a general methodology considers the following steps: 1. Study the structure of the problem to be solved 2. Find good priority rules that construct feasible solutions 3. Modify priority rules using randomness 4. Construct feasible solutions using priority rule and randomness 5. Pick the best answer for a number of iterations 6. Improve solutions

2.0 Application Study #1: The Traveling Salesman Problem

In order to illustrate the use of the Meta-RaPS procedure, the Traveling Salesman Problem (TSP) will be considered. The Traveling Salesman Problem (TSP), in its simplest form, involves finding the shortest distance route for visiting N cities and returning to the point of origin, where the inter-city distances are symmetric and known. Since each city is only visited once, the number of possible tours will be (N-1)! when the original city is given. Regardless of the

fact that TSP can be easily formulated, its nature exhibits all aspects of the NP-hard combinatorial optimization [27]. This is the reason why TSP has been intensively studied, because any improvement in finding TSP optimized solutions can be adapted to the entire class of NP-hard problems [27]. For a discussion of how to mathematically formulate a TSP, see [13]. The Meta-RaPS approach described in section 1.0 was applied to the TSP using the cheapest insertion (CI) heuristic as the priority rule. The CI heuristic starts with two randomly chosen cities. The method then finds the best insertion (i.e., the insertion into the current route that minimizes the total distance) for each unvisited city. The overall cheapest insertion is then made. Unvisited cities are evaluated in this manner until all cities have been placed in the tour. This simple TSP heuristic can be modified using Meta-RaPS to incorporate randomness. Meta-RaPS introduces randomness into the cheapest insertion heuristics using two parameters: %priority and %restriction. The %priority parameter defines the percentage of time the next city added to the tour is the city chosen by the simple heuristic (CI). The remaining time (i.e. 100 - %priority) the next visited city is randomly chosen from those cities that are within some distance (%restriction) of a city chosen by the simple heuristic. For example, if parameters of 20% priority and 40% restriction are used, then 20% of the time the next scheduled city is the cheapest insertion city and the remaining 80% of the time the next scheduled city is randomly selected from a list of those unvisited cities that are within a distance of 1.40 * (distance to the cheapest insertion city). This method is thought to decrease the possibility of arriving at solutions that are caught in local optima, and helps the user to find an answer closer to the global optimum. The node insertion heuristic is also included in the Meta-RaPS TSP solution technique to improve solutions. In addition to the two parameters previously discussed (%priority and %restriction), Meta-RaPS will now use the parameter %improvement for choosing whether or not to improve the tour. If a tours solution value before improvement is within %improvement of the best unimproved solution value, the node insertion improvement heuristic will be performed. The idea behind this strategy is that good unimproved solutions lead to good improved solutions. The Meta-RaPS TSP approach using cheapest insertion as the priority rule is outlined in Figure 1. The Meta-RaPS TSP technique shown in figure 1 is repeated a user specified number of iterations and the best solution is reported. The Meta-RaPS TSP technique described above was used to solve five 100-city problems found in TSPLIB [28]. Table 1 shows a summary of Meta-RaPS results for 10,000 iterations with 20% priority, 10% restriction and 5% improvement for comparison with the results of other researchers. Although it does not necessarily represent the best setting of parameters, this setting has proved to give fairly good results according to experimentation. As can be seen in Table 12, the solution quality of the Meta-RaPS approach compares favorably to the vast majority of the other methods. The shaded cells in Table 2 show those instances where Meta-RaPS was outperformed. Table 1 shows Meta-RaPS outperforms the other meta-heuristics such as neural networks, simulated annealing, tabu search, and genetic algorithms for these selected problems. An AMD Athlon microprocessor PC and Visual Basic 6.0, were used throughout this research. Run times for the 100-city KroA problem were appro ximately 16.5 minutes.

3.0 Application Study #2: Resource Constrained Project Scheduling Problem

In todays world of highly efficient, lean production systems, limited resources are a common occurrence that can significantly complicate project scheduling. To minimize a projects duration, project managers must not only consider the precedence relationships of activities, but also resource constraints. In most organizations, resources such as manpower, equipment, energy, and raw materials are limited. Often activities require a combination of these resources to be completed. The allocation of these multiple resources to competing activities further complicates the project scheduling procedure. Meta-RaPS proved to be a good tool to solve this constrained multiple resource allocation problem or resource constrained project scheduling problem (RCPSP) as it is often called. The Meta-RaPS RCPSP technique involved the use of only one parameter, %priority. As described in the previous section, this parameter is used to define the percentage of time the next activity scheduled was determined using a priority rule (in this case, ACTIM) and, therefore, the percentage of time the next scheduled activity was chosen randomly from the list of feasible unscheduled activities. To evaluate its solution quality, the results of the MetaRaPS heuristic were compared to the results of several other resource allocation heuristics for two well-known problem sets.

Start

Randomly select 2 initial cities and form initial tour

All cities in tour?

Yes

Do tour improvement? i.e. tourdist within %improvement of best unimproved tourdist?

No

End

No
Generate random number, X X = Rnd(1, 100)

Yes

Do tour improvement Use node insertion to improve tour

X %priority?

Yes

No
Do insertion of random city Randomly select an unvisited city, k Find best insertion for city k Randomly select a city k insertion position from all those positions within %restriction of bestposition Insert city k in tour

Do overall cheapest insertion Find cheapestcity and associated cheapestposition Insert cheapestcity in tour

Figure 1. Flowchart of single iteration of Meta-RaPS TSP Procedure

First, all 110 Patterson problems [26] were solved using Meta-RaPS and were then compared to the results of several other researchers for the 110 Patterson problems. The Meta-RaPS heuristic was run for 5000 iterations as this led to run times that were the same or less than the other heuristics reported run times of approximately 1 minute. A parameter of 25% priority was used. The results are presented in Table 2. The Table 2 results indicate the Meta-RaPS heuristic performed as well or better than other, more complicated RCPSP heuristics. The average run time for each of the 110 Patterson problems was less than 1 minute for all the heuristics shown in Table 2. The Meta-RaPS heuristic had an average run time of 35 real time seconds on a Pentium III personal computer. The Meta-RaPS heuristic was also evaluated using the PSPLIB [16] test set of resource allocation problems. As noted in Kolisch and Drexl [15], the Patterson problems seem to be much easier problems to solve than the PSPLIB problems. The PSPLIB contains sets of both 30 node and 60 node resource constrained project scheduling problems. For the 30 node PSPLIB problem Meta-RaPS heuristic led to statistically significantly smaller average percent differences from optimal than the Khattab and Choobineh [14], Brown [4], and Kolisch and Drexl [15] heuristics. Meta-RePS does not perform as well as the Brucker et al. [5] method for the 30 node PSPLIB problems. However it should be noted the branch and bound method presented by Brucker et al. [5] has a much longer average run time than the Meta-RaPS heuristic.

Meta-RaPS compares much more favorably with the Brucker et al. [5] results for the 60 node problems than the 30 node problems. The Meta-RaPS heuristic led to a much smaller maximum percent deviation from optimal and shorter run times for the 60 node problems, while the Brucker et al. [5] method had a higher incidence of finding the optimal solution. Table 1. Comparison of Meta-RaPS TSP results to the results of other TSP solution techniques. Problem Solution Methodology KroA KroB KroC KroD 0.00% 0.25% 0.00% 0.00% Meta-RaPS CI Cheapest Insertion & node insertion Christofides & 2opt 3opt * Convex Hull & 3opt * Farthest Insertion & 3opt * Nearest Neighbor &2opt 3opt * 2opt 3opt * LK (by Padberg and Rinaldi [25]) mod LK(by Mak and Morton [20]) GA by Chatterjee et al. [7], Exp A GA by Chatterjee et al. [7], Exp B Neural Net by Modares et al. [23] Neural Net by Matsuyama [22]** Neural Net by Burke and Damany [6]** Neural Net by Durbin and Willshaw [10]** Simulated Annealing by Malek et al. [21] Tabu Search by Malek et al. [21] Tabu Search & 3 opt by Tsubakitani and Evans [30] 0.50% 2.51% 0.37% 0.46% 0.14% 0.81% 0.26% 0.00% 0.70% 1.80% 0.31% 1.81% 5.00% 7.47% 0.09% 0.33% 1.37% 2.46% 1.40% 1.46% 1.57% 1.46% 1.44% 0.00% 0.17% N/A N/A 1.43% 3.37% 4.31% 9.04% N/A N/A N/A 0.82% 1.53% 1.06% 0.49% 1.06% 0.53% 0.70% 0.00% 1.78% 2.10% N/A N/A N/A N/A N/A N/A N/A 1.43% 0.17% 0.04% 0.45% 0.73% 1.74% 0.17% 0.00% 1.45% 1.30% N/A N/A N/A N/A N/A N/A N/A

KroE 0.17% 1.10% 3.03% 2.46% 0.98% 2.46% 0.18% 0.16% 0.21% N/A N/A N/A N/A N/A N/A N/A N/A N/A

Shaded cells indicate Meta-RaPS CI was outperformed * as reported by Lawler et al. [18] ** as implemented by Modares et al. [23]

Table 2: Comparison of the Meta-RaPS to Other Heuristic Results for all 110 Patterson Problems Average Std. Dev. Maximum Solution Method % Difference % Difference % Difference from Optimal from Optimal from Optimal Meta-RaPS Khattab and Choobineh [14] Bell and Han [3] Alvarez and Tamarit [1] Sampson and Weiss [29] Brown [4] Leon and Balakrishnan [19] Kolisch and Drexl [15] 0.35% 2.95% 2.60% 2.28% 1.98% 1.37% 0.74% 0.43% 1.20% 3.02% 3.10% 3.76% 2.62% 2.44% 1.41% 1.15% 6.45% 10.53% Not reported Not reported Not reported 13.89% Not reported 6.45%

Percentage Optimal Solutions 89.10% 39.09% 44.54% 57.27% 55.45% 68.18% 75.45% 85.45%

4.0 Application Study #3: Multidimensional Knapsack Problem

The Multidimensional Knapsack Problem (MKP) is a generalization of the 0-1 simple knapsack problem. The MKP consists of a knapsack with m constraints with restrictions b 1 , b2 , b3 ,..., b m , and n objects, each of which has a profit ci . The coefficients of the ith object in the MKP takes j values, 1 j m. The ith object has a coefficient of aij when it is considered for jth constraint. Typical constraints might include total weight, total volume, number of units, etc.

Eilon et al. [11] suggest a greedy algorithm for solving the MKP. This can be done by transforming all the constraints into just one constraint for computing the ratios and then solving an equivalent problem with the simple knapsack algorithm proposed by [17]. The items are added to the knapsack in order of their ratios; those items with the largest ratios are added first. The Meta-RaPS MKP technique makes use of 2 of the parameters introduced in the Meta-RaPS TSP section; %priority and %restriction. Therefore a portion of the time (%priority) the item with the largest ratio is added to the knapsack next and the remainder of the time the next item added to the knapsack is randomly chosen from those items whose ratios are within %restriction of the largest unassigned item ratio. The iteration ends when no items can be added to the knapsack due to a lack of resources. Table 3 presents results for 10,000 iterations of each of ten typical MKP problems [12] using parameters of 20% priority and 70% restriction. Run times were approximately 1 minute. As shown in Table 3, the Meta-RaPS approach of including randomness outperforms the Eilon et al. [11] MKP greedy algorithm. Table 3. Meta-RaPS Results for Typical MKP Problems # of # of % Deviation from Optimal Problem Items Constraints Using Eilon et al. [11] heuristic pb5 weing1 hp1 weish01 pet6 pb6 weish15 weish25 weish30 weing8 20 28 28 30 39 40 60 80 90 105 10 2 4 5 5 30 5 5 5 2 10.00 1.42 8.72 0.44 6.88 6.83 3.83 1.53 3.32 0.68

% Deviation from Optimal Using Meta-RaPS MKP 0.00 0.35 0.00 0.00 0.52 0.00 0.00 0.40 0.46 0.00

5.0 Conclusions
The main objective of this research was to develop a simple method to find good solutions to combinatorial problems. The Meta-RaPS heuristic, was developed and applied to typical combinatorial problems. Meta-RPS should be particularly attractive to the analyst because it is easy to comprehend, easy to put into practice, and gets good results in a reasonable amount of time even for large problems. The Meta-RaPS heuristics developed in this paper achieved good results when compared to both the optimal solution and other heuristics for the Knapsack Problem, Traveling Salesman Problem and the Resource Allocation Problem. The advantages of the Meta-RaPS heuristic over other heuristics include: (a)The run times of the Meta-RaPS heuristic are not dramatically affected by the size of the problem, (b) Meta-RaPS is easy to understand and easy to implement, (c)Meta-RaPS generates a feasible project schedule at every iteration. Therefore the method can be stopped at any time and a feasible solution is available, and Meta-RaPS produces results that are, in most cases, significantly better than the results of other heuristics. Future research is directed at applying the Meta-RaPS heuristic to other combinatorial problems such as the Vehicle Routing Problem, Assembly Line Balancing Problem, Machine Scheduling, Bin Packing and others. Efforts are also being directed at determining the best Meta-RaPS parameters and solving larger problems.

References
[1] Alvarez-Valdes, A., and Tamarit, J.M., 1989, Heuristic Algorithms for Resource Constrained Project Scheduling: A Review and Empirical Analysis, appears in Advances in Project Scheduling, Slowinski and Weglarz (eds.), Elsevier , Amsterdam, 113-134. [2] Arcus A., 1966, COMSOAL: A Computer Method of Sequencing Operations for Assembly Lines, I The Problem in Simple Form, appears in Readings in Production and Operations Management, Buffa E. (ed.), John Wiley & Sons, New York.

[3] Bell, C.E. and Han, J., 1991, A New Heuristic Solution Method in Resource Constrained Project Scheduling Problems, Naval Research Logistics, 38, 315-331. [4] Brown, J.T., 1995, Priority Rule Search Technique for Resource Constrained Project S cheduling, Ph.D. dissertation. Orlando, FL: University of Central Florida. [5] Brucker, P., Knust, S., Schoo, A., and Thiele, O., 1998, A Branch and Bound Algorithm for the ResourceConstrained Project Scheduling Problem, European Journal of Operational Research, 107, 272-288. [6] Burke, L., 1996, Conscientious neural nets for tour construction in the Traveling Salesman Problem, Computers and Operations Research, 23(2), 121-129. [7] Chatterjee S., Carrera C., and Lynch L., 1996, Genetic algorithms and traveling salesman problems, European Journal of Operational Research, 93, 490-510. [8] DePuy G., and Whitehouse G., 2000, Applying the COMSOAL Computer Heuristic to the Constrained Resource Allocation Problem, Computers and Industrial Engineering, 38, 413-422. [9] DePuy G., and Whitehouse G., 2001, A Simple and Effective Heuristic for the Multiple Resource Allocation Problem, International Journal of Production Research, 39(14), 3275-3287. [10] Durbin, R., Willshaw, D., 1987, An analogue approach to the traveling salesman problem using an elastic net method, Nature, 326, 689-691. [11] Eilon S., Watson-Gabdy C., and Christofides N., 1971, Distribution Management: Mathematical modelling and practical analysis, Hafner Publishing Company. [12] Heitktter, J. 2001, SAC-94 Suite of 0/1-Multiple -Knapsack Problem Instances, <http://elib.zib.de/pub/Packages/mp -testdata/ip/sac94-suite/index.html> [13] Ingnizio J., and Cavalier T., 1994, Linear Programming. Prentice Hall, New Jersey. [14] Khattab, M. and Choobineh, F., 1991, A New Approach for Project Scheduling with a Limited Resource, International Journal of Production Research, 29, 185-198. [15] Kolisch, R. and Drexl, A., 1996, Adaptive Search for Solving Hard Project Scheduling Problems, Naval Research Logistics, 43, 23-40. [16] Kolisch, R. and Sprecher, A., 1996, PSPLIB A Project Scheduling Problem Library, European Journal of Operational Research, 96, 205-216. [17] Lagoudakis M., 2001, The 0-1 Knapsack Problem: An Introductory Survey,ResearchIndex, The NECI scientific literature digital library<http://citeseer.nj.nec.com/ >, 8-9. [18] Lawler, E.L.; Lenstra, J.K.; Rinnooy Kan, A.H.G.; Shmoys, D.B., 1985, The Traveling Salesman Problem, John Wiley & Sons, New York. [19] Leon, V.J. and Balakrishnan, R., 1995, Quality and Adaptibility of Problem-Space Based Neighborhoods for Resource Constrained Scheduling, OR Spektrum. [20] Mak, K., Morton, A., 1992, A modified Lin -Kernighan traveling salesman heuristic, ORSA Journal on Computing, 13, 127-132. [21] Malek, M., Guruswamy, M., Pandya, M., Owens H., 1989, Serial and Parallel simulated annealing and tabu search algorithms for the traveling salesman problem, Annals of Operations Research, 21, 59-84. [22] Matsuyama, Y., 1992, Self-organization neural networks and various Euclidean traveling salesman problems, Systems and Computers in Japan, 23(2), 101-112. [23] Modares, A., Somhom, S., Enkawa, T., 1999, A self-organizing neural network approach for multiple traveling salesman and vehicle routing problems, International Transactions in Operational Research, 6, 591-606. [24] Moraga R., Whitehouse G., DePuy G., Neyveli B., and Kuttuva S., 2001, Meta-RaPS: an Efficient and Practical Approach for Solving the Traveling Salesman Problem, The 5th International Conference on Engineering Design and Automation, August 5-8, Las Vegas, NV, 191-195. [25] Padberg, M., Rinaldi, G., 1991, A branch-and-cut algorithm for the solution of large-scale traveling salesman problems, SIAM Review, 33, 60-100. [26] Patterson, J.H., 1984, Comparison of Exact Approaches for Solv ing the Multiple Constrained Resource Project Scheduling Problem, Management Science, 30, 854-867.. [27] Reinelt G., 1994, The Traveling Salesman: Computational Solutions for TSP Applications, Lectures Notes in Computer Science, Springer-Verlag. [28] Reinelt G., and Bixby W., 1995, TSPLIB--a library of travelling salesman and related problem Instances, <http://www.crpc.rice.edu/softlib/catalog/tsplib.html> [29] Sampson, S.E., and Weiss, E.N., 1993, Local Search Techniques for the Generalized Resource Constrained Project Scheduling Problem, Naval Research Logistics, 40, 365-375. [30] Tsubakitani, S. Evans, J.R., 1998, An empirical study of a new metaheuristic for the traveling salesman problem, European Journal of Operational Research, 104, 113-128.