On the Computer Implementation of Feasible Direction Interior Point Algorithms For Nonlinear Optimization

Jos e Herskovits and Gines Santos COPPE, Federal University of Rio de Janeiro Caixa Postal 68503, 21945-970 Rio de Janeiro, BRAZIL jose@com.ufrj.br
Abstract

We discuss the computer implementation of a class of interior point algorithms for the minimization of a nonlinear function with equality and inequality constraints. These algorithms use xed point iterations to solve Karush-Kuhn-Tucker rst order optimality conditions. At each iteration a descent direction is de ned by solving a linear system. In a second stage, the linear system is perturbed in such a way to de ect the descent direction and obtain a feasible descent direction. A line search is then performed to get a new interior point with a lower objective. Newton, quasi - Newton or rst order versions of the algorithm can be obtained. This paper is mainly concerned with the solution of the internal linear systems, the algorithms that we employ for the constrained line search and also with the quasi - Newton matrix updating. We also present some numerical results obtained with a quasi Newton Algorithm of the family. A set of test problems were solved very e ciently with the same set of values of the internal parameters.

1 Introduction
This paper is concerned with the problem of minimizing a function submitted to a set of equality and inequality constraints, when nonlinear smooth functions are involved. Engineering design is a natural application for this problem, since designers want to nd the best design that satis es all the requirements of feasibility. Calling x1 ; x2 ; :::; xn the design variables, f x the objective function, gix; i = 1; 2; ::; m the inequality constraints and hix; i = 1; 2; ::; p the inequality constraints, the optimization problem can be denoted minimize f x = subject to gx  0 and hx = 0; ; where g 2 Rm and h 2 Rp are functions in Rn.


1.1

Research partially developed at INRIA, Institut National de Recherche en Informatique et en Automatique, Rocquencourt, France.

This problem is said to be a Mathematical Program and, the discipline that studies the numerical techniques to solve it, Mathematical Programming, 15 . Even if mathematical programs arise naturally in optimization problems related to a wide set of disciplines that employ mathematical models, several physical phenomena can also be modeled by means of mathematical programs. This is the case when "the equilibrium" is attained at the minimum of an energy function. Feasible direction algorithms are an important class of methods for solving constrained optimization problems. At each iteration, the search direction is a feasible direction of the inequality constraints and, at the same time, a descent direction of the objective or an other appropriate function. A constrained line search is then performed to obtain a satisfactory reduction of the function, without loosing the feasibility. The fact of giving interior points makes feasible direction algorithms very e cient in engineering design, where functions evaluation is in general very expensive. Since any intermediate design can be employed, the iterations can be stopped when the cost reduction per iteration becomes small enough. There are also several examples that deal with an objective function, or constraints, that are not de ned at infeasible points. This is the case of size or shape constraints in some examples of structural optimization. When applying feasible direction algorithms to real time problems, as feasibility is maintained and cost reduced, the controls can be activated at each iteration. In this paper we discuss the numerical implementation of a class of feasible direction algorithms that uses xed point iterations to solve the nonlinear equations, in the primal and the dual variables, given by the equalities included in Karush-Kuhn-Tucker optimality conditions. With the object of ensuring convergence to Karush-Kuhn-Tucker points, the system is solved in such a way as to have the inequalities in Karush-Kuhn-Tucker conditions satis ed at each iteration. Based on the present general technique, rst order, quasi Newton or Newton algorithms can be obtained. In particular, the algorithms described in 10, 12 and the quasi Newton one presented in 11 . The present algorithm is simple to code, strong and e cient. It does not involve penalty functions, active set strategies or quadratic programming subproblems. It merely requires the solution of two internal linear systems with the same matrix at each iteration, followed by an inexact line search. In practical applications it can be taken advantage of the structure of the problem and of particularities of the functions in it to improve the calculus e ciency. This is the case of the Newton algorithm for linear and quadratic programming presented in 29 , the Newton algorithm for limit analysis of solids,  32  and the Newton algorithm for stress analysis of linear elastic solids in contact,  1, 31 . Several problems in Engineering Optimization were solved using the present method. We can mention applications in structural optimization,  13, 28 , uid mechanics,  2, 4, 5  and multidisciplinary optimization with aerodynamics and electromagnetism,  2, 3 . In the next section we discuss the basis of the method and then an algorithm for inequality constraints is presented. The following sections are devoted to the implementation of rst order, quasi - Newton and Newton versions of the algorithm, to the solution of the internal systems of equations and to the line search criteria and algorithms. Finally and algorithm for equality and inequality constraints is introduced and the numerical results on a set of test problems are shown.

2 The Feasible Direction Interior Point Method

To describe the basic ideas of these technique, we consider the inequality constrained optimization problem minimize f x 2.1 subject to gx  0; We denote rgx 2 Rnm the matrix of derivatives of g and call  2 Rm P the vector of dual 2 variables, Lx;  = f x + t gx the Lagrangian and H x;  = r2f x + m i=1 i r gi x its Hessian. Gx denotes a diagonal matrix such that Giix = gix. The corresponding Karush Kuhn Tucker rst order optimality conditions can be expressed as 2.2 Gx = 0 2.3 0 2.4 gx  0: 2.5 A Newton's iteration to solve the nonlinear system of equations 3-4 in x;  is stated as
+1 k k k k B rgxk  xk 0 , x = , rf x  + rg x  2.6 k +1 k t k k k k  rg x  Gx  Gx  0 , k +1 k+1 where xk ; k  is the starting point of the iteration and xk 0 ; 0  is the new estimate, B = k H xk ; k  and k a diagonal matrix with k ii  i . In our approach, we can also take B equal to a quasi-Newton estimate of H x;  or to the identity matrix. Depending on the way in which the symmetric matrix B 2 Rnn is de ned, 2.6 represents a second order, a quasi-Newton or a rst order iteration. However, to have global convergence, B must be positive de nite, see 14 . Iterations 2.6 are modi ed in a way to get, for a given interior pair xk ; k , a new interior +1 k estimate with a better objective. With this purpose, a direction dk xk 0 , x in the primal 0 =k+1 space, is de ned. Then, from 2.6, we have the linear system in dk 0 ; 0 
k k+1 k B k dk 2.7 0 + rg x 0 = ,rf x  k k+1 k rgtxk dk 2.8 0 + Gx 0 = 0; k It can be proved that dk 0 is a descent direction of f . However, d0 is not useful as a search direction since it is not necessarily feasible. This is due to the fact that as any constraint goes to zero, 2.8 forces dk 0 to tend to a direction tangent to the feasible set, see 14 . To obtain a feasible direction, by adding a negative vector in the right side of 2.8, we k+1 de ne the new linear system in dk and 
"  "  "  

rf x + rgx = 0

k+1 = ,rf xk  B k dk + rgxk  k+1 = , k k ; k rgtxk dk + Gxk  3

2.9 2.10

Figure 1: Search direction obtained by adding a negative vector to the right side of 2.8, where k 0, dk is the new k+1 is the new estimate of . We have now that dk is a feasible direction, since direction and  rgitxk dk = , k 0 for the active constraints. The inclusion of a negative number in the right hand side of 2.8 produces a de ection of k dk , 0 proportional to , in the sense of the interior of the feasible region. To ensure that d is a descent direction also, we establish an upper bound on k in order to ensure that
t k dk trf xk   dk 2.11 0  rf x ; 2 0; 1; which implies dk t rf xk  0. Thus, dk is a feasible descent direction. In general, the rate of descent of f along dk will be smaller than along dk 0 . This is a price that we pay for obtaining a feasible descent direction. k+1 To obtain the upper bound on k , we solve the linear system in dk 1 ; 1  k k+1 Bdk 1 + rg x 1 = 0 k k+1 k k rgtxk dk 1 + Gx 1 = , : k dk = dk 0 + d1 ; k t k  , 1dk 0  r f x  ; dk 1 t rf xk 

Since we have that, if

2.12 2.13

2.14

then 2.11 holds. To determine a new primal point, an inexact line search along dk is done, looking for a new interior point xk + tk dk  with a satisfactory decrease of the function. Di erent rules can be employed to de ne new positive dual variables k+1. In Figure 1, the search direction of an optimization problem with two design variables and one constraint is illustrated. At xk on the boundary, the descent direction d0 is tangent to the constraint. Even if in this example, we can prove that d1 is orthogonal to d0, in general we can only say that d1 is in the subspace orthogonal to the active constraints at xk . Since d1 points to the interior of the feasible domain, it improves feasibility. 4

3 A Basic Algorithm
Based on the ideas presented above, we state the following algorithm for inequality constrained optimization: FEASIBLE DIRECTION INTERIOR POINT ALGORITHM Inequality Constraints Parameters. 2 0; 1 and positive ' 0; ; and I : Data. Initialize x such that gx 0,  0 and B 2 Rnn symmetric and positive de nite. Step 1. Computation of the search direction d. i Solve the linear system for d0; 0

If d0 = 0; stop. ii Solve the linear system for d1; 1

Bd0 + rgx0 = ,rf x; rgtxd0 + Gx0 = 0:

3.1 3.2 3.3 3.4 3.5 3.6 3.7

iii If dt1 rf x 0; set Otherwise, set

Bd1 + rgx1 = 0; rgtxd1 + Gx1 = ,:

= inf ' k d0 k2 ;  , 1dt0rf x=dt1rf x : = ' k d0 k2 :

iv Compute the search direction

d = d0 + d1 :

Step 2. Line search. Find a step length t satisfying a given constrained line search criterion on the objective function f and such that gx + td 0. Step 3. Updates. i De ne a new x: x = x + td ii De ne a new : Set, for i = 1; m; i := sup 0i; k d0 k2 : 3.8 If gix  , and i I , set i = I . iii Update B symmetric and positive de nite. iv Go back to Step 1. We assume that the updating rules for B are stated in a way that: Assumption There are positive 1 and 2 such that
1

k d k2 dt Bd  2 k d k2 for any d 2 Rn:

5

The new components of  are a second order perturbation of the components of 0 , given by Newton's iteration 2.7. If and I are taken small enough, then after a nite number of iterations, i becomes equal to 0i for the active constraints. In Step 1 is bounded as in 2.14 and, in addition, we don't allow it to grow faster than d2 0. The theoretical analysis in 14 includes rst a proof that the solutions of the linear systems 3.1-3.2 and 3.3-3.4 are unique, provided that the vectors rhi x, for i = 1; 2; :::; p, and rgix for i 2 I x are linearly independent. Then, it is shown that any sequence fxk g generated by the algorithm converges to a Karush-Kuhn-Tucker point of the problem, for any way of updating B , if the previous assumption is true. We also have that xk ; k  converges to a Karush-Kuhn-Tucker pair x ; . Updating rules for B will be discussed in the next section.

4 Di erent Ways of Computing B

In the present method B can be equal to the second derivative of the Lagrangian, to a quasiNewton estimate or to the identity matrix. In this section, these alternatives will be discussed. FIRST ORDER ALGORITHMS. Taking B  I; we have an extension of gradient method. In particular, when all the constraints are active, the search direction becomes the same as the projected gradient, 15 . There are not theoretical results about the speed of convergence, but probably the rate is no greater than linear. Since the computer e ort and memory storage are smaller, this algorithm can be e cient in engineering applications that do not need a very precise solution. QUASI-NEWTON ALGORITHMS. In quasi - Newton methods for constrained optimization, B is an approximation of the Hessian of the Lagrangian H x; . Then, it should be possible to obtain B with the same updating rules as in unconstrained optimization, but taking rxLx;  instead of rf x. However, since H x;  is not necessarily positive de nite at a K-K-T point, it is not always possible to get B positive de nite, as required by the present technique. To overcome this di culty, we employ BFGS updating rule as modi ed by Powell, 27 : Take = xk+1 , xk and k k = rxLxk+1 ; k 0  , rx Lx ; 0 : If t 0:2 tB k ; then computed tBk 0 : 8 = t k B , t and take = + 1 , B k : 6

Set

B k+1 := B k +

t t

, B tB kB

k t k

In 14 it was proved that the convergence is two-step superlinear, provided that a unit step length is obtained after a nite number of iterations. NEWTON ALGORITHMS. To have a Newton algorithm we take B = H xk ; k  as in the iteration 2.6. However, as mentioned above, B must be positive de nite. Since this is in general not true, Newton versions of the present method can be obtained for particular applications only. This is the case of the algorithm for linear stress analysis with contact described in 1 .

5 The Internal Linear Systems

The matrices corresponding to the linear systems 3.5, 3.6 and 3.7, ?? solved in Step 2 are not symmetric neither de nite. However, in 25 it was proved that the solution is unique, provided the following regularity assumption is true at x: Assumption Regularity Condition For all x 2 gix = 0, are linearly independent.
a

the vectors rgix, for i such that

This condition must be checked in practical applications. There are several examples when the previous assumption is not true. In the case when there are sets constraints that take the same values due to symmetries of the problem, only one constraint of each set must be considered. It follows from 3.5 that:

d0 = ,B ,1 rf x + rgx0
and, by substitution in 3.6:

5.1 5.2

rgtxB ,1 rgx , ,1Gx 0 = ,rgt xB ,1rf x: rgtxB ,1 rgx , ,1Gx 1 = ,;

Then, instead of 3.5, 3.6, we can get 0 by solving 5.2, and d0 by substitution in 5.1. In a similar way, 1 can be obtained by solving

d1 = ,B ,1 rgx1: 5.4 Both systems, 5.2 and 5.3, with m equations has the same matrix. This one is symmetric and positive de nite when he regularity condition is true, 11 and 12 . However the condition number becomes worst as some of the components of  grows. To compute the matrices we need B ,1 or the products B ,1rgx and B ,1 rf x. In quasi - Newton algorithms, this can be ease obtained by working with an approximation of H ,1x. On the other side, it follows from 3.6 that
7

and d1 follows from:

5.3

0 = ,G,1 xrgtxd0 and, by substitution in 3.5, we get a system in d0: B , rgxG,1rgtx d0 = ,rf x Once d0 was computed 0 follows from 5.5. in a similar way, we have
and

5.5 5.6 5.7

1 = ,G,1xrgtxd1 , G,1 

B , rgxG,1rgtx d1 = rgxG,1 5.8 The linear systems 5.6 and 5.8 have he size of the number of variables n, they are symmetric and positive de nite. This is true, since B is positive de nite and rgxG,1rgtx is negative semide nite. When some constraint goes to zero the system becomes strongly bad conditioned.

6 Line Search Procedures for Interior Point Algorithms

Interior Point algorithms need a constrained line search. In the present method, once a search direction is obtained, the rst idea consists on nding t that minimizes f xk + tdk  subject to gxk + tdk   0. Instead of making an exact minimization on t, it is much more e cient to employ inexact line search techniques. For that, we have to state a criterion to know whether the step length is good or not and to de ne an iterative algorithm to obtain a good step length. In 15 we introduced inexact line search criteria based on Armijo's and on Wolfe's criteria for unconstrained optimization. ARMIJO'S CONSTRAINED LINE SEARCH De ne the step length t as the rst number of the sequence f1; ;  2;  3 ; :::g satisfying

f x + td  f x + t1 rf txd

and where 1 2 0; 1 and  2 0; 1 also.

6.1 6.2

gx + td  0;

WOLFE'S CONSTRAINED LINE SEARCH CRITERION Accept a step length t if 6.1 and 6.2 are true and at least one of the followings m + 1 conditions hold: rf t x + tdd  2 rf txd 6.3 and gix + td  gix; i = 1; 2; :::; m 6.4 where now 1 2 0; 1=2, 2 2 1; 1 and 2 0; 1. 8

Conditions 6.1 and 6.2 de ne upper bounds on the step length in both criteria and, in Wolfe's criterion, a lower bound is given by one of the conditions 6.3 and 6.4. A step length satisfying Wolfe's criterion for interior point algorithms can be obtained iteratively in a similar way as in 16 . Given an initial t, if it is too short, extrapolations are done until a good or a too long step is obtained. If a too long step was already obtained, interpolations based on the longest short step and the shortest long step are done, until the criterion is satis ed. Since the function and the directional derivative are evaluated for each new t, cubic interpolations of f can be done. As the criterion of acceptance is quite wide, the process generally requires very few iterations. A LINE SEARCH ALGORITHM WITH WOLFE'S CRITERION Parameters. 1 2 0; 0:5, 2 2 1 ; 1 and 2 0; 1: Data. De ne an initial estimate of the step length, t 0. Set tR = tL = 0 Step 1. Test for the upper bound on t. If, f x + td  f x + t1 rf txd and gx + td  0; Go to Step 2. Else go to Step 4. Step 2. Test for the lower bound on t. If rf tx + tdd  2rf t xd; or any gix + td  gix for i = 1; 2; :::; m; then t veri es Wolfe's Criteria, STOP. Else, go to Step 3. Step 3. Get a longer t. Set tL = t: i If tR = 0, nd a new t by extrapolation based on 0; t. ii If tR 0, nd a new t by interpolation in t; tR . Step 4. Get a shorter t. Set tR = t. Find a new t by interpolation in tL; t Return to Step 1.

7 Equality and Inequality Constrained Optimization

In what follows, we describe an approach to extend the present technique in order to solve the equality and inequality constrained problem 1.1. The resulting algorithm requires an initial point at the interior of the inequality constraints, not necessarily verifying the equality constraints. It generates a sequence fxk g also at the interior of the inequalities that converges to a Karush-Kuhn-Tucker point. In general, the equalities are only veri ed at the limit. Karush-Kuhn-Tucker rst order optimality conditions of problem 1.1 can be expressed as follows: 9

rf x + rgx + rhx = 0;

Gx = 0; hx = 0;
and

7.1 7.2 7.3 7.4

gx  0

  0: 7.5 where  2 Rp is the dual variables vector corresponding to the equality constraints. Now, the Lagrangian is Lx; ;  = f x + tgx + t hx, and its second derivative becomes p m X X H x; ;  = r2 f x + ir2 gix + ir2 hix. i=1 i=1 A Newton iteration for the solution of 7.1 to 7.3 is de ned by the following system:
2 6 6 6 6 6 4

B 6

rgtxk 

+1 k rgxk  rhxk  7 6 xk 0 ,x

3 2 7 7 7 7 7 5 6 6 6 6 6 4

Gxk 

0 0

k+1 ,
0

7.6 +1 ; k+1; k+1 a new estimate. with B = H xk ; k ; k ; xk ; k ; k  is the current point and xk 0 0 0 k+1 , xk ; then 7.6 becomes If we de ne dk = x 0 0
k k+1 k k+1 k Bdk 0 + rg x 0 + rhx 0 = ,rf x ; k k+1 k rgtxk dk 0 + Gx 0 = 0 k rht xk dk 0 = ,hx 

rht xk  0

+1 k k 0 ,

7 7 7 k  7 7= 7 5

rf xk  + rgxk k + rhxk k 7 6

6 6 6 6 6 4

3 7 7 7 7 7 5

Gxk k hxk 

7.7 7.8

7.9 which is independent of the current value of . As before, we can deduce that dk 0 is not useful as a search direction, since it is not always a feasible direction. When only inequality constraints are considered, as all the iterates are feasible, we know that a minimum is approached as the objective decreases. In the present problem, an increase of the function can be necessary to satisfy the equality constraints. Then we need an appropriate cost function for the line search. With this objective we consider the function
c x

and

= f x +

p X i=1

cikhixk;

7.10

where ci are positive constants. It can be shown that, if ci are large enough, then cx is an Exact Penalty Function of the equality constraints, 19 . In other words, there exists a nite c 10

such that the minimum of c subject only to the inequality constraints occurs at the solution of the problem 1.1. Then, the use of c as a penalty function is numerically very advantageous, since it does not require parameters going to in nite. On the other hand, c has no derivatives at points where there are active equality constraints. In 14 it is shown that, when xk veri es the inequality constraints and c is such that

sg hixk  ci + 0ik  0; i = 1; 2; ::; p;

7.11

where sg: = :=j:j: Then, dk 0 is a descent direction of c x. Following, we state an algorithm that uses c in the line search, but avoids the points where this function is nonsmooth. It generates a sequence with decreasing values of c at the interior of the inequality constraints and such that the equalities are negative. With this purpose, the system 7.7-7.9 is modi ed in a way to obtain a feasible direction of the inequalities that points to the negative side of the equalities and, at the same time, is a descent direction of c. FEASIBLE DIRECTION INTERIOR POINT ALGORITHM Equality and Inequality Constraints Parameters. 2 0; 1 and positive '; ; and I : Data. Initialize x such that gx 0, hx 0,  0, c 2 Rp and B 2 Rnn symmetric and positive de nite. Step 1. Computation of the search direction d. i Solve the linear system for d0; 0; 0

Bd0 + rgx0 + rhx0 = ,rf x;

rgtxd0 + Gx0 = 0; If d0 = 0; stop. ii Compute d1; 1; 1 by solving the linear system

rhtxd0 = ,hx

7.12

Bd1 + rgx1 + rhx1 = 0

rgtxd1 + Gx1 = ,; where e  1; 1; :::; 1 t. iii If ci ,1:20i, then set ci = ,20i; i = 1; :::; p: iv If dt1r cx 0; set Otherwise, set = min ' k d0 k2 ;  , 1dt0r cx=dt1r cx : = ' k d0 k2 : 11 7.13 7.14

rht xd1 = ,e;

v Compute the search direction Step 2. Line search. Find a step length t satisfying a given constrained line search criterion on the auxiliary function c and such that gx + td 0 and hx + td 0. Step 3. Updates. i De ne a new x: x = x + td ii De ne a new : Set, for i = 1; m; i := sup 0i; k d0 k2 : 7.15 If gix  , and i I , set i = I . iii Update B symmetric and positive de nite. iv Go back to Step 1. In the case when linear equality constraints are included and the initial point veri es them, it follows from 7.12 that d0 is feasible for those constrains. Then, if d0 is not de ected with respect to the equality constraints, they are always active.

d = d0 + d1 ;

8 Numerical Tests
We present in this section some numerical results obtained with a quasi Newton Algorithm. The line search considers Wolfe's criterion and makes cubic interpolations and extrapolations. That means that the objective function, the constraints and their derivatives are computed for each iteration in the line search. We report here our experience with 115 examples, described in 17 . The results are summarized in Tables 1, 2 and 3, where n = number of variables, m = number of inequality constraints, p = number of equality constraints, iter = number of iterations, evf = number of functions and gradients evaluations, cfv = computed objective function value, ofv = optimum function value,  17 . and sup jhij = maximum computed error of the equality constraints. The inequality constraints are feasible. All the problems were solved with the same set of values for the involved parameters, taken as follows: = 0:7, 1 = 0:1, 2 = 0:7, = 0:5, = 1 and c = 0.

12

Ex. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

n m p iter evf cfv ofv sup jh j 2 1 0 36 68 0.0000 0.0000 2 1 0 29 47 0.0504 0.0504 2 1 0 15 16 0.0000 0.0000 2 2 0 4 5 2.6667 2.6667 2 4 0 3 5 -1.9132 -1.9132 2 0 1 6 8 0.0000 0.0000 0.33D-13 2 0 1 10 11 -1.7321 -1.7321 0.33D-08 2 0 2 6 9 -1.0000 -1.0000 0.97D-08 2 0 1 5 6 -0.5000 -0.5000 0.17D-14 2 1 0 10 18 -1.0000 -1.0000 2 1 0 6 10 -8.4985 -8.4985 2 1 0 5 10 -30.0000 -30.0000 2 3 0 30 46 1.0000 1.0000 2 1 1 4 8 1.3935 1.3935 0.92D-08 2 3 0 5 7 306.5000 306.5000 2 5 0 10 12 0.2500 0.2500 2 5 0 23 31 1.0000 1.0000 2 6 0 13 16 5.0000 5.0000 2 6 0 19 39 -6961.8139 -6961.8139 2 5 0 21 22 38.1987 38.1987 2 5 0 15 17 -99.9600 -99.9600 2 2 0 8 15 1.0000 1.0000 2 9 0 7 14 2.0000 2.0000 2 5 0 11 20 -1.0000 -1.0000 3 6 0 92 104 0.0000 0.0000 3 0 1 11 15 0.0000 0.0000 0.71D-04 3 0 1 13 19 0.0401 0.0400 0.13D-05 3 0 1 3 5 0.0000 0.0000 0.00D+00 3 1 0 7 14 -22.6274 -22.6274 3 7 0 11 12 1.0000 1.0000 3 7 0 6 8 6.0000 6.0000 3 4 1 4 7 1.0000 1.0000 0.22D-15 3 6 0 12 20 -4.5858 -4.5858 3 8 0 18 34 -0.8340 -0.8340 3 4 0 5 7 0.1111 0.1111 3 7 0 12 15 -3300.0000 -3300.0000 3 8 0 10 14 -3456.0000 -3456.0000 4 8 0 16 29 0.0000 0.0000 4 0 2 12 13 -1.0000 -1.0000 0.16D-08 4 0 3 5 6 -0.2500 -0.2500 0.21D-10
i

Table 1: Numerical Results

13

Ex. 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 70 71 72 73 74 75 76 77 78 79 80

n 4 4 4 4 5 5 5 5 5 5 5 5 5 6 6 7 2 2 2 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 5 5 5 5

m 8 0 3 10 10 0 0 0 0 0 0 0 10 12 8 0 3 5 7 6 0 6 3 4 7 8 20 9 9 10 6 10 10 7 0 0 0 10

p iter evf cfv ofv 1 32 37 1.9259 1.9259 2 7 10 13.8579 13.8579 0 8 16 -44.0000 -44.0000 0 21 31 -15.0000 -15.0000 0 34 35 1.0000 1.0000 2 22 46 0.0000 0.0000 3 14 25 0.0000 0.0000 2 3 7 0.0000 0.0000 2 10 17 0.0000 0.0000 3 10 18 0.0000 0.0000 3 2 5 0.0000 0.0000 3 5 8 5.3266 5.3266 3 8 11 4.0930 4.0930 1 56 64 -0.8674 -0.9081 5 6 9 6.3333 6.3333 4 8 13 -3.4560 -3.4560 0 31 36 0.0285 0.0285 0 11 16 100.9901 | 0 12 14 -7.8042 -7.8042 1 5 8 0.0326 0.0326 2 8 11 -143.6461 -143.6461 1 5 9 -26272.51 -26272.51 2 4 7 961.7152 961.7152 0 31 43 6299.8424 6299.8424 0 8 13 0.9535 0.9535 0 5 9 0.5182 0.5182 0 118 263 -1162.00 -1162.0365 0 35 43 0.0075 0.0075 1 11 22 17.0180 17.0140 0 144 203 727.6794 727.6794 1 14 19 29.8944 29.8944 3 28 45 5126.4981 5126.4981 3 30 40 5174.4129 5174.4129 0 7 9 -4.6818 -4.6818 2 13 21 0.2415 0.2415 3 9 12 -2.9197 -2.9197 3 8 10 0.0788 0.0788 3 9 14 0.0539 0.0539

sup jh j 0.13D-16 0.70D-10

i

0.61D-07 0.17D-08 0.44D-15 0.00D+00 0.00D+00 0.00D+00 0.34D-15 0.33D-15 0.00D+00 0.36D-11 0.97D-07 0.33D-07 0.29D-11 0.11D-15 0.39D-06

0.12D-05 0.68D-14 0.34D-04 0.37D-08 0.15D-08 0.10D-08 0.38D-08 0.21D-09

Table 2: Numerical Results

14

Ex. 81 83 84 85 86 87 88 89 90 91 92 93 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 116 117 118 119

n 5 5 5 5 5 6 2 3 4 5 6 6 6 6 6 6 7 7 7 7 7 8 8 8 9 9 9 10 10 10 10 10 13 15 15 16

m 10 16 16 48 15 12 5 7 9 11 13 8 16 16 16 16 14 4 20 20 20 22 17 22 8 14 20 20 20 10 8 28 15 20 59 32

p 3 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 6 0 6 0 3 3 0 3 0 0 0 8

iter 15 13 32 172 13 60 6 7 10 12 10 13 4 5 3 3 14 13 39 39 37 23 57 119 12 21 47 5 107 18 17 118 35 31 45 54

evf cfv ofv sup jh j 25 0.0539 0.0539 0.17D-13 15 -30665.54 -30665.54 36 -5280335 -5280335 216 -1.9051 -1.9051 20 -32.3487 -32.3487 95 8927.6078 8927.5977 0.10D-05 15 1.3627 1.3627 27 1.3627 1.3627 24 1.3627 1.3627 39 1.3627 1.3627 33 1.3627 1.3627 24 135.0760 135.0760 7 0.0002 0.0156 11 0.0002 0.0156 6 0.0010 3.1358 6 0.0010 3.1358 22 -831080000 -831079891 0.41D-05 25 680.6301 680.6301 93 1809.7689 1809.7648 83 911.8806 911.8806 80 543.6680 543.6680 61 3.9512 3.9512 66 1138.4126 1138.4162 181 7049.3309 7049.3309 14 5055.0118 5055.0118 0.12D-10 30 -0.6749 -0.8660 56 5362.0693 5362.0693 0.43D-01 6 -45.7785 -45.7785 370 -47.7605 -47.7611 0.18D-05 26 -47.7611 -47.7076 0.46D-14 32 24.3063 24.3062 251 -1743.71 -1768.8070 0.34D-03 76 98.2996 97.5884 33 32.3487 32.3487 51 664.8204 664.8204 66 244.8997 244.8997 0.15D-13
i

Table 3: Numerical Results

15

9 Conclusions
The present algorithm is simple to code since it does not require the solution of quadratic programming subproblems but merely of two linear systems with the same matrix. We can take advantage of the structure of this matrix to solve e ciently very large problems. Depending on the information available about the problem to be solved, a rst order, a quasi - Newton or a Newton version of the algorithm can be employed. The numerical examples were solved very e ciently and, it is important to remark that, all of them were solved without any change in the code and with the same parameters of the algorithm.

References
1 Auatt, S. S., Borges, L. A. and Herskovits, J., An Interior Point Optimization Algorithm for Contact Problems in Linear Elasticity, Numerical Methods in Engineering'96, Edited by D esid eri, J. A., Le Tallec, P., O~ nate, E., P eriaux, J. and Stein, E., John Wiley & Sons, 1996. 2 Baron, F. J. and Pironneau, O., Multidisciplinary Optimal Design of a Wing Prole, Proceedings of STRUCTURAL OPTIMIZATION 93, Rio de Janeiro, Edited by J. Herskovits, 8 1993. 3 Baron, F. J. Constrained Shape Optimization of Coupled Problems with Electromagnetic Waves and Fluid Mechanics, in Spanish, University of Malaga, Spain, PhD Thesis, 1994. 4 Baron, F. J., Duffa, G., Carrere, F. and Le Tallec, P., Optimisation de forme en aerodynamique , in French, CHOCS, Revue scienti que et technique de la Direction des Applications Militaires du CEA, France, 1994. 5 Bijan, M. and Pironneau, O., New Tools for Optimum Shape Design , CFD Review, Special Issue, 1995. 6 Dew, M. C. A First Order Feasible Directions Algorithm for Constrained Optimization, Technical Report No. 153, Numerical Optimization Centre, The Hat eld Polytechnic, P. O. Box 109, College Lane, Hat eld, Hertfordshire AL 10 9AB, U. K., 1985. 7 Dew, M. C. A Feasible Directions Method for Constrained Optimization based on the Variable Metric Principle, Technical Report No. 155, Numerical Optimization Centre, The Hat eld Polytechnic, P. O. Box 109, College Lane, Hat eld, Hertfordshire AL 10 9AB, U. K., 1985. 8 Dennis, J. E. and Schnabel, R., Numerical Methods for Constrained Optimization and Nonlinear Equations, Prentice Hall, 1983. 9 Dikin, I. I., About the Convergence of an Iterative Procedure in Russian, Soviet Mathematics Doklady, Vol. 8, pp. 674 -675, 1967. 10 Herskovits, J., A Two-Stage Feasible Directions Algorithm for Non-Linear Constrained Optimization , Research Report No. 103, INRIA, BP 105, 78153 Le Chesnay CEDEX, France, 1982. 16

11 Herskovits, J., A Two-Stage Feasible Directions Algorithm Including Variable Metric Techniques for Nonlinear Optimization , Research Report n0 118, INRIA, BP 105, 78153 Le Chesnay CEDEX, France, 1982. 12 Herskovits, J., A Two-Stage Feasible Directions Algorithm for Nonlinear Constrained Optimization , Mathematical Programming, Vol. 36, pp. 19-38, 1986. 13 Herskovits, J. and Coelho, C.A.B. An Interior Point Algorithm for Structural Optimization Problems , in Computer Aided Optimum Design of Structures: Recent Advances, edited by C.A. Brevia and S.Hernandez, Computational Mechanics Publications, SpringerVerlag, June 1989. 14 Herskovits, J.,An Interior Point Technique for Nonlinear Optimization, Research Report No. 1808, INRIA, BP 105, 78153 Le Chesnay CEDEX, France, 1992. 15 Herskovits, J., A View on Nonlinear Optimization, Advances in Structural Optimization, Edited by J. Herskovits, Kluwer Academic Publishers, Dordrecht, pp 71-117, 1995. 16 Hiriart-Urruty, J. B. and Lemar echal, C. Convex analysis and Minimization Algorithms, Springer - Verlag, Berlin, Heildelberg, 1993. 17 Hock, W. and Schittkowski, K., Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems 187, Springer - Verlag, Berlin, 1981. 18 Kojima, M., Mizuno, S. and Yoshise, A., A Primal-Dual Interior Point Method for Linear Programming , Progress in Mathematical Programming: Interior-Point and Related Methods, Edited by N. Meggido, Springer-Verlag, New York, pp 29-47,1989. 19 Luenberger D.G., Linear and Nonlinear Programming , 2nd. Edition, Addison-Wesley, 1984. 20 Maratos N., Exact Penalty Function Algorithms for Finite Dimensional Optimization Problems , Ph.D. thesis, Imperial College of Science and Technology, London, 1978. 21 Mayne, D.Q. and Polak, E., Feasible Directions Algorithms for Optimization Problems with Equality and Inequality Constraints , Mathematical Programming 11, pp. 67-80, 1976. 22 Meggido, N., Pathways to the Optimal Set in Linear Programming , Progress in Mathematical Programming: Interior-Point and Related Methods, Edited by N. Meggido, Springer-Verlag, New York, 131-158, 1989. 23 Monteiro, R.D.C., Adler, I. and Resende, M.G.C., A Polynomial-Time PrimalDual A ne Scaling Algorithm for Linear and Convex Quadratic Programming and its Power Series Extension, Mathematics of Operations Research 151990, 191-214. 24 Panier, E.R.,Tits A.L., A Superlinearly Convergent Algorithm for Constrained optimization problems, SIAM J. Control Optim.,251987,pp.934-950. 17

25 Panier, E.R.,Tits A.L. and Herskovits J. A QP-Free, Globally Convergent, Locally Superlinearly Convergent Algorithm for Inequality Constrained Optimization, SIAM Journal of Control and Optimization, Vol 26, pp 788-810, 1988. 26 Powell, M.J.D. The Convergence of Variable Metric Methods for Nonlinearly Constrained Optimization Calculations , in Nonlinear Programming 3, edited by O.L. Mangasarian, R.R. Meyer and S.M. Robinson, Academic Press, London, 1978. 27 Powell M. J. D., Variable Metric Methods for Constrained Optimization , in Mathematical Programming - The State of the Art, Edited by A. Bachem, M. Grotschet and B. Korte, Springer-Verlag, Berlin, 1983. 28 Santos, G., Feasible Directions Interior Point Algorithms for Engineering Optimization , in Portuguese, DSc. Dissertation, COPPE - Federal University of Rio de Janeiro, Mechanical Engineering Program, Caixa Postal 68503, 21945-970, Rio de Janeiro, Brazil, 1996. 29 Tits, L. A. and Zhou J. L. A Simple, Quadratically Convergent Interior Point Algorithm for Linear Programming and Convex Quadratic Programming , Large Scale Optimization: State of the Art, Edited by W.W. Hager, D. W. Hearn and P.M. Pardalos, Kluwer Academic Publishers B.V,1993. 30 Vanderbei, R. J. and Lagarios, J. C., I. I. Dikin's Convergence Result for the A ne Scaling Algorithm, Technical Repport, AT&T Bell Laboratories, Murray Hill, NJ,1988. 31 Vautier, I., Salaun, M. and Herskovits, J., Application of an Interior Point Algorithm to the Modeling of Unilateral Contact Between Spot-Welded Shells, Proceedings of STRUCTURAL OPTIMIZATION 93, Rio de Janeiro, Edited by J. Herskovits, 8 1993. 32 Zouain, N.A., Herskovits J., Borges, L.A. and Feijoo, R. An Iterative Algorithm for Limit Analysis with Nonlinear Yield Functions, International Journal on Solids and Structures, Vol 30, No 10, pp 1397-1417, Gt. Britain, 1993.

18