AIAA 2009-4140

19th AIAA Computational Fluid Dynamics

22 - 25 June 2009, San Antonio, Texas

Performances of Numerical and Analytical Jacobians

in Flow and Sensitivity Analysis
A. A. Ezertas1 and S. Eyi.2
Middle East Technical University , Ankara,,06531, Turkey

The effects of flux Jacobian evaluation on flow and sensitivity analysis are studied. A cell
centered finite volume method with various upwinding schemes is used. A Newtons method
is applied for flow solution, and the resulting sparse matrix is solved by LU factorization.
Flux Jacobians are evaluated both numerically and analytically. The sources of the error in
numerical Jacobian calculation are studied. The optimum finite difference perturbation
magnitude that minimizes the error is searched. The effects of error numerical Jacobians on
the convergence of flow solver are studied. The sensitivities of the flow variables are
evaluated by direct-differentiation method. The Jacobian matrix which is constructed in the
flow solution is also used in sensitivity calculation. The influence of errors in numerical
Jacobians on the accuracy of sensitivities is analyzed. Results showed that, the error in
Jacobians significantly affects the convergence of flow analysis and accuracy of sensitivities.
Approximately the same optimum perturbation magnitude enables the most accurate
numerical flux Jacobian and sensitivity calculations.

I. Introduction
Recent advances in computer technology and solution algorithms allow efficient solution of very large linear
systems of equations. These advances have been motivating researchers to develop implicit algorithms to solve the
flow equations since usage of implicit methods is more beneficial compared to the explicit ones. Implicit flow
solvers are more stable and the residual can be reduced to very low values within a small number of iterations. The
equations of different disciplines can be strongly coupled with flow equations in an implicit algorithm. Another
advantage of implicit methods is that, sensitivities can be calculated very efficiently in design optimization.
Providing quadratic convergence, Newtons method is a widely used implicit technique for solving non-linear
flow equations. Newtons method requires the calculation of Jacobian matrices whose entries are the derivatives of
discretized residual vector with respect to the flow variable vector. Although the size of this matrix can be very
large, it is sparse in most of the flow problems. There are two methods available for calculating the Jacobian matrix:
analytical and numerical methods. Analytical evaluation is more accurate but the differentiation procedure needs
effort and it is time consuming1,2Moreover for each different flux discretization scheme, Jacobians are needed to be
recalculated. Numerical evaluation can be performed easily by finite differencing the residual vector. Although this
method is simple and independent from the complexity of the scheme, numerical evaluation has a lack of accuracy.
Errors in numerical Jacobian calculations may hamper the convergence of Newtons method. In numerical Jacobian
calculations, error strongly depends on the finite difference perturbation magnitude. In small magnitudes of
perturbation, condition error is dominant and in large magnitudes, truncation error becomes dominant. Hence the
error becomes large for both small and large perturbation magnitudes. In order to evaluate the most accurate
numerical Jacobian, the optimum perturbation magnitude should be used. As shown in Ref. 1, the optimum
perturbation magnitude for numerical Jacobian calculation can be accurately estimated with a simple formula.
In gradient-based aerodynamic design optimization, the derivatives of objective function with respect to the
design variables are needed. These derivatives are also called sensitivities. Sensitivities can be calculated by finite
difference or analytical methods. Although the finite difference method is easy to use, analytical method has more
advantages. In analytical method, sensitivities can be calculated more accurately and in a shorter CPU time. In
analytical methods, due to the implicit relation between flow and design variables, the evaluation of sensitivities is
not straightforward. There are two methods to calculate sensitivities analytically: direct-differentiation and by
1
2

Graduate Research Assistant, Aerospace Engineering Department.

Assoc. Professor, Aerospace Engineering Department
1
American Institute of Aeronautics and Astronautics

Copyright 2009 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

adjoint methods. In direct differentiation method, the discretized residual equations are differentiated with respect to
design variables, and the resulting equations are solved for flow variable sensitivities. In adjoint method, the
discretized residual equations are introduced as constraint functions, and the system of equations is solved for
adjoint variables. Both methods require the construction and the solution of Jacobian matrix. Therefore, using
Newtons method for flow analysis has advantageous in the calculation of analytical sensitivities. Jacobian matrix
which is constructed for flow analyses can also be used for sensitivity analyses. Sensitivities can be calculated very
efficiently by solving the LU decomposed Jacobian matrix with different right hand sides. In the solution of flow
equations with Newtons method the flux Jacobian is used as a driver to the iterative procedure. Even tough the
accuracy of flux Jacobian affects the convergence behavior of Newtons method; it does not affect the accuracy of
converged solution. However, in calculation of flow variable sensitivities, the flux Jacobian is the part of sensitivity
equation. Therefore the accuracy of the sensitivities strongly depends on the accuracy of the flux jacobian3.
This study mainly composes of two parts. In the first part, the accuracy of the numerically evaluated flux
Jacobian and its effect on the convergence of flow solution are studied. In the second part, the effects of the
accuracy of numerical flux Jacobians on the accuracy of the sensitivities are analyzed. Sensitivities are calculated by
direct differentiation method. The accuracy study is performed by comparing the results obtained from the
numerically and analytically evaluated Jacobians. To perform the study, cell centered finite volume code is
developed. The fluxes are computed by Steger Warming4, Van-Leer5,AUSM6 and Roe7 upwinding schemes. The
analytical Jacobians are evaluated by differentiating Steger-Warming, AUSM, Van Leer fluxes for both first and
second order discretizations. The sparse flux Jacobian matrix is LU factorized and solution is executed by using
UMFPACK sparse matrix solver. The boundary conditions are implemented implicitly. The study is carried out for
flows over the 10 percent circular arc, Ni bump8 geometry and the NACA0012 airfoil geometry.

II. Flow Model

The steady state 2-D Euler equations in generalized coordinates can be written in non-dimensional form as
below:

F (W ) G (W )
+
=0

(1)

W is the conserved flow variable vector and F and G are the flux vectors in , and directions:

u
W = J 1
v

et

uU + p
x

F = J 1
vU + y p

( et + p)U

uV + p
x

G = J 1
vV + y p

( et + p)V

(2)

where is the density, u and v are the components of the velocity vector, p is the pressure, et is the total energy per
unit volume, U and V are contravariant velocity components. In Equation (2), J is the coordinate transformation
Jacobian, , and are the curvilinear coordinates, and x , y , x , y are the transformation metrics.
The discretized form of the steady 2-D Euler equations given in Eq(1) can be written as:

F G
+
=0

(3)

For a cell centered finite volume method Eq (3) can then be written as:
( Fi +1/ 2, j Fi 1/ 2, j ) + (G i , j +1/ 2 G i , j 1/ 2 ) = 0

(4)

The inviscid fluxes of the Euler equations represent the convective phenomena. The upwind flux splitting
schemes are used for the spatial discretization of the flux vectors. In this study the splitted fluxes, F+ ,F- ,G+ and G-,
are calculated by Steger Warming, Van Leer, AUSM and Roe method.

2
American Institute of Aeronautics and Astronautics

Fi 1/ 2, j = F + (W iL1/ 2, j ) + F (W iR1/ 2, j )

+
L

R
G i , j 1/ 2 = G (W i , j 1/ 2 ) + G (W i , j 1/ 2 )

(5)

where the i1/2 and j1/2 denotes a cell interface. The fluxes are calculated at the cell interfaces by using the flow
variables interpolated from the cell center. The simple scheme with first order accuracy in space is obtained by
assuming the values at the cell faces are equal to the values at the nearest cell centers.
W iL+1/ 2 = Wi

W iR+1/ 2 = Wi +1

(6)

Higher order accuracy in space can be obtained by using MUSCL (Monotonic Upstream-Centered Scheme
Conservation Law) 9 scheme interpolation whose formulation is given below

1
W iL+1/ 2 = Wi + ( r ) [ (1 ) + (1 + ) ]
i
4
1
W iR+1/ 2 = Wi +1 ( r ) [ (1 + ) + (1 ) ]
4

(7)

i +1

In Eq(7) r equals to:

ri =

i
i

(8)

where forward and backward operators are defined as:

i = Wi +1 Wi

i = Wi Wi 1

(9)

MUSCL scheme requires the limiter function , in order to prevent oscillations and spurious solutions in regions of
high gradients. The limiter function reduces the slopes used in the interpolation of flow variables to the cell faces.
The parameter defines the order of the accuracy of the interpolation. For = -1, purely one sided upwind
interpolation; for = 0, linear interpolation between one upstream and one downstream point, for =1/3 three
point interpolation; are obtained with second order accuracy. For = 1 the upwind influence is lost and face values
are calculated by the arithmetic mean of the neighboring cells.
Using the Van Albadas limiter For = 0, and the Koren s limiter for =1/3 the resulting interpolation
formulation is given below:
W iL+1/ 2 = Wi + iL+1/ 2
W iR+1/ 2 = Wi +1 iR+1/ 2

For = 0

For = 1/3

( a 2 + )bi + (b 2 + )a
a 2 + b2 + 2

(2a 2 + )b + (b 2 + 2 )a
a 2 + b 2 ab + 3

3
American Institute of Aeronautics and Astronautics

(10)

(11)

(12)

where

aL = i

, bL = i

aR = i +1 , bR = i +1

(13)

is a small number which is used to prevent the activation of the limiter in the smooth regions of the flow domain.
is defined as 0.008 for Van Albadas limiter and as the square root of the cell area in 2-D problems for Korens
limiter.

III. Solution Method

The system of non-linear discretized governing equations can be written in the form:
R (W ) = 0

(14)

where R is the residual vector and is defined as

F (W ) G (W )
R (W ) =
+

(15)

Expanding R (W ) in a Taylor series about (n)th iteration and discarding high order (or nonlinear) terms yields :

R
R n +1 (W ) = R n (W ) +
W

where

n
W

(16)

R
is the Jacobian matrix. Solving above equation for R n +1 (W ) = 0 formulates Newtons Method as:

W
R

W

n
n
W = R(W )

(17)

The new values of flow variable vector W at the (n+1)th iteration can be calculated as:
W n +1 = W n + W n

(18)

IV. Flux Jacobian

In the solution of Euler equations with Newtons method, the evaluation of the flux Jacobian matrix is needed.
The entries of Jacobian matrix are the derivatives of the residual vector with respect to the flow variables vector. In
the calculation of these derivatives a finite difference method or analytical derivation method can be used, and the
resulting matrices are called numerical or analytical Jacobians, respectively.
A. Analytical Jacobian Derivation
Substituting Eq (5) into Eq (4) , the discretized residual vector can be calculated as:

4
American Institute of Aeronautics and Astronautics

Ri , j = F + 1 (W L ) + F 1 (W R) F + 1 (W L ) + F 1 (W R)
i+ , j
i+ , j
i , j
i , j
2
2
2
2

+ G + 1 (W L ) + G 1 (W R) G + 1 (W L ) + G 1 (W R)
i, j +
i, j +
i, j
i, j
2
2
2
2

(19)

Taking the derivatives of residual Ri , j with respect to a flow variable Wk ,l , the residual Jacobian defined as:
W i+L1/2, j
W i+R1/2, j +
W iL1/2, j
W iR1/2, j
Ri , j
= A i++1/ 2, j
+ Ai +1/ 2, j
Ai 1/ 2, j
Ai 1/ 2, j
Wk ,l
Wk ,l
Wk ,l
Wk ,l
Wk ,l
+ B i+, j +1/ 2

W i ,Lj +1/2
W i ,Rj+1/2
W i ,Lj1/ 2
W i ,Rj1/2
+ B i, j +1/ 2
B i+, j 1/ 2
B i, j 1/ 2
Wk ,l
Wk ,l
Wk ,l
Wk ,l

F +
A + =
W L

where,

+
F
+ = G
A =
B
,
W R
W L

(20)

G
, B =
W R

Analytical derivation of Residual Jacobian needs two sets of derivatives. The first set composes of the
derivatives of splitted fluxes with respect to the interpolated flow variables at cell faces, which

are A i +1/ 2, j , A
i 1/ 2, j , B i , j +1/ 2 , B i , j 1/ 2 . The analytical relations of these derivatives are function of flux schemes but
they are independent of order of spatial discretization. In this study, to evaluate these derivatives the StegerWarming, Van Leer and AUSM flux schemes are differentiated.
The second set consists of the derivatives of interpolated flow variables at cell faces with respect to the flow
variables at cell centers. The analytical relation of these derivatives varies according to order of spatial discretization
but they are independent of flux schemes. As shown in Eq(6), for the first order discretization right ( W R ) and left
( W L ) flow variables are equal to the values at the cell center of the two cells that are just located at the right and left
of the corresponding cell face. Therefore, in first order discretization k and l values in Eq(20) changes from i-1 to
i+1 and j-1 to j+1, respectively.

W iL+ 1

W iL 1

, j

= Wi , j

, W iR+ 1

, j

= Wi 1, j

, W iR 1

, j

= Wi +1, j

, j

= Wi , j

W iL, j + 1 = Wi , j

, W iR, j + 1 = Wi , j +1

W iL, j 1 = Wi , j 1

, W iR, j 1 = Wi , j

(21)

Jacobian matrices in first order discretization can be evaluated as:

Ri , j
= A i++ 1 , j A i 1 , j + B i+, j + 1 B i, j 1
2
2
2
2
W

(22)

i, j

Ri , j
= A i+ 1 , j
2
W
i +1, j

Ri , j
= B i, j + 1
2
W
i , j +1

Ri , j
= A i+ 1 , j
2
W
i 1, j

Ri , j
= B i+, j 1
2
W

5
American Institute of Aeronautics and Astronautics

i , j 1

(23)

where

F + i 1/ 2, j
A i+1/ 2, j =
,
W iL1/ 2, j

F i 1/ 2, j
G + i , j 1/ 2
G i , j 1/ 2
+

A i1/ 2, j =
,
B
=
,
B
=
,
1/
2
,
1/
2
i
j

i
j

W iR1/ 2, j
W iL, j 1/ 2
W iR, j 1/ 2

For the second order discretization, the flow variables at the cell faces are calculated from interpolation of the
flow variables at the center of the 4 neighboring cell using MUSCL method. Therefore, in second order
discretization k and l values in Eq(20) changes from i-2 to i+2 and j-2 to j+2, respectively. Thanks to the continuous
limiter functions used the MUSCL scheme is differentiable of flow variables. Hence analytical Jacobians are
evaluated without any difficulty for high order schemes..

(
(W

W iL R12 , j = W iL R12 , j Wi 1, j ,Wi , j , iL 1R2 , j

W iL+ R1 2 , j = W iL+ R12 , j

i, j

(
(W

, Wi +1, j , iL+ 1R2 , j

)
)

iL R , j = iL R , j Wi 2, j ,Wi 1, j ,Wi , j ,Wi +1, j

1

L R
i + 12 , j

L R
i + 12, j

i 1, j

, Wi , j , Wi +1, j , Wi + 2, j

)
)

(24)

Jacobian matrices in second order discretization can be evaluated as:

W i+L1/2, j W i+R1/2, j + W iL1/2, j W iR1/2, j
Ri , j
= A i+, j
+ Ai , j
Ai 1, j
Ai 1, j
Wi , j
Wi , j
Wi , j
Wi , j
Wi , j
+ B

+
i, j

W i ,Lj +1/2
W i ,Rj +1/2
W i ,Lj 1/2
W i ,Rj1/2
+ B i, j
B i+, j 1
B i, j 1
Wi , j
Wi , j
Wi , j
Wi , j

(25)

W i+L1/2, j W i+R1/2, j + W iL1/2, j W iR1/2, j

Ri , j
= A i+, j
+ Ai , j
Ai 1, j
Ai 1, j
Wi 1, j
Wi 1, j
Wi 1, j
Wi 1, j
Wi 1, j

(26)

W i ,Lj+1/2
W i ,Rj +1/2
W i ,Lj1/2
W i ,Rj 1/2
Ri , j
= B i+, j
+ B i, j
B i+, j 1
B i, j 1
Wi , j 1
Wi , j 1
Wi , j 1
Wi , j 1
Wi , j 1

(27)

W i+L1/2, j
W i+R1/2, j
Ri , j
= A i++1/ 2, j
+ Ai +1/ 2, j
Wi + 2, j
Wi + 2, j
Wi + 2, j
W iL1/2, j
W iR1/2, j
Ri , j
= A i+1/ 2, j
Ai 1/ 2, j
Wi 2, j
Wi 2, j
Wi 2, j
W i ,Lj+1/2
W i ,Rj +1/2
Ri , j
= A i+, j +1/ 2
+ Ai , j +1/ 2
Wi , j + 2
Wi , j + 2
Wi , j + 2

(28)

W i ,Lj 1/2
W i ,Rj 1/2
Ri , j
= A i+, j 1/ 2
Ai , j 1/ 2
Wi , j 2
Wi , j 2
Wi , j 2
The main advantage of the analytical method is that the residual Jacobian can be calculated accurately. The order
of error in the analytical method can be as small as the round-off error. Although the analytical method requires
code development, run time of an analytical code is short. However, as the complexity of the discretized residual
equations increases, the derivation of the analytical Jacobian becomes more complicated.

6
American Institute of Aeronautics and Astronautics

B. Numerical Jacobian Calculation

Another alternative for Jacobian evaluation is to compute the Jacobian numerically. Using a small finitedifference perturbation magnitude , the numerical Jacobian can be calculated by the forward-difference method as
follows10

R m R m (W ) R m (W + en ) R m (W)
=
=

Wn
Wn

(29)
where m = 1, mmax and

n = 1, (mmax + nbound )

where, R m is the mth component of the residual vector and the Wn is the nth component of the flow variable vector,
en is the nth unit vector. The value of the nth component of the unit vector en is one, and all other components are
zero. The size of the residual vector is defined by mmax, which equals to 4 times the number of interior cells in 2D
flow problems. The size of the flow variable vector is larger as much as the number of boundary cells, nbound. In
the numerical method, Jacobian evaluation does not require the large coding effort as needed in the analytical
method. The same residual discretization is used for both the original and perturbed flow variables. This reuse of the
same code is one of the important advantages of the numerical approach. Moreover, for cases in which the analytical
derivation is difficult, numerical Jacobian can be obtained without any difficulty.
The inaccuracy and long computation time are the two main disadvantages of the numerical Jacobian evaluation.
The error in numerical Jacobian is function of the finite-difference perturbation magnitude. The accuracy of the
numerical jacobians can be improved with the usage of an optimum perturbation magnitude that minimizes the total
error in the finite difference evaluation. The main reason that causes long computation time is the necessity of the
residual vector calculation with each perturbed flow variable in the whole domain. For a given cell, the residual is
only a function of flow variables in that cell and the neighboring cells according to the discretization used. In order
to reduce computation time, the perturbed residual is computed only with flow variables in these cells. For firstorder discretization, in addition to the cell in which the flow variable is perturbed, four neighboring cells are used.
Considering four flow variables in each cell, 20 perturbed residual vector evaluations are required for the given cell.
In second-order discretization, using eight neighboring cells in addition to the given cell, 36 perturbed residual
vector evaluations are required. Although the speed and the accuracy of the analytical method may not be reached,
the numerical Jacobian evaluation method may become faster and more accurate with some precautions.

C. Accuracy of Numerical Jacobians

Error Analysis
In numerical Jacobian calculation, mainly two types of errors occur. These are truncation and condition errors12.
While truncation error is due to neglected terms in the Taylors series expansion, condition error is caused by loss of
computer precision. In Eq(30), the truncation error due to the neglected terms in the Taylor series expansion can be
written as:
ETrun ( ) =

2 R m ( )
W 2 2

(30)

where =[ Wn , Wn + ].

( )

Because of computer precision, the exact values of the mth components of the vector R m W and their computed

( )

value R m (W ) can be different due to round-off error Erm W :

R m (W ) = R m (W ) + Erm (W )
R m (W + ) = R m (W + ) + Erm (W + )

7
American Institute of Aeronautics and Astronautics

(31)

By using the computed function, R m (W ) , Eq(29) can be written as:

R m (W ) R m (W + en ) R m (W)
=

Wn
Er (W + e ) Er (W)

R m (W ) Rm (W + en ) Rm (W)
n
m
=
+ m

Wn

(32)

R m (W ) Rm (W )
=
+ Ecm ( )
Wn
Wn

() is the condition error. Considering a bound of round-off error E = max Er (W ) , Er (W + )

where Ec
R

} the

maximum of the condition error can be approximated as:

Ec( ) =

2 ER

(33)

This bound of round-off error can be considered the precision error, which depends on the computer processor and
compiler. For the computations of normalized variables where the magnitude of the computed values are around
one, the precision error equals to machine epsilon M . A reasonable estimate of M can be given as follows:
M =

1
2m

such that

1 + M > 1

(34)

where m is the number of possible highest bits in the binary representation of the mantissa. The machine epsilon
M values of the compiler-computer configuration are found according to Eq(34). In this study, for single precision
M 1.19 107 , and for double precision M 2.2 1016 values are reached.

Optimal Perturbation Magnitude Analysis

The error in numerical Jacobian is highly dependent on perturbation magnitude, . For the small values of
perturbation magnitude the condition error grows up and dominates the total error. On the other hand as the
magnitude of perturbation gets larger, the truncation error becomes dominantly larger. Hence, there should be an
optimal value for the perturbation magnitude that minimizes the total error in numerical Jacobian. The error in the
numerical Jacobian matrix equals to the sum of the errors in each matrix element. To minimize the error, each finite
difference computations can be performed with their own optimum perturbation magnitudes. However this approach
to minimize error will be highly impractical due to its cost. Alternative way is to find an optimum perturbation
magnitude which minimizes the global total error arising from the finite differencing of each element. This can be
achieved by evaluating the magnitude which satisfies the least squares minimization of the error. The formulation
for the square of the total numerical Jacobian error is given below:

( ETOTAL ( ) )

mmax 4( neigh +1)

m =1

nn =1

2 ERm 2 Rm ( )
+

2
wnn
2

(35)

In the Eq(35) the outer summation loops are constructed for the whole domain, excluding the ghost boundary
cells where the residuals are not computed. The number of neighboring cells which are related to the residual vector
is represented by neigh; it has a value of 4 in 1st order discretization and 8 in 2nd order discretization. The least
squares minimization is performed by differentiation of the Eq(35) with respect to perturbation magnitude.

8
American Institute of Aeronautics and Astronautics

 ( ETOTAL ( ) )

mmax 4( neigh +1)

m =1

( ETOTAL ( ) )

( ETOTAL ( ) )

2 mmax 4( neigh +1) 2 ERm 2 Rm ( ) 2 ERm 2 Rm ( )

+

2
2
m =1 nn =1
2
2
wnn
wnn

2
4 ER2m
2 mmax 4( neigh +1) 2 Rm ( )

m =1 nn =1 wnn2 4
2

nn =1

2
2
2 ERm Rm ( ) 2 ERm Rm ( ) 1
+

2
2
2
2 2
2
wnn
wnn

(36)

The optimum which minimizes the total error is found by equating Eq (37) to zero
( ETOTAL ( ) )

=0
= OPT

(37)

2 m max 4( neigh +1) 2 Rm ( )

4

m =1

nn =1

4
2

2
nn

m max 4( neigh +1)

(E )
m =1

Rm

nn =1

=0

The optimum perturbation magnitude can be evaluated as:

( 4(neigh + 1) ) ( ER
m max

OPT = 2

m =1

m max 4( neigh +1)

m =1

nn =1

2 Rm ( )

2
wnn

(38)

Using the definition of the L2 norm, above equation can be simplified into the following form:
4(neigh + 1) ER

OPT = 2

R ( )

2
w

(39)

Substituting above optimum perturbation magnitude into the Eq(35) the minimum error is evaluated as follows:
ETOTAL ( OPT ) = 2 4(neigh + 1) ER

m max 4( neigh +1)

2 R ( )
2 Rm ( )

+ 2 ERm
2
2
wnn
m =1
nn =1
w 2

(40)

Derived equations show that to estimate the numerical Jacobian with minimum error requires the calculation of
second derivatives of residual vector with respect to flow variables and the round-off error. The round-off error in
single precision can be estimated as the difference in residual vectors that are calculated with single and double
precisions whereas the estimation of the round-off error in higher order precision calculations may not be easy. The
evaluation of second derivatives can be employed by finite difference relation. Although the evaluated values may
not be very accurate they may be useful for estimating the order of magnitudes.

9
American Institute of Aeronautics and Astronautics

Approximating Eq(39) for an arbitrary norm of Lp gives:

4(neigh + 1) ER

OPT = 2

2 R ( )

2
w

(41)

The solution of CFD problems are commonly performed with normalized variables. Therefore assumption of the
round-off errors to be equal to the machine epsilon may be reasonable. Calculation of second derivatives may not be
easy. Therefore in most of the practical calculation, the order of second derivatives can be assumed as unit. These
approximations will result in a simple formula which can be used to estimate optimum perturbation magnitude.

OPT = 2

4(neigh + 1) M

ETOTAL ( OPT ) = 2 4(neigh + 1) M

(42)

The flux Jacobians are evaluated using wide range of perturbation magnitudes and the optimum value obtained
from trial error procedure was compared with corresponding results of Eq (42).

D. Structure of Jacobian Matrix and Solution Strategies

The most of the entries in the Jacobian matrix equal to zero since the discretized residual equations only depend
on flow variables in neighboring cells. The Jacobian is a square matrix whose dimensions equal to the total number
of flow variables. In the first order spatial discretization, five stencils are required and Jacobian is a block diagonal
matrix that is made up of five 4x4 blocks. In second order discretization, nine stencils are required and the matrix is
made up of nine 4x4 blocks. As a result whole matrix entries, except those block bands and boundary entries equal
to zero. Although there is no need to compute and store the zero elements of the Jacobian matrix, full matrix solvers
require the whole matrix to be constructed which is computationally expensive. The sparse matrix solvers that store
only the non-zero elements in the matrix should be used to overcome the high cost of whole matrix storage.
In this study, the UMFPACK (Unsymmetric-pattern MultiFrontal PACKage) sparse matrix solver package11 is
used in order to solve the linear system of equations. In this method, the full matrix is converted into sparse storage
mode and then factorized using a sequence of small dense frontal matrices by LU factorization. The usage of a
sparse matrix solver increases the efficiency of the flow solver significantly.
Newtons method requires a good initial guess for convergence. This is a considerable drawback. In this study,
flow variables are initialized with their free-stream values, although it may be a poor guess. Several ideas are
available to modify Newtons method to improve stability. For example, a time-like term can be added to the
diagonal of the Jacobian matrix to make it more diagonally dominant. Increased diagonal dominance leads to a more
stable linear solution. With the addition of a time-like term, the modified Newtons method becomes:
1
R
[ I ] +
W
t

n
n
W = R(W )

(43)

The original Newtons method can be constructed as t . In the modified Newtons method, a small initial
value t0 is chosen and a new value of t can be obtained using L2-norm of the residuals as
t n = t 0

R(W 0 )
2

R (W n )

(44)

In this study, to improve the convergence from free-stream initial conditions, the modified Newtons method is
used. For all calculations the L2-norm is computed including all the residuals in the domain. The convergence of this
10
American Institute of Aeronautics and Astronautics

method is slow until t gets very large. However, the modification in Newtons method is useful in the early stages
of iterations. Later, the solution becomes more accurate, and the diagonal term addition may not be needed. The
withdrawal of the diagonal term from the matrix at the proper convergence level significantly reduces the number of
iterations and CPU time. Initial and the withdrawal values of diagonal term that gives the best convergence
performance are chosen by trial-error.

V. Effects of Flux Jacobian Evaluation on Flow Solution

The accuracy of numerical Jacobian is studied in both internal and external flow computations. As an internal
flow application 10 percent circular arc, Ni bump geometry is studied. Computations are performed at three different
incident Mach numbers: M = 0.5, M = 0.675 and M = 2. Three different grid sizes are used. The medium sized
grid has 65x17 nodes and it is shown in Figure-1. The characteristic boundary conditions based on Riemann
invariants are used at the inflow and outflow boundaries4. Wall and symmetry boundary conditions are employed at
the lower and upper sides of the boundaries. Figure 2 shows the Mach number contours of solution calculated for a
free-stream Mach number of 0.675. In this solution, fluxes are calculated by using a second order AUSM scheme
As an external flow application, flow around NACA 0012 airfoil is studied at a free-stream Mach number of
0.85, and an angle of attack of 1degree. The grid sizes of 129x33, 193x49 and 225x65 were used. The 225x65 grid,
which has 140 nodes on the airfoil is shown in Figure-3. The flow solution was performed for 0.85 Mach, 1degree
angle of attack flow case.
In Figure-5 the pressure distribution obtained with solutions using analytical and numerical Jacobian evaluation
is given. For numerical Jacobian the finite difference perturbation magnitude was taken as 3x10-8 which results from
Eq(39). It can be seen from the figure that
using the optimal value suggested both
Jacobian evaluation methods gave identical
flow solution results.

Figure 1. Mach Contours for M= 0.675 .

One of the main objectives of this study is

evaluate the effects of perturbation
magnitudes on the accuracy of numerical
Jacobians. Hence, numerical Jacobians are
calculated for a wide range of perturbation
magnitudes. The deviation between the
numerical and analytical Jacobian matrices is
defined as error. Since the error itself is also a
matrix, the norm values of error matrix are

Figure 2. 65x17 grid for Ni bump geometry.

Figure 3. SurfaceMach number distribution.

M= 0.85, =1o.

Figure 4. Mach number distribution

M= 0.85, =1o

11
American Institute of Aeronautics and Astronautics

Figure 5. 225x65 grid for Naca0012 airfoil.

Figure 6. Numerical Jacobian Error in Single Precision

(AUSM ,1st order,single, Mach#= 0.675)

defined. Different types of norms are used.

The change of the norm values of error
matrix with respect to perturbation
magnitude is plotted, and the results read
from theses plots are compared with the
results calculated by theoretical equations.
The read perturbation magnitude that gives
minimum error is compared with the
optimum perturbation magnitude that is
estimated with Eq.(39). The read norm
values of minimum error is compared with
the norm value of error at the optimum
perturbation, that is estimated by
neglecting the second term in Eq.(40).
Effects of different discretization schemes,
flow conditions, geometries and grid
resolution on the error in numerical
Jacobians are also studied. As stated
earlier, the error in numerical Jacobian
theoretically depends on two parameters:
the norm values of second derivatives of
the residuals with respect to the flow
variables, and the round-off error. The
evaluation of both of these terms may be
impractical for general CFD applications.
However, the evaluation of these terms
may give an opportunity to validate the
theoretical equations. In this study, for
single precision computations, the roundoff error is estimated as the differences
between the residuals calculated by single
and double precisions. The second
derivatives can be calculated by a finite
difference method. In order to find an
appropriate perturbation magnitude for
second derivatives, various perturbation
magnitudes are tested. Results show that
the norm values of second derivatives
increases for perturbation magnitudes
smaller than 10-8, but almost have no
variation between 10-8 and 10-3. In the
calculation of the norm values of second
derivatives
with
double
precision
computations, the perturbation magnitude
is set to 10-5.

Table-1. The estimated round-off error for single precision computation

Ni bump geometry; 65x17 grid , 0.675 inlet Mach #, 1st order AUSM fluxes

L1
L2
L

Round-off error
4.75 x10-9
7.76 x10-10
2.44 x10-9

Second derivatives
0.047
0.016
0.0202

optimum
Minumum ET
( Eq.39 )
( Eq.40 )
4.34 x10-4
0.885
7.8 x10-4
1.3x10-2
-

optimum
(trial error)
3.96x10-4
3.1x10-4
2.0x10-4

12
American Institute of Aeronautics and Astronautics

Minumum ET
(trial error)
0.927
8.9 x10-3
5.6 x10-4

Figure 7. Effects of precison on the error in numerical Jacobian

Matrix. (AUSM ,1st order, Mach#= 0.675)

The norm values of the round-off error

for single precision and the second
derivatives were tabulated in Table-1. The
same table also presents the optimum
perturbation magnitudes and the resulting
errors from the usage of those values
according to the Eq(39) and Eq(40).Figure6 presents the variation of the error in
numerical jacobian with perturbation
magnitude, for the single precision 1st order
flux calculation. To obtain the error values
given in the vertical axis, the L1 and the L2
vector norms are used. Both optimum
perturbation magnitudes and corresponding
error calculated by theoretical relations also
shown in Figure-6. Theoretical and trial
and error results agree very well with each
other. In the rest of the study, the numerical
Jacobian error plots will be given for the
double precision calculations and the error
will be given in matrix norm definition.

The value of machine epsilon in single precision is approximately the square root of the value computed by double
precision. Figure 7 shows that with the change of the computation precision from single to the double, the optimum
values becomes the square of the values obtained for single precision. Since the change of the precision affects only
the round-off error, and reduces the numerator of the Eq(39) by square, the location of the optimum and the total
error changes accordingly. In Eq(39) the norm values of round-off and second derivatives are in division, hence the
optimum perturbation magnitude does not vary with different norm definitions. However in Eq(40) the norms of
round-off error and second derivatives are in multiplication. Hence, the total error varies with norm definition. The
norms used in Figure 7 are L1, L induced matrix norms and Frobenious entry-wise matrix norm, and they are
defined below:
m

|| A ||1 = max | aij |

1 j n

m n

|| A ||F = | aij |2
i =1 j =1

|| A || = max | aij |
1 j m

i =1

j =1

(45)

The formulation of the second derivatives of flux residuals with respect to the flow variables can be obtained by
differentiating the residual Jacobian as below:

2 Ri , j

=
W k2,l
Wk ,l

Ri , j

Wk ,l

2 F + i +1/ 2, j Wi+L1/2, j
2Wi+L1/2, j 2 F i +1/ 2, j Wi+R1/2, j
2Wi+R1/2, j
+

=
A
A
+
+
+
i +1/ 2, j
i +1/ 2, j
2
W L 2 W
Wk2,l
Wk2,l
Wk ,l
Wi+R1/2, j
k ,l
i +1/2, j

F
2

i 1/ 2, j
2
L

Wi1/2, j

Wi1/2, j
L

Wk ,l

2 G + i , j +1/ 2 Wi ,Lj +1/2

Wi ,Lj +1/2

Wk ,l

2 G + i , j 1/ 2 Wi ,Lj1/2

Wi ,Lj 1/2

Wk ,l

Wi1/2, j
2

A+ i 1/ 2, j
+ B + i , j +1/ 2
B + i , j 1/ 2

Wk2,l
2Wi ,Lj +1/2
Wk2,l
2Wi ,Lj 1/2
Wk2,l

F
2

i 1/ 2, j
2
R

Wi1/2, j

WiR1/2, j
Wk ,l

2 G i , j +1/ 2 Wi ,Rj+1/2

Wi ,Rj +1/2

Wk ,l

2 G i , j 1/ 2 Wi ,Rj 1/2

Wi ,Rj 1/2

Wk ,l

13
American Institute of Aeronautics and Astronautics

A+ i 1/ 2, j
+ B i , j +1/ 2
B i , j 1/ 2

2WiR1/2, j
Wk2,l
2Wi ,Rj +1/2
Wk2,l
2Wi ,Rj 1/2
Wk2,l

(46)

Table-2. Effect of flux splitting

method on second derivatives
2
Flux
n
2 R ( w)
Evaluation

Method

n =1 w
SW
23.859
VL
25.186
AUSM
24.405

In first order flux discretization, the

second derivatives of interpolated face
variables with respect to the flow variables
equal to zero. Hence in first order
discretized fluxes, the magnitudes of the
second order derivatives depend only on
Figure 8. Effects of flux splitting method on error in Jacobian
the flux evaluation method. The effects of
Matrix
the flux evaluation method on the
( AUSM, Ni bump,M#=0.675)
magnitudes of the second derivatives are
investigated. The flow over bump geometry with inlet Mach number equals to 0.675 is solved using a grid size of
65x17 nodes with different first order methods. Each flux evaluation methods resulted with approximately the same
sized second derivatives. The calculated L2 norm values of the second derivatives of the residual vectors are given in
Table-2. The same table shows that the second derivatives do not vary with flux splitting method. Since the same
flow problem is solved on the same grid the magnitudes of the flow variables and fluxes will approximately same
even if the flux evaluation method varies. Hence the round off error is about the same in each flux calculation
methods. The effect of flux evaluation method on the error in 1st order numerical discretization is presented in
Figure-8. The same figure shows that the optimum perturbation magnitude does not vary with 1st order StegerWarming, Van-Leer and AUSM flux evaluation methods.
In second order flux discretization, the second derivatives of interpolated face variables with respect to the flow
variables are not zero. Hence these terms will affect the second derivatives of the residual vector. The formulation
of the interpolated face variables is given in Eq(10). Differentiating this equation with respect to the flow variables
twice gives :

2W iL1/ 2
Wk2,l

2 iL1/ 2
Wk2,l

2W iR1/ 2
Wk2,l

2 iR1/ 2
Wk2,l

(47)

The second derivatives of the interpolated face variables directly depend on the second derivatives of the limiter
functions. Limiters are known to stall the convergence of an iterative scheme, because of accidental switching in
smooth flow regions20 In smooth flow regions difference between the values of flow variables at the neighboring
cells is nearly zero. Therefore assuming that backward and forward differencing of flow variables (a and b in Eq 11
and Eq12) are equal to each other would be reasonable. The formulation for the second derivatives of the limiter
function with this assumption will become very simple and it is given in Eq(47). This formulation presents that in
smooth flow regions, as the values of get smaller the second derivatives of the limiter functions becomes larger.
2
W 2

=
a =b

a
a+

14
American Institute of Aeronautics and Astronautics

(48)

Table-3. Effect of used in limiter

function on second derivatives
used in
limiter
= 0.008
= 10-3
= 10-6
= 10-9
= 10-12

1
n

2 R ( w)

n =1 w
0.035
0.165
7.02
58.7
260
n

In the formulation of limiter functions,

in order to prevent the activation of
limiters a small number is used. There
are different studies performed to define
Figure 9. Effects of on the accuracy of the Jacobian
the values of . 18,19. In this study, the
( Mach#= 0.675, Ni bump)
effects of the magnitude of on the
accuracy of numerical Jacobians are studied. In Table 3, the root mean square of the second derivatives of the
residual vector evaluated with different sizes of is tabulated. Consistent to Eqs (46),(47) and(48), the second
derivatives of residual vector get larger as the size of the gets smaller. The variation of the L1 norm of the error in
numerical Jacobian matrix with respect to the size of is given in Figure-9. For small values the second
derivatives of interpolated face variables become larger compared to the second derivatives of splitted flux vectors.
Hence for second order discretization, the size of the second derivatives of the residual vector is dominated by the
second derivatives of the limiter functions for small values. The value of has no affect on the round-off error.
Therefore, as the values of become smaller the magnitude of second derivatives increases while the round-off
error stays constant. As a result, the small values of causes an increase in total error and a decrease in the
optimum perturbation magnitude. The results shows that limiter functions calculated with small values
significantly increase the magnitudes of the second derivative of residual vector and they can not be assumed as one
in Eqs(39) and (40) . Hence the optimum perturbation magnitude deviates from the one proposed by Eq(42)
significantly. For larger values of the the second derivatives of interpolated face variables become smaller
compared to the second derivatives of splitted flux vectors. In this case, the size of the second derivatives of the
residual vector is significantly affected by the second derivatives of the splitted flux vectors. Hence the usage of
larger will not affect the accuracy of the Jacobian any more. Moreover the usage of very large values in limiter
function will be improper. Since the variation in flow variable will become negligible compared to the large
values, the definition of the interpolation with limiter function will be degraded. As a summary, the accuracy of the
second order numerical Jacobian is bounded by that of the first order numerical Jacobian. On the other hand, the
evaluation may become erroneous with the usage of small in the limiter function. In this study the most accurate
evaluation for the second order numerical Jacobian is achieved with the usage of the values proposed in
References[20] and[21]

15
American Institute of Aeronautics and Astronautics

The effects of different geometries and

flow conditions on the accuracy of
numerical Jacobian evaluation are studied.
Figures 10 and 11 show the variation of the
error with respect to perturbation
magnitude for internal and external flow
applications. In these figures, the effects of
different flux splitting methods on the
accuracy of numerical Jacobian evaluation
for flows over bump and airfoil geometries
are presented. The total error in the vertical
axis is given in terms of the induced L1
matrix norm. In both results, the optimum
perturbation magnitudes for first and
second order discretizations are same since
a proper value is used. Moreover the
optimum perturbation magnitudes provided
by these figures agree well with the
proposed magnitude by Eq(42) . One of the
Figure 10. Effects of flux splitting method on error in Jacobian important conclusions from these figures is
Matrix ( Mach#= 0.675, Ni bump)
that, although the effects of the flow
condition and geometry on the magnitude
of optimum perturbation are small the total
error is significantly affected. The total
error in the airfoil case is about two order
of magnitudes larger compared to the bump
case. To resolve the cause of this situation
the Eq(39) and Eq(40) should be revised.
In formulation which gives optimum
perturbation magnitude the round off error
and second derivatives are in division. In
the formulation which gives the total error
those two terms are in multiplication.
Increasing the total error without changing
the optimum perturbation magnitude may
be caused by the proportional enlargement
in both of round-off error and the second
derivatives.
In
both
flow
cases,
approximately the similar flow conditions
are used. In both cases, transonic flow
conditions cause a supersonic region on the
Figure 11. Effects of flux splitting method on error in Jacobian
geometry with a maximum mach number
o
Matrix ( Mach#= 0.85,=1 , Naca0012)
of 1.4. The main difference between the
two cases was the grids used in the solutions. In the problem of flow over Ni-bump, an H-grid whose inlet and outlet
are located 1.75 times of chord length away from the leading and trailing edges. In flow over airfoil problem, a Cgrid in which distance between far-field boundary and the airfoil geometry varies between 10 to 60 chord length is
used. To be able to resolve the gradients, nodes were clustered near to airfoil geometry and were expanded at the
far-field. The sizes of the cell faces at the far-field of C-grid are approximately 15 times larger than the ones used at
the inlet and the outlet of H-grid. Due to the large cell faces it will be reasonable to expect large flux values near the
far-field of the C-grid compared to the H-grid. This enlargement in the flux magnitudes may amplify the round-off
error in residual computations. Since there is no practical way of round-off error estimation for double precision
computations, the variation of round-off error according to the those two grids checked for single precision
computation. Although the error results in Figures 10 and 11 were presented for double precision the single
precision computation of the difference of round-off errors in two cases gives satisfactory clues. For single precision
computation, the L2 norm of the round off error for c-grid is computed as approxiamtely4 x10-6 whereas it is
16
American Institute of Aeronautics and Astronautics

Table-4. The estimated round-off error for single precision computation

imax 1 jmax 1

grid

i =2

gle
Ridouble
Risin
,j
,j

j =2

7.41x10-9
6.5x10-9
1.4x10-7

65x17 H-grid
129x33 H-grid
129x33 C-grid

imax 1 jmax 1

(
i =2

Ridouble
,j

j =2

gle
Risin
,j

1.3x10-8
1.2x10-8
4x10-6

calculated as 1.2x108 for the H-grid with the same size of node numbers. The calculated single precision round-off
errors with different norm definitions are tabulated in
The second derivatives of the residual at a given cell depends on the flow variable at that cell and 4 or 8
neighboring cells according to the order of
discretization. Figure 12 and 13 present the
contour plots of the averaged values of the second
derivatives of the residual vector. Figure 13 shows
that the second derivatives of the residual vector
gets larger near boundaries and gets smaller near
the wall. The comparison of the values of second
derivatives for those two cases shows that in the
regions closer to the wall geometries the second
derivatives are in the same order of magnitudes.
However in the far-field regions of the c-grid the
second derivatives of the residual vector are
approximately 2 orders of magnitude larger.
Table-4, Figures 12 and 13 present that in flow
solutions with larger grid cells, the round-off error
and the second derivatives of the residual vector
get larger compared to the ones that uses a grid
with smaller sized cells.
Figure 12.Contours of second derivatives
The effects of the cell size on the second
derivatives and on the total error in numerical
Jacobian are shown by two alternative ways.
Firstly, only the cells closer to the airfoil which
has smaller sizes are used for the comparison with
H-grid which has small sized cells. Secondly, the
same flow problem is re-solved with a c-grid
whose farfield is closer to the airfoil geometry
with same number of nodes. The error plots of the
numerical Jacobian for those cases are given in
Figure-16 with the error plot of the H-grid case.
The Figure 16 is drawn with the results obtained
with 2nd order discretization. In the small region of
the c-grid which is very close to the airfoil
geometry, the magnitude of the total error is
almost equal to the total error of the H-grid case
since the cell size are almost equal. The results
also show that by using a grid with closer far-field
boundaries, uniformly distributed grid cells can be
constructed and the total error may be reduced.
However, as the far-field boundary gets closer to
geometry the accuracy of the boundary conditions
becomes more critical.
Figure 13. Contours of second derivatives

17
American Institute of Aeronautics and Astronautics

Figure 14. Alternative-1, The small cell sized

region chosen for the comparison

Figure 15.

Alternative-2. The 129x33 Cgrid with

closer farfield boundary

After completing the search for the

optimum perturbation magnitude for
different flux evaluation methods and for
different flow problems with different grid
types, the variation of the error with grid
resolution is studied. The analysis is
performed both for the h-grid and c-grid
cases. The results are presented in Figures
16 and 17. Calculations are performed
using AUSM scheme for first and second
order spatial discretizations. In second
order discretization, limiter function of the
Van Albada is used with value of 0.008.
For the bump geometry an inlet Mach
number of 0.675 and for the airfoil
geometry the free stream Mach number of
0.85 and 1 degree of angle of attack is
used. Like in the previous cases, for
different grid resolutions, the optimum
perturbation locations for bump and airfoil
geometries coincides with each others,
Figure 16. Effects of thegrid size on the accuracy of Jacobian
whereas the total error is larger in C-grid
solution. To make the grid finer or coarser
does not change the magnitudes of the flow variables significantly. Therefore, the second derivatives and the round
off error are almost constant as the grid resolution changes. As a result the optimum perturbation magnitude
minimizing the total error is same for all coarse and fine grids. The optimum perturbation magnitudes presented by
Figures 17 and 18 agree well with the magnitudes proposed by Eq(42).
As the last factor affecting the accuracy, the effect of the free-stream flow condition is analyzed. Figure-19
shows the change of errors with perturbation magnitude for three different flow conditions. The results presented are
calculated for bump geometry by AUSM scheme with first and second order spatial discretizations. Results show
that the optimum perturbation magnitude is not sensible to the free stream conditions of the flow problem chosen.
The optimum magnitude did not varied significantly for the cases with different inlet mach numbers and its
approximately equal to the magnitude proposed by Eq(42).

18
American Institute of Aeronautics and Astronautics

Figure 17.

Effects gridre solution on error in Jacobian Matrix

( AUSM, MACH#=0.675, Ni bump)

Up to here, different factors which may

affect the accuracy of the numerical
Jacobian are analyzed. All the results show
that usage of the finite difference
perturbation magnitude prescribed by
Eq(42) is satisfactory enough to construct
accurate numerical Jacobian matrices
compared to analytical ones. The Eq(42)
was the simplification of the Eq(39) with
the assumption of second derivatives equal
to one and round-off error equals to
machine epsilon. Although the assumed
values for round-off error and second
derivatives are not reasonable for most of
the cases the proportion of these two terms
does not change. Therefore, the assumption
works well and it will be concluded that the
optimum perturbation magnitude nearly
equals to 3.0x10-8 for double precision and
6.9x10-4 for single precision computations.
Analyzing the ways of accurate
computation of numerical Jacobians, the
effect of the accuracy of the Jacobian on
the convergence of the flow solution and
on the accuracy of the sensitivity analysis
will be presented in remaining parts.

To make the flow solution with

Newtons method more stable some
modifications are applied on it. In earlier
iterations, diagonal terms are added to
Jacobian to have stable linear solution.
Reaching a proper convergence level,
diagonal terms are withdrawn to reduce the
total number of iterations and CPU time.
The number of iterations and overall CPU
time depend on the magnitudes of the
diagonally added terms. The best
convergence behavior can be achieved with
the usage of optimum values for the initial
and the withdrawal values of these terms.
The optimum initial and the withdrawal
values vary for different flow solutions
since the Jacobian matrix changes. In this
Figure 18. Effects grid resolution on error in Jacobian Matrix study, the initial and withdrawal values for
( AUSM, MACH#=0.675, Naca0012)
diagonally added terms are fixed at a
reasonable values, rather than searching the
optimums for each cases. Different fixed values are used for the first and the second order discretizations since the
change of order of discretization greatly changes the Jacobian matrix.
The effects of accuracy of the Jacobian matrix on the convergence behavior of the Newtons method is studied
before by 1,2 . In those studies, it was shown that usage of estimated (numerical) Jacobian rather than exact
(analytical) Jacobian will degrade the convergence performance of the solver. In next section, the variation of
convergence histories and the CPU time with the usage of analytical and numerical Jacobian are presented. Figure
19 and Figure 20 give the convergence histories for the flow solutions over bump and airfoil geometries.
Calculations are performed with both first and second order spatial discretizations. Figures present the convergence
performances of the analytically evaluated Jacobian and the numerically constructed Jacobians with different finite
19
American Institute of Aeronautics and Astronautics

Figure 19. Effects of flow condition on error in Jacobian Matrix

( AUSM, Ni bump)

Figure 20. Convergence History

( AUSM, Ni bump)

difference perturbation magnitudes. The

number of node points of grids for the
bump and airfoil geometries are 65x17
and 129x33, respectively. For solution on
bump geometry initial and withdrawal
values of diagonally added terms are
chosen as 3 and 5, respectively for the first
order spatial discretization and in second
order discretization chosen as 50 and 500.
In airfoil case initial and withdrawal
values of diagonally added terms are
chosen as 3 and 5, respectively for the first
order spatial discretization and in second
order discretization chosen as 200 and
20000.
Figures for convergence histories shows
that the best convergence performances
are given by the analytically derived
Jacobian and the numerical one which is
computed by the optimum perturbation
magnitude. As the accuracy of the
numerical
Jacobian
degrades
the
convergence performance also degrades.

Figure 21 Convergence-history
( AUSM, Naca0012)

20
American Institute of Aeronautics and Astronautics

Figure 22. Effects of on convergence history

( AUSM, Ni bump)

Figure 23.

CPU time spent in an iteration

( AUSM, Naca0012)

In Figure 22 the effect of the used in the limiter function on the convergence performance is given. To generate
the results given in Figure 22, Each numerical Jacobian is calculated by its own optimum perturbation magnitude
which can be read from Figure-9. Figure 23 presents the CPU time spent in an Newton iteration. Figure shows that,
the most of the CPU time is spent for Jacobian matrix evaluation, when the large grid size is used eventhough 1st
order discretization is utilized.

VI. Sensitivity Governing Equations

The gradient based optimization method is widely used technique in aerodynamic shape optimization. The
objective is maximizing or minimizing the aerodynamic loads by using design variables. To be able to perform this,
gradients of objective functions in design variable space is required. The gradients of aerodynamic loads with
respect to design variables are known as objective function sensitivities. In general, aerodynamic loads are functions
of state flow variables, geometrical variables and design variables .14
C j = C j {W ( ) , X ( ) , }

(49)

Hence applying the chain rule sensitivities of the aerodynamic loads can be calculated as follows
. dC j = C j W + C j X + C j

d k W k X k k

(50)

The second term in the right hand side is zero when design variables are independent from the geometry. The
third term in the right hand side is zero when geometrical design variables are used since their effect on total
derivative is already stated in the second term. In this study geometrical design variables are used. The aerodynamic
geometry is modified by using Hicks Henne functions15. The weightings of the functions are chosen as design
variables.

C j W C j X
=
+

d k W k X k
dC j

21
American Institute of Aeronautics and Astronautics

(51)

where Cj, w, X , k are aerodynamic loads, flow variables, grid coordinates and design variables, respectively The
aerodynamic loads have explicit dependence on the flow variables and the coordinates of the geometry. Therefore
the evaluations of Cj /W and Cj /X derivatives are simple. As an example, for Euler equations, the dependence
of lift coefficient, CL, and drag coefficient, CD, on the flow variables and coordinates of the geometry can be
illustrated as follows.

C L = C y cos C x sin

(52)

C D = C y sin + C x cos
Cx and Cy are the force coefficients in x and y directions and can be evaluated as:
NE

NE

Cx = Cx j

Cy = Cy j

j =1

(53)

j =1

In above equation, NE denotes the number elements on the geometry.

(
(x

Cx j = C p j yb j+1 yb j
Cy j = C p j

bj

xb j+1

)
)

(54)

Pressure coefficient can be calculated as:

Cpj =

Pwall j
1
V2l
2

(55)

The pressure on the geometry can be written in terms of flow variables as below:

Pwall j

2
2

u ) wall + ( v ) wall j
(
j
= ( 1) ( E ) wall
j

2 wall j

(56)

In this study, aerodynamic geometry is perturbed by Hicks Henne functions and new coordinates of the grid is
obtained by using Equation 34. The modification to geometry is applied such that the domain is perturbed mostly
near the wall and effect of perturbation is diminished away from the wall. The grid points furthest to the wall are not
affected by the perturbations applied to the wall.

j j
X new = X old + k f k ( x) max

jmax 1

(57)

Hence the grid coordinates have an explicit dependence on the design variables, and the grid sensitivities can be
evaluated simply as:

X
j j
= f k ( x ) max
k
jmax 1

(58)

Equations from52 to 58 show some explicit relations such that Cj= Cj(X,w) and X =X(k). Therefore the evaluation
of Cj /w, Cj /X, X/k derivatives are straightforward. However, the evaluation of derivative w/k is not
simple since there is no explicit relation between flow variables and design variables. There are two methodologies
22
American Institute of Aeronautics and Astronautics

to pass this step of derivation to reach final

formulation that gives sensitivities of the
objective functions. These methods are
direct differentiation method and adjoint
method. Adjoint method bypasses the
requirement for evaluation of the flow sate
sensitivities, w/k, by introducing
Lagrange multipliers. The choice between
those two methods is made according to the
size of the objective functions and the
design variables. If the number of design
variables is larger than the number of
objective functions, the adjoint method is
computationally more desirable compared
to the direct differentiation method for
iterative solvers. However the usage of
direct solvers in flow solution removes the
computational difference between those
Figure 24. Effects of flux splitting method on error in
two methods. The LU factors computed in
sensitivity
flow solution can be reused in sensitivity
( AUSM, Ni bump)
analysis and both of these methods can be
applied efficiently. In this study the direct
differentiation method is chosen for sensitivity analysis. Direct differentiation method is formulated by the
differentiation of the discrete residual vector.
At the steady state flow condition, residual vector, which is a function of flow variables and grid coordinates, equals
to zero.

{ R (W (

) , X ( k ) ) } = {0}

(59)

The flow sate sensitivity, w/k is evaluated from direct differentiation of steady state flow governing equations as
follows16,17

R w R X
w + X = 0
k
k

(60)

w
R R X
=

k
W X k

(61)

Above equation shows that, the calculation of flow state sensitivities requires the evaluation of flux Jacobian matrix,
R /w. In previous sections, the flux Jacobian were evaluated analytically and numerically, and the accuracy of
numerical Jacobians were analyzed. The main interest of this section is to investigate the effect of accuracy of the
numerical flux Jacobian on the accuracy of flow state sensitivities. For this purpose, various numerical flux
Jacobians with different finite difference perturbation magnitudes are used to obtain sensitivities. The error in
sensitivities calculated with numerical flux Jacobians is defined as.

Errorsensitivity =

W
W

k numeric k analytic
jacobian

(62)

jacobian

The Newtons method is efficiently used in evaluation of sensitivities since the jacobian matrix appearing in
Equation (54) was already LU decomposed to solve Equation (14). Hence in Equation (54) each of flow state
sensitivities are calculated in single iteration by simple backward and forward substitution.
23
American Institute of Aeronautics and Astronautics

VII. Effects of Flux Jacobian Evaluation on Sensitivities

The effect of the accuracy of the numerical Jacobian on the sensitivities calculated by direct differentiation
method is studied. As explained in previous section, flow state sensitivities are the dominant components of the
objective function sensitivities, hence the accuracy of the flow state sensitivities are analyzed in detail. The error
analysis is performed by constructing a
flow state sensitivity error vector by
substituting vectors evaluated from
numerical Jacobians from the analytical
ones. The L1 norm of the error vector is
presented in results.
The code solves the flow equations
with Newtons method for a given
conditions. After calculating a converged
solution analytical Jacobian is calculated
by using final flow variables. The right
hand side of the sensitivity governing
equations is evaluated. Using the analytical
Jacobian, analytical sensitivities are
evaluated. A loop is constructed in which
the finite difference perturbation magnitude
is incremented. In the loop for the each
perturbation
magnitudes
numerical
Jacobians are calculated. After the
numerical Jacobian calculation numerical
sensitivity is evaluated. For the each
Figure 25. Effects of flux splitting method on error in
perturbation magnitude in the loop,
sensitivity
calculated
numerical
Jacobian
and
(AUSM, Naca0012,M#=0.85)
sensitivity is compared with the previously
( AUSM)
stored analytical Jacobian and the
sensitivities, and error plots are drawn. The
Hicks Henne functions are used to perturb
the geometry. The function which has the
maximum perturbation at 30% of the chord
is chosen.
The effect of the flux splitting methods
on the accuracy of the sensitivity analysis
is studied. Figures 24 and 25 show the
effect of the accuracy of different flux
splitting schemes on the error in numerical
sensitivities. Results are given for identical
cases used in the Jacobian accuracy
analysis. Comparisons of Figures 24 and
25 with Figures10 and 11 show that the
locations of the optimum perturbation in
numerical Jacobian and sensitivity
evaluations are same. The location of the
optimum perturbation can be estimated
accurately by Eq(42).
The effect of the grid resolution on the
accuracy of the sensitivity analysis is
studied. Figures 26 and 27 present
sensitivity accuracy analysis for bump and
airfoil geometries, respectively. The same
Figure 25. Effects grid resolution on error in sensitivity
grids used in analysis of Jacobian accuracy
(AUSM, Ni bump,M# =0.675)
are also used in the sensitivity analysis.
24
American Institute of Aeronautics and Astronautics

Figure27. Effects of grid resolution on error in sensitivity

( C-grid, AUSM, M# = 0.85)

Results show that grid resolution does not

affect the optimum perturbation magnitude
whereas the magnitude of the error in
sensitivity is increased by the grid
resolution. When the number of the grid
nodes increases, the size of the sensitivity
error vector also increases. Therefore L1
vector norm gets larger with increasing
vector size.
The effect of the free-stream flow
condition on the accuracy of the numerical
sensitivity vector is studied. Sensitivity
equations are solved for flows over the
bump geometry with different free-stream
Mach numbers. Figure- 29 shows that the
location of the optimum perturbation
magnitude is not affected significantly by
the flow conditions. But the error may
change according to the flow conditions
used.

VIII. Conclusion
The Euler flow equations are solved with
the Newtons method. The source of the
error in numerical Jacobian evaluation is
analyzed. The finite difference perturbation
magnitude significantly affects the
accuracy of the numerical Jacobian. Using
the optimum finite difference perturbation
magnitude the error in numerical Jacobian
can be minimized. Some simple formulas
are derived to evaluate the optimum
perturbation magnitude. Evaluated values
are compared with the ones given by trial
and error method with a range of
perturbation magnitudes. The optimum
perturbation magnitude is function of the
magnitudes of round-off error and the
second derivatives of the residual vector. In
first order flux discretization the magnitude
Figure 29. Effects of of flow condition on error in sensitivity
of second derivatives are not very large. In
( AUSM, Ni bump)
second
order
discretization
these
derivatives may have large values. This
may be due to the large values of the
second derivatives of the limiter functions
in the smooth flow regions. The effect of the accuracy of the Jacobian matrix on the convergence of the flow
solution is studied. The numerical Jacobian evaluated with the optimum perturbation magnitudes can show the same
convergence performance that is obtained by the analytic Jacobian. The effect of the finite difference perturbation
magnitude on the accuracy of the numerical sensitivities is analyzed. The same optimum perturbation magnitude
enabled the most accurate numerical Jacobian and sensitivities for a wide range of flow problems.

25
American Institute of Aeronautics and Astronautics

IX. References
1
Onur, O. and Eyi, S., Effects of the Jacobian Evaluation on Newtons Solution of the Euler Equations, International
Journal for Numerical Methods in Fluids, Vol. 49, pp 211-231, 2005.
2

Orkwis,P.,D., and Venden, K.,J., On the Accuracy of Numerical Versus Analytical Jacobians, AIAA Paper 94-0176,

1994
3
Rizk, M. H., The Use of Finite-Differenced Jacobians for Solving the Euler Equations for Evaluating Sensitivity
Derivative, AIAA Paper 94-2213, 1994.
4

Steger, J. L., and Warming, R. F., Flux Vector Splitting of the Inviscid Gasdynamic Equations with Application to
Finite-Difference Methods, Journal of Computational Physics, Vol. 40, 1981, pp. 263-293.
5

Van Leer, B., Flux Vector Splitting for the Euler Equations, ICASE Report 82-30, September 1982.

Liou, M.-S. A sequel to AUSM: AUSM+, Journal of Computational Physics, Vol 129 (1996), pp. 364382

7
Roe, P. L., Characteristics-Based Schemes for the Euler Equations, Annual Review of Fluid Mechanics, Vol. 18, 1986,
pp. 337-365.
8

Ni, R. H., A Multiple-Grid Scheme for Solving the Euler Equations AIAA Journal vol.20, no.11 pp.1565-1571, 1982.

Hirsch, C., Numerical Computation of Internal and External Flows, Vol. I-II, John Wiley & Sons, Chichester, 1990.

10
Dennis J.E., Schnabel R.B., Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall,
New Jersey, 1983.
11

Davis, T. A., UMFPACK Version 4.1 User Manual, University of Florida, Florida, 2003.

12

Gill, P.E., Murray W., and Wright M.H., Practical Optimization, Academic Press, London, 1992.

13
Kelley C.T., Iterative Methods for Linear and Nonlinear Equations, Society for Industrial and Applied Mathematics,
Philadelphia, 1995.
14

Korivi V.M., Taylor A.C., Newman P.A., Hou G.J.W. and Jones H.E.., An Approximately Factored Incremental Strategy
For Calculating Consistent Discrete Aerodynamic Sensitivity Derivatives, AIAA Paper 92-4746, 1992
15
Lee, K. and Eyi, S., Aerodynamic Design Via Optimization ,Journal of Arcraft Vol. 29, No.6 November-December
1992, pp 1012-1029.
16

Taylor, A. C. III, Korivi, V. M., and Hou, G. W., Sensitivity Analysis Applied to The Euler Equations : A Feasibility
Study with Emphasis on Variation of Geometric Shape, AIAA Paper 91-0173 1991
17
Taylor, A. C. III, , Hou, G. W., and Korivi, V. M., A Methodology for Determining Aerodynamic Sensitivity
Derivatives with Respect to Variation of Geometric Shape AIAA Paper 91-1101 1991

18
Radespiel, R., and Kroll, N., Accurate Flux Vector Splitting for Shocks and Shear Layers, Journal of Computational
Physics 121,1995, pp 66-78.
19

Godlewski, E., and Raviart, P.-A., Numerical Approximation of Hyperbolic Systems of Conservation Laws ,Springer,
New York,1996.
20
V. Venkatakrishnan, On the accuracy of limiters and convergence to steady state solutions,J.Comp. Phys., 118 (1995), pp.
120--130.
21

Van Albada, G.D., Van Leer, B., Roberts, W.W., A Comparative Study of Computational Methods in Cosmic Gas
Dynamics, Astronomy and Astrophysics, Vol 108, 1982, pp. 76-84,

26
American Institute of Aeronautics and Astronautics