Systems of Nonlinear Equations

Solving a set of nonlinear equations is much more difficult than solving a single non-
linear equations.

Problem: Consider the two nonlinear equations in two variables,

f (x, y) = 0 g(x, y) = 0
Find the roots (α, β) which satisfy both equations.

Example: 2.5

2 x2+y2=4

x2 + y 2 = 4 1.5

ex + y = 1 1

0.5

−0.5

−1

−1.5

−2
exp(x)+y=1

−2.5
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5

30
How do we iterate from the nth to the (n + 1)th approximately; i.e. from (xn, yn)
to (xn+1, yn+1)? As with Newton-Raphson we Taylor expand about the root.

Notation: fx = ∂f
∂x and f y = ∂f
∂y .

Taylor expand a function of two variables,

f (α, β) = 0 = f (xn, yn) + (α − xn)fx(xn, yn) + (β − yn)fy (xn, yn)

+O(α − xn)2 + O(β − yn)2
g(α, β) = 0 = g(xn, yn) + (α − xn)gx(xn, yn) + (β − yn)gy (xn, yn)
+O(α − xn)2 + O(β − yn)2
Define ∆xn = α − xn and ∆yn = β − yn, and truncate the series,

fx(xn, yn) · ∆xn + fy (xn, yn) · ∆yn ≈ −f (xn, yn)

gx(xn, yn) · ∆xn + gy (xn, yn) · ∆yn ≈ −g(xn, yn)

i.e. two linear equations for ∆xn and ∆yn.

31
For Newton-Raphson: treat this approximate equation for (α, β) as an exact equa-
tion for xn+1 = xn + ∆xn and yn+1 = yn + ∆yn.

Thus, one Newton-Raphson iteration consists of solving,

! ! !
fx(xn, yn) fy (xn, yn) ∆xn f (xn, yn)
=−
gx(xn, yn) gy (xn, yn) ∆yn g(xn, yn)
for ∆xn and ∆yn and then updating,

xn+1 = xn + ∆xn
yn+1 = yn + ∆yn .

e.g.
f (x, y) = 4 − x2 − y 2 = 0
⇒ fx = −2x , fy = −2y
g(x, y) = 1 − ex − y = 0
⇒ gx = −ex , gy = −1

32
 x2 2
! ! !
−2xn −2yn ∆xn n + yn − 4
=
−exn −1 ∆yn exn + yn − 1
First guess, (x1, y1) = (1, −1.7)
! ! !
−2.0 3.4 ∆x1 −0.11
⇒ =
−2.7183 −1.0 ∆y1 0.0183
Solve for ∆x1 and ∆y1 (by Gaussian elimination - see shortly),

⇒ ∆x1 = 0.0043 , ∆y1 = −0.0298

⇒ x2 = 1.0043 , y2 = −1.7298

Repeat once more to get,

x3 = 1.004169, y3 = −1.729637

33
Systems of linear equations

Solving systems of linear equations is the most important problem of scientific com-
puting.

Example Solve for x, y, z in

3x + 5y − z = 10
7x − 2y + 3z = 12
x + 5y − 4z = −1

( ☞ we need three independent equations to solve for three unknowns)

In matrix form, 3 5 −1
 
x
 
10


 7 −2 3   y  =  12 
    

1 5 −4 z −1
In shorthand: we have Ax = b, must solve for x
34
A is non-singular if there exists a B such that,

AB = I ⇒ B = A−1 (inverse ofA)

Ax = b
⇒ x = A−1 · b (unique solution)

A is singular if there exists no B such that AB = I then

Ax = b
has no solution or infinitely many solutions!!

The line between singular and non-singular can be very thin:

" #
1 1
is non − singular only if  6= 0 .
1 1+

35
Errors, norms and sensitivity:

1.0
 
1.01
 
1.0


 1.0  ; x1 =  1.01  ; x2 =  1.02 

x=     

1.0 1.01 1.0

Which of x1, x2 are closer to x?

Define error vectors, 1 = x1 − x; 2 = x2 − x.

The norm of a vector is a measure of its size

 
x1
 x2 
 
 
 · 
 
 
x= · 

 kxk denotes the norm .
 · 
 
 
 · 
 

xn
There are many ways of defining a norm!!
36
 
0.01
 
0.0


1 =  0.01  ; 2 =  0.02 
   

0.01 0.0

 1
n 2
|xi|2
X
i) Euclidean norm = 
i=1

k1k = 0.0173 , k2k = 0.02

n
X
ii) “1-norm” = |xi|
i=1
k1k = 0.03 , k2k = 0.02
iii) Maximum norm = max |xi|
i=1,...,n

k1k = 0.01 , k2k = 0.02 .

37
Matrix norms: As for vectors there are a number of possibilities. In a calculation, for
consistency, we will use the maximum norm for A and x.
n
X
kAk = max |aij |
1≤i≤n
j=1

Example:
" #
1 −2
= max(3, 1) = 3 .


0 1

“Theorem” This definition ensures that kAxk ≤ kAk·kxk.

Example " # " # " #

1 −2 2 −4
A= x= Ax =
0 1 3 3

kAk = 3 kxk = 3 kAxk = 4

38
Proof:

kAxk = max |(Ax)i|

1≤i≤n
 
 n 
X 
= max   aij xj 
1≤i≤n 
j=1 
X n
≤ max |aij ||xj |
1≤i≤n
j=1
X n
≤ max |aij |kxk = kAk·kxk
1≤i≤n
j=1

39