Tutorial - week9

To minimiz
2
2
2
f ( x , y , z ) x x ( 1 - y ) y - yz z z ,
with the starting poi nt ( x 0 , y0 , z 0 ) ( 0 ,0 ,0 ) by
( 1 ) Steepest Descent M ethod
( 2 ) Conjugate Gradient M eth od

Firstly , calculating f x , y , z by

( 1) Solution:

f x , y , z 2x - y 1, 2y- x - z , 2z - y 1
T

r0 -f ( x0 , y0 , z0 ) - 2x0 - y0 1, 2y0 - x0 - z0 , 2z0 - y0 1

-1, 0 , -1 .
T

Next , finding 0 which minimizes f ( x0 r0 ) by

(x1 , y1 , z1 ) (x0 , y0 , z0 ) r0 (0 ,0 ,0 ) -1, 0 , -1

-, 0, -

f ( x1 , y1 , z1 ) f ((x0 , y0 , z0 ) r0 ) f (- , 0 , - ) q( )
(- ) 2 (- )( 1 - 0 ) 0 2 - 0 (- ) (- ) 2 (- ) 2 2 - 2.
dq( ) d ( 2 2 - 2 )
1

4- 2 0 0 .
d
d
2

( 1) - con. Therefore

1
1
1
(x1 , y1 , z1 ) (x0 , y0 , z0 ) 0 r0 (0 ,0 ,0 ) -1, 0 , -1 - , 0 , - .
2
2
2
Thirdly, take the negative gradient to find the next search direction:
r1 -f ( x1 , y1 , z1 ) - 2x1 - y1 1, 2y1 - x1 - z1 , 2z1 - y1 1

1
1
1
1
T

- 2 (- ) - 0 1, 2 (0 )- (- )- (- ) , 2 (- ) - 0 1 0 , -1, 0 .
2
2
2
2

Then, update the iteration formula by

1
1
(x2 , y2 , z2 ) (x1 , y1 , z1 ) r1 - , 0 , - 0 , -1, 0
2
2
1
1
- , - , - .
2
2

( 1) - con. Thus , we have

1
1
f (x2 , y2 , z2 ) f ((x1 , y1 , z1 ) r1 ) f (- , - , - ) q( ).
2
2
1 2
1
1
1 2
1
2
q( ) (- ) (- )( 1 - ( - )) ( - ) - ( - )(- ) (- ) (- )
2
2
2
2
2
1
2
-- .
2
1
2
dq( ) d ( -- 2 )
1
Set

2-1 0 1
d
d
2
1 1
1
(x2 , y2 , z2 ) ( x1 , y1 , z1 ) 1r1 - , 0 , - 0 , -1, 0
2 2
2
1 1 1
- , - , - .
2 2 2

( 1) - con. Take the negative gradient to find

r2 -f (x2 , y2 , z2 ) - 2x2 - y2 1, 2y2 - x2 - z2 , 2z2 - y2 1

T
T

1
1
1
1
1
1
1

- 2(- ) - (- ) 1, 2(- )- (- )- (- ) , 2(- ) - (- ) 1

2
2
2
2
2
2
2

1
1
- , 0 , - .
2
2
Again , update the iteration formula by
1
1 1 1
1
(x3 , y3 , z3 ) ( x2 , y2 , z2 ) r2 - , - , - - , 0 , -
2
2 2 2
2
1 1
1

- ( 1) , - , - ( 1) .
2 2
2

( 1) - con. Thus, we obtain

1 1
1

f (x3 , y3 , z3 ) f - ( 1) , - , - ( 1) q( ).
2 2
2

1
1 2
1
1

q( ) - ( 1) - ( 1) ( 1 - (- )) (- )
2
2
2
2

1 1
1
1

- (- ) - ( 1) - ( 1) - ( 1)
2 2
2
2

1
1
2 3
( 1) - ( 1)

2
2
4
1
1
2 3
d ( 1) - ( 1)
1
1
dq( )
2
2
4

- 0 2 .
d
d
2
2

( 1) - con. Thus

(x3 , y3 , z3 ) ( x2 , y2 , z2 ) 2 r2

1 3 1 3
1 1 1 1 1
- , - , - - , 0 , - - , - , - .
2 4 2 4
2 2 2 2 2
Find the next search direction:
r3 -f (x3 , y3 , z3 ) - 2x3 - y3 1, 2y3 - x3 - z3 , 2z3 - y3 1

T
T

3
1
1
3
3
3
1

- 2(- ) - (- ) 1, 2(- ))- (- )) ( - ) , 2 ( - ) - (- ) 1

4
2
2
4
4
4
2

T
1

0 , - , 0
(x4 , y4 , z4 ) (x3 , y3 , z3 ) r3
2

3
3 1 3 1 3 1
(x4 , y4 , z4 ) - , , - 0 , , 0 - , - ( 1) , - .
4
4 2 4 2 4 2

( 1) - con. Find 3 by
f (x4 , y4 , z4 ) x42 x4 ( 1 - y4 ) y42 - y4 z4 z42 z4 q( ).
)
2

1
1
3 3
q( ) - - ( 1 - (- ( 1))) (- ( 1)) 2
2
2
4 4
2

1
3 3 3
- (- ( 1)) - - -
2
4 4 4
1
9
2 3
( 1) -
4
4 8
9
1
2 3
d ( 1) - -
dq ( )
1
4
4 8 1 1

- 0 3 .
d
d
2 4
2

( 1) - con. Thus , (x4 , y4 , z4 ) (x3 , y3 , z3 ) 3 r3

1 3 3 3
3 1 3 1
- , , 0 , - , 0 - , - , - .
2 4 4 4
4 2 4 2
Find the next search direction:
r4 -f (x4 , y4 , z4 ) - 2x4 - y4 1, 2y4 - x4 - z4 , 2z4 - y4 1

T
T

3
3
3
3
3
3
3

- 2(- ) - (- ) 1, 2(- ))- (- )) (- ) , 2 (- ) - ( - ) 1

4
4
4
4
4
4
4

T
1
1
- , 0, -
(x5 , y5 , z5 ) (x4 , y4 , z4 ) r4
4
4
1 1
3 1
3 3 3 1

- , , - , 0 , - ( 3) , - , - ( 3) .
4 4
4 4
4 4 4 4

( 1) - con. Find 4 by
f (x5 , y5 , z5 ) x52 x5 ( 1 - y5 ) y52 - y5 z5 z52 z5 q( )).
2

3
3 2
1
1

q( ) - ( 3) - ( 3) ( 1 - (- )) (- )
4
4
4
4

3 1
1
1

- (- ) - ( 3) - ( 3) - ( 3)
4 4
4
4

1
51
2 7
( 3) -
8
8 16
51
5
1
2 7
d ( 3) - -
1
dq( )
8
8 16 1 1

- 0 4 .
d
d
4 8
2
1 7 3 7
3 3 3 1 1
Thus , (x5 , y5 , z5 ) - , , - , 0 , - , - , - .
4 8 4 8
4 4 4 24

( 1) - con. Find the next search direction:

r5 -f (x5 , y5 , z5 ) - 2x5 - y5 1, 2y5 - x5 - z5 , 2z5 - y5 1
T

3
3
7
7
7
3
1
7

- 2(- ) - (- ) 1, 2(- )- (- )- (- ) , 2(- ) - (- ) 1 0 , - , 0 .
8
4
4
8
8
8
4
4

7 3 7
1
(x6 , y6 , z6 ) (x5 , y5 , z5 ) r5 - , - , - 0 , - , 0
4
8 4 8

7
7 1
- , - ( 3) , - .
8
8 4
Find 5 by f (x6 , y6 , z6 ) x62 x6 ( 1 - y6 ) y62 - y6 z6 z62 z6 q( ).
2

1
1
7 7
q( ) - - ( 1 - (- ( 3))) (- ( 3)) 2
4
4
8 8
2

1
49
7 7 7 1
2 7
- ( - ( 3 )) - - -
( 3) - - .
4
16 32
8 8 8 16

7
49
1
d ( 3) 2 - -
dq ( )
1
16
16 32 1

( 1) - con.

0
d
d
8 16
1
5 (x6 , y6 , z6 ) (x5 , y5 , z5 ) 5 r5
2
7 3 7 1 1 7 7 7
- , , - 0 , , 0 - , , .
8 4 8 2 4 8 8 8
Since (x6 , y6 , z6 ) is not very close to (x , y , z ) ( - 1, -1, -1) ,

(1) - con.
con

1
6
2

1
7 7 7 1 1
(x7 , y7 , z7 ) (x6 , y6 , z6 ) 6 r6 - , , - , 0 ,
8
8 8 8 28
15 7 15
- , , .
16 8 16
15 7 15
1
(x8 , y8 , z8 ) (x7 , y7 , z7 ) 7 r7 - , - , - - 7 0 , , 0
16 8 16
8
15
15 1
- , - (7 7 ) , - .
16
16 8

( 1) - con. Find 7 by f (x8 , y8 , z8 )

15 1
15
1
195
2 15
f ( - , - ( 7 ) , - ) q( )
( 7 ) - .
16 8
16
64
64 128
15 195
1
d ( 7 ) 2 -
1
dq ( )
64
64
128

0 7
d
d7
2
1 15 15 15
15 7 15 1
(x8 , y8 , z8 ) - , - , - 0 , - , 0 - , - , - .
8 16
16 16 16
16 8 16 2
If we think that (x8 , y8 , z8 ) is very close to (x , y , z ) ( - 1, -1, -1) ,
we can check the convergence criteria by evaluating
f ( x8 , y8 , z8 ) 2x8 - y8 1, 2y8 - x8 - z8 , 2z8 - y8 1
1 16 , 0 , 1 16 ( 1 16 ) 2 (0 ) 2 ( 1 16 ) 2 2 16 0.0884 ,
which is quite small.
We may take it to be close enough to solution in Example 2.

( 2 ) Solution:
Fi l , we can rewrite
Firstly
i f (x , y , z ) to a quadractic
d
i form
f
by
b
1 T
f (x , y , z ) x Ax bT x
2
2 -1 0 x
x
1
(x , y , y) -1 2 -1 y 1, 0 , 1 y ,
0 -1 2 z
z
2

2 -1 0
-1
thus , we have A -1 2 -1 and b 0 .
0 -1 2
-1

( 2 ) - con. At the begining of iterations, we calculate the residual

vector r0 from x0
-1 2 -1 0 0 -1
r0 b - Ax0 r0 0 - -1 2 -1 0 0 .
-1
0 -1
0 -1

1
1
2
1


Next , we use r0 as our initial search direction p0 , and compute 0
1
-1
-1, 0 , -1 0

-1
r0T r0
2 1

0 T

(same as ( 1)).
)
p0 Ap0
2 -1 0 -1 4 2
-1, 0 , -1 -1 2 -1 0
1 2 -11
0 -1
We can now compute (x1 , y1 , z1 ) (x0 , y0 , z0 ) 0 p0
1
1
1
(0 ,0 ,0 ) -1, 0 , -1 - , 0 , - .
2
2
2
Thi result
This
lt completes
l t the
th first
fi t iteration.
it ti

( 2 ) - con. So we can compute

-1 1 2 -1 0 -1 0
r1 r0 - 0 Ap0 0 - -1 2 -1 0 -1 ,
-1 2 0 -1 2 -1 0

Next , computing the scalar 0 and the next search direction p1 .
0 , -1, 0 0 , -1, 0

1 r r
1
0

, and
T
0 r r -1, 0 , -1 -1, 0 , -1
2
T
1
1
T
T
1
p1 r1 0 p0 0 , -1, 0 -1, 0 , -1 - , -1, - .
2
2
2
We now compute the scalar 1 using p1 .
T
1 1
T
0 0

T
1 1
T
1
1

r r
1

p Ap
1
- , -1,
2

0 , -1, 0 0 , -1, 0

1
1.
T
1
1 2 -1 0 1
1
- -1 2 -1 - , -1, -
2 0 -11 2 2
2

( 2 ) - con.
Using 1 , we can find (x2 , y2 , z2 ) by
(x2 , y2 , z2 ) (x1 , y1 , z1 ) 1 p1
1
1
1
1
- , 0 , - 1 - , -1, - -1, -1, -1 .

2
2
2
2
(x2 , y2 , z2 ) is the minimum point of f (x , y , z ).
So we only use the two iterations to find (x , y , z ).