Homework 1 Solutions – ECE6553, Spring 2011

1
Let v be the direction, with kvk = 1. The growth in h along this direction is (ignoring higher order terms)

∂h ∂h
h(u + v) = h(u) + v = h(u) + ∂u cos θ,

∂u

∂h T
where θ is the angle between ∂u and v. This is maximized if θ = 0 i.e., if the vectors are parallel. In
∂h T
order words, the maximizing v is the one that points in the same direction as ∂u .

2
The Lagrangian is
1 T
`(u, λ) = u Qu − bT u + λ(Au − c).
2
The FONC is
∂`
= uT Q − bT + λA = 0 ⇒ u = Q−1 (b − AT λT ).
∂u
Plugging this back into the constraint gives

AQ−1 (b − AT λT ) = c ⇒ λT = (AQ−1 AT )−1 (AQ−1 b − c)

with the optimal u given by

u = Q−1 b − AT (AQ−1 AT )−1 (AQ−1 b − c) .

 

3
a
F (M + N ) = (M + N )T (M + N )(M + N )T = (M T M + M T N + N T M + o())(M + N )T
= M T M M T + M T M N T + M T N M T + N T M M T + o().
Hence
δF (M ; N ) = M T M N T + M T N M T + N T M M T .

b
∂f
f (x + y) = f (x) +  y + o(),
∂x
i.e.,
f (x + y) − f (x) ∂f
δf (x; y) = lim = y.
→0  ∂x

1
4
The volume of the box is given by xyz, while the total cardboard area is 2xz + 2xy + 2yz. The first
thing to realize is that we should, of course, use as much area as possible. In other words, the inequality
constraint is really an equality constraint. In other words, solve
max xyz
s.t. xz + xy + yz = 2c .
The Lagrangian is  c
` = xyz + λ xz + yz + xy − .
2
Let u = (x, y, z)T .
∂`
= (yz + λ(y + z), xz + λ(x + z), xy + λ(x + y)) = (Expr.1,Expr.2,Expr.3) = 0.
∂u
Note that
Expr.1 + Expr.2 + Expr.3 = (xy + yz + xz) + 2λ(x + y + z) = 0,
which means that
c + 4λ(x + y + z) = 0
and hence that λ 6= 0. But, x = 0 ⇒ z = 0 (Expr.2) and y = 0 (Expr.3) and hence x, y, z 6= 0 as well.
Now, x · Expr.1 − y · Expr.2 = λ(x − y)z = 0 ⇒ x = y. Similarly y · Expr.2 − z · Expr.3 = 0 ⇒ y = z.
Hence the optimal box is a cube with sides
r
c
x=y=z= .
6

5
Assume u? is a local minimum. Hence
L(u? ) ≤ L(u? + u),
for all u (as long as  is small enough). Computing the directional derivative gives
1
δL(u? ; u) = lim (L(u? + u) − L(u)) = [linear in u] ≥ 0, ∀u.
→0 

But, the only way something linear in u can non-negative for all u is if the linear thing is 0. Otherwise, one
can always pick a u to make the expression negative, which would contradict the optimality assumption.
In other words, the FONC is
δL(u? ; u) = 0, ∀u.

6
By definition we have that
L(αu + (1 − α)u? ) ≤ αL(u) + (1 − α)L(u? ).
Hence
L(u? + α(u − u? )) − L(u? )
≤ L(u) − L(u? ).
α
Letting α → 0 gives that
∂L ?
L(u) ≥ L(u? ) + (u )(u − u? ) = L(u? ), ∀u,
∂u
and the global minimization result follows.