M31

4/15/22, 1:25 PM M31(HW) Least -squares Problems (6.
5)-Kyle Hoang
Instructor: Larissa Williamson

Student: Kyle Hoang Course: MAS3114- Assignment: M31(HW) Least -squares
Date: 04/15/22 1888,0329(15272,28798) - Computatnl Problems (6.5)
Linear Alg
Find aleast-squares solution of Ax = b by (a) constructing the normal equations for x and (b) solving for x .
−1 2 7
A= 2 − 3 , b = 14
−1 3 2
(a) The set ofleast-squares solutions of Ax = b coincides with the nonempty set of solutions of the normal equations
T T
A Ax = A b. A solution of these equations is often denoted by x .
T
To find the normalequations, first compute A A.
−1 2
T −1 2 −1
A A= 2 −3
2 −3 3
−1 3
T 6 − 11
A A=
− 11 22
T
Now compute A b.
7
T −1 2 −1
A b= 14
2 −3 3
2
T 19
A b=
− 22
Thus, the normal equations for x are as shown below.
6 − 11 x1 19
=
− 11 22 x2 − 22
6 − 11
(b) The easiest way to solve the normal equations shown in the previous step is to use the inverse of if it exists.
− 11 22
6 − 11 6 − 11
A 2 × 2 matrix A is invertible if and only if det(A) ≠ 0. Because = 11 ≠ 0, it follows that is invertible.
− 11 22 − 11 22
This means that the normal equations for x can be solved for x as shown below.
https://xlitemprod.pearsoncmg.com/api/v1/print/highered 1/2
4/15/22, 1:25 PM M31(HW) Least -squares Problems (6.5)-Kyle Hoang
6 − 11 x1 19
=
− 11 22 x2 − 22
−1
6 − 11 x1 6 − 11 19
=
− 11 22 x2 − 11 22 − 22
Compute the inverse.
−1 2 1
6 − 11
= 6
− 11 22 1
11
Substitute this result into the formula foundpreviously, then simplify.
−1
x1 6 − 11 19
=
x2 − 11 22 − 22
x1 2 1
19
= 6
x2 1 − 22
11
x1 16
=
x2 7
x1 16 16
Thus, = . Thatis, x = .
x2 7 7

Linear Alg
Find aleast-squares solution of Ax = b by (a) constructing the normal equations for x and (b) solving for x .
1 2 6
A= 1 − 1 , b = 2
1 1 0
(a) The set ofleast-squares solutions of Ax = b coincides with the nonempty set of solutions of the normal equations
T T
A Ax = A b. A solution of these equations is often denoted by x .
T
To find the normalequations, first compute A A.
1 2
T 1 1 1
A A= 1 −1
2 −1 1
1 1
T 3 2
A A=
2 6
T
Now compute A b.
6
T 1 1 1
A b= 2
2 −1 1
0
T 8
A b=
10
Thus, the normal equations for x are as shown below.
3 2 x1 8
=
2 6 x2 10
3 2
(b) The easiest way to solve the normal equations shown in the previous step is to use the inverse of if it exists.
2 6
3 2 3 2
A 2 × 2 matrix A is invertible if and only if det(A) ≠ 0. Because = 14 ≠ 0, it follows that is invertible.
2 6 2 6
This means that the normal equations for x can be solved for x as shown below.
3 2 x1 8
=
2 6 x2 10
−1
3 2 x1 3 2 8
=
2 6 x2 2 6 10
Compute the inverse.
3 1
−1 −
3 2 7 7
=
2 6 1 3
−
7 14
Substitute this result into the formula foundpreviously, then simplify.
−1
x1 3 2 8
=
x2 2 6 10
3 1
x1 −
7 7 8
=
x2 1 3 10
−
7 14
x1 2
=
x2 1
x1 2 2
Thus, = . Thatis, x = .
x2 1 1

Linear Alg
Describe allleast-squares solutions of the equation Ax = b.
1 0 1 12
1 0 1 8
A= , b =
1 1 0 1
1 1 0 1
Recall that the set ofleast-squares solutions of Ax = b coincides with the nonempty set of solutions of the normal equations
T T
A Ax = A b. Solutions of these equations are often denoted by x .
T
First compute A .
1 1 1 1
T =
A 0 0 1 1
1 1 0 0
T
Then compute A A.
1 0 1
1 1 1 1 4 2 2
T
1 0 1
A A = 0 0 1 1 = 2 2 0
1 1 0
1 1 0 0 2 0 2
1 1 0
T
Now compute A b.
12
1 1 1 1 22
T
8
A b = 0 0 1 1 = 2
1
1 1 0 0 20
1
T T
To find theleast-squares solutions to Ax = b, row reduce the augmented matrix for the system A Ax = A b.
Write the augmented matrix T T

A A A b .
4 2 2 22
T T = 2 2 0 2
A A A b
2 0 2 20
Reduce the augmented matrix to reduced row echelon form.
4 2 2 22 1 0 1 10
2 2 0 2 ~ 0 1 −1 −9
2 0 2 20 0 0 0 0
Columns 1 and 2 are pivotcolumns, so x3 is a free variable. The general solution for the augmented matrix has x3 as a free
variable.
The general solution is x1 = 10 − x3 , x2 = − 9 + x3 , and x3 is free.
Write the general solution as a column vector. Then write the result as the sum of two column vectors and factor the scalar x3
from the second vector.
The generalleast-squares solutions of Ax = b for the given matrix A and vector b are all vectors of the form
10 −1
x= −9 + x3 1 , with x3 free.
0 1

Linear Alg
1 2 6
2
For A = 1 −1 and b = 2 , aleast-squares solution of Ax = b is x = . Compute theleast-squares error associated
1
1 1 0
with this solution.
When a least squares solution x is used to produce Ax as an approximation to b, the distance from b to Ax is called the
least-squares error of this approximation.
First, approximate b by computing Ax .
1 2
2
Ax = 1 −1
1
1 1
4
Ax = 1
3
The distance from b to Ax , written as dist(b,Ax ) is the length of the vector b − Ax . Thatis, dist(b,Ax ) = b − Ax .
The expression below is used to compute the vector b − Ax .
6 4
2 − 1
0 3
Compute b − Ax .
6 4
b − Ax = 2 − 1
0 3
2
b − Ax = 1
−3
Use this result to write an expression for b − Ax .
2 2 2
2 + 1 + ( − 3)
Compute b − Ax .
2 2 2
b − Ax = 2 + 1 + ( − 3)
b − Ax = 14
2
Thus, theleast-squares error associated with the solution x = is 14 .
1

Linear Alg
Find(a) the orthogonal projection of b onto Col A and(b) aleast-squares solution of Ax = b.
−1 4 14
A= 1 8 , b = −4
−1 4 0
a. The columns a1 and a2 of A are shown below.
−1 4
a1 = 1 , a2 = 8
−1 4
Since a1 • a2 = 0, a1 and a2 orthogonal.
Because a1 and a2 areorthogonal, the orthogonal projection of b onto Col A, b, is given by the formula shown below.
b • a1 b • a2
b= a1 + a
a1 • a1 a2 • a2 2
Compute the inner products that appear in this formula. First 14 −1 4

compute b • a1 and b • a2 .
b= − 4 , a1 = 1 , a2 = 8
b • a1 b • a2 0 −1 4
b= a1 + a2
a1 • a1 a2 • a2
− 18 24
b= a1 + a
a1 • a1 a2 • a2 2
Now compute a1 • a1 and a2 • a2 . −1 4

a1 = 1 , a2 = 8
− 18 24
b= a1 + a2
a1 • a1 a2 • a2 −1 4
− 18 24
b= a1 + a
3 96 2
Thus, b is given by the formula shown below.
1
b = − 6a1 + a
4 2
−1 4
1
b= −6 1 + 8
4
−1 4
Simplify this expression.
−1 4 7
1
b= −6 1 + 8 = −4
4
−1 4 7
7
Thus, the orthogonal projection of b onto Col A is b = −4 .
7
b. Since b is in the column space ofA, the equation Ax = b isconsistent, and there is an x in ℝn such that Ax = b. Since b is the
closest point in Col A to b, a vector x in ℝn is a list of weights that will build b out of the columns of A.
−1 4
1
Recall from part (a) that b = − 6 1 + 8 .
4
−1 4
1
The weights that must be placed on columns a1 and a2 of A to produce b are − 6 and , respectively.
4
−6
Thus, aleast-squares solution of Ax = b is x = 1 .
4

Linear Alg
Find(a) the orthogonal projection of b onto Col A and(b) aleast-squares solution of Ax = b.
0 1 −1 2
1 1 1 3
A= , b =
−1 0 1 3
−1 1 0 2
a. The columns a1 , a2 , and a3 of A are shown below.
0 1 −1
1 1 1
a1 = , a2 = , a3 =
−1 0 1
−1 1 0
Since a1 • a2 = 0, a1 • a3 = 0, and a2 • a3 = 0, it follows that a1 , a2 , and a3 are orthogonal.
Because a1 , a2 , and a3 areorthogonal, the orthogonal projection of b onto ColA, b, is given by the formula shown below.
b • a1 b • a2 b • a3
b= a1 + a2 + a
a1 • a1 a2 • a2 a3 • a3 3
Compute the inner products that appear in this formula. First 2 0 1 −1

compute b • a1 , b • a2 , and b • a3 .
3 1 1 1
b= , a1 = , a2 = , a3 =
b • a1 b • a2 b • a3 3 −1 0 1
b= a1 + a2 + a3
a1 • a1 a2 • a2 a3 • a3 2 −1 1 0
−2 7 4
b= a1 + a2 + a
a1 • a1 a2 • a2 a3 • a3 3
Now compute a1 • a1 , a2 • a2 , and a3 • a3 . 0 1 −1
−2 7 4 1 1 1
b= a1 + a2 + a3 a1 = , a2 = , a3 =
a1 • a1 a2 • a2 a3 • a3 −1 0 1
−2 7 4 −1 1 0
b= a1 + a + a
3 3 2 3 3
Thus, b is given by the formula shown below.
2 7 4
b= − a1 + a2 + a
3 3 3 3
0 1 −1
2 1 7 1 4 1
b= − + +
3 −1 3 0 3 1
−1 1 0
Simplify this expression.
0 1 −1 1
2 1 7 1 4 1 3
b= − + + =
3 −1 3 0 3 1 2
−1 1 0 3
1
3
Thus, the orthogonal projection of b onto Col A is b = .
2
3
b. Since b is in the column space ofA, the equation Ax = b isconsistent, and there is an x in ℝn such that Ax = b. Since b is the
closest point in Col A to b, a vector x in ℝn is a list of weights that will build b out of the columns of A.
0 1 −1
2 1 7 1 4 1
Recall from part(a) that b = − + + .
3 −1 3 0 3 1
−1 1 0
2 7 4
The weights that must be placed on columns a1 , a2 , and a3 of A to produce b are − , , and , respectively.
3 3 3
2
−
3
7
Thus, aleast-squares solution of Ax = b is x = .
3
4
3

Linear Alg
10 2 5
2 2
Let A = − 5 1 , b = − 4 , u = , and v = . Compute Au and Av, and compare them with b. Could u possibly be
−7 −8
10 2 3
aleast-squares solution of Ax = b?
(Answer this without computing aleast-squares solution.)
Use matrix multiplication to find Au and Av. Start with Au.
10 2 6
2
−5 1 = − 17
−7
10 2 6
Now find Av.
10 2 4
2
−5 1 = − 18
−8
10 2 4
Now, compare Au and Av with b to decide whether v can be aleast-squares solution of Ax = b.
If A is an m × n matrix and b is in ℝm , aleast-squares solution of Ax = b is an x in ℝn such that b − Ax ≤ b − Ax for all x in

n
ℝ . When theleast-squares solution x is used to produce Ax as an approximation to b, the distance b − Ax from b to Ax is
called theleast-squares error of this approximation.
Find and compare b − Au and b − Av . Start with b − Au by first finding b − Au.
5 6 −1
−4 − − 17 = 13
3 6 −3
Now find the magnitude of the result.
2 2 2
b − Au = ( − 1) + (13) + ( − 3)
b − Au = 179
Find b − Av by first finding b − Av.
5 4 1
−4 − − 18 = 14
3 4 −1
Now find the magnitude of the result.
2 2 2
b − Av = (1) + (14) + ( − 1)
b − Av = 3 22
Compare the results.
179 < 3 22
So, b − Au < b − Av . Thus, Au is a better approximation of b because it has a smaller error than Av.
Therefore, the vector u could be aleast-squares solution of Ax = b because both u and v exist in ℝ2 , and b − Au < b − Av .

Linear Alg
8. Decide whether statements(a) through(e) are true or false. Justify each answer. Assume A is an m × n matrix and b is in
m
ℝ .
a. The generalleast-squares problem is to find an x that makes Ax as close as possible to b.
Choose the correct answer below.
A. The statement is false because the generalleast-squares problem attempts to find an x that
maximizes b − Ax .
B. The statement is true because the generalleast-squares problem attempts to find an x that
minimizes b − Ax .
C. The statement is false because the generalleast-squares problem attempts to find an x such
that Ax = b.
D. The statement is true because the generalleast-squares problem attempts to find an x such that
Ax = b.
b. Aleast-squares solution of Ax = b is a vector x that satisfies Ax = b, where b is the orthogonal projection of b onto Col A.
A. The statement is false because Ax = b and Ax = b are not equivalent.
The statement is false because b is not the closest point in Col A to b. So, Ax = b is not
B.
necessarily consistent.
The statement is true because b is the closest point in Col A to b. So, Ax = b is consistent and x
C.
such that Ax = b is aleast-squares solution of Ax = b.
D. The statement is true because Ax = b and Ax = b are equivalent.
c. Aleast-squares solution of Ax = b is a vector x such that b − Ax ≤ b − Ax for all x in ℝn .

A. The statement is true because the generalleast-squares problem attempts to find an x that
minimizes b − Ax .
B. The statement is true because the generalleast-squares problem attempts to find an x that
maximizes b − Ax .
C. The statement is false because aleast-squares solution of Ax = b is a vector x such that
b − Ax < b − Ax for all x in ℝn .
D. The statement is false because aleast-squares solution of Ax = b is a vector x such that
b − Ax ≤ b − Ax for all x in ℝn .
T T
d. Any solution of A Ax = A b is aleast-squares solution of Ax = b.
A. The statement is false because the set ofleast-squares solutions of Ax = b coincides with the
T T
nonempty set of solutions of the normalequations, defined as A x = A Ab.
B. The statement is true because AT Ax = AT b is equivalent to Ax = b.
C. The statement is true because the set ofleast-squares solutions of Ax = b coincides with the
T T
nonempty set of solutions of the normalequations, defined as A Ax = A b.
D. The statement is false because the set ofleast-squares solutions of Ax = b coincides with the
T
nonempty set of solutions of the normalequations, defined as A x = b.
e. If the columns of A are linearlyindependent, then the equation Ax = b has exactly oneleast-squares solution.
A. The statement is false because if the columns of A are linearlyindependent, there are infinitely
manyleast-squares solutions to Ax = b.
B. The statement is false because theleast-squares solution is only guaranteed to be unique if the
columns of A are orthogonal.
C. The statement is true because if the columns of A are linearlyindependent, then AAT is
T −1 T
invertible and x = AA A b is theleast-squares solution to Ax = b.
D. The statement is true because if the columns of A are linearlyindependent, then AT A is
T −1 T
invertible and x = A A A b is theleast-squares solution to Ax = b.

M31

Uploaded by

Copyright:

Available Formats

You might also like

M31

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

M31

Uploaded by

Copyright:

Available Formats

4/15/22, 1:25 PM M31(HW) Least -squares Problems (6.

Instructor: Larissa Williamson

Thus, the normal equations for x are as shown below.

Compute the inverse.

Substitute this result into the formula found​previously, then simplify.

Instructor: Larissa Williamson

Thus, the normal equations for x are as shown below.

Compute the inverse.

Substitute this result into the formula found​previously, then simplify.

Instructor: Larissa Williamson

Describe all​least-squares solutions of the equation Ax = b.

Write the augmented matrix T T

Reduce the augmented matrix to reduced row echelon form.

The general solution is x1 = 10 − x3 ​, x2 = − 9 + x3 ​, and x3 is free.

Instructor: Larissa Williamson

First, approximate b by computing Ax .

The expression below is used to compute the vector b − Ax .

Use this result to write an expression for b − Ax .

Instructor: Larissa Williamson

Find​(a) the orthogonal projection of b onto Col A and​(b) a​least-squares solution of Ax = b.

a. The columns a1 and a2 of A are shown below.

Since a1 • a2 = ​0, a1 and a2 orthogonal.

Compute the inner products that appear in this formula. First 14 −1 4

Now compute a1 • a1 and a2 • a2 . −1 4

Thus, b is given by the formula shown below.

Simplify this expression.

Instructor: Larissa Williamson

Find​(a) the orthogonal projection of b onto Col A and​(b) a​least-squares solution of Ax = b.

a. The columns a1 ​, a2 ​, and a3 of A are shown below.

Compute the inner products that appear in this formula. First 2 0 1 −1

Now compute a1 • a1 ​, a2 • a2 ​, and a3 • a3 . 0 1 −1

Thus, b is given by the formula shown below.

Instructor: Larissa Williamson

Use matrix multiplication to find Au and Av. Start with Au.

Now find Av.

Now, compare Au and Av with b to decide whether v can be a​least-squares solution of Ax = b.

If A is an m × n matrix and b is in ℝm ​, a​least-squares solution of Ax = b is an x in ℝn such that b − Ax ≤ b − Ax for all x in

Find and compare b − Au and b − Av . Start with b − Au by first finding b − Au.

Now find the magnitude of the result.

Find b − Av by first finding b − Av.

Now find the magnitude of the result.

Compare the results.

Instructor: Larissa Williamson

a. The general​least-squares problem is to find an x that makes Ax as close as possible to b.

Choose the correct answer below.

Choose the correct answer below.

A. The statement is false because Ax = b and Ax = b are not equivalent.

D. The statement is true because Ax = b and Ax = b are equivalent.

c. A​least-squares solution of Ax = b is a vector x such that b − Ax ≤ b − Ax for all x in ℝn .

Choose the correct answer below.

Choose the correct answer below.

You might also like

Substitute this result into the formula foundpreviously, then simplify.

Substitute this result into the formula foundpreviously, then simplify.

Describe allleast-squares solutions of the equation Ax = b.

The general solution is x1 = 10 − x3 , x2 = − 9 + x3 , and x3 is free.

Find(a) the orthogonal projection of b onto Col A and(b) aleast-squares solution of Ax = b.

Since a1 • a2 = 0, a1 and a2 orthogonal.

Find(a) the orthogonal projection of b onto Col A and(b) aleast-squares solution of Ax = b.

a. The columns a1 , a2 , and a3 of A are shown below.

Now compute a1 • a1 , a2 • a2 , and a3 • a3 . 0 1 −1

Now, compare Au and Av with b to decide whether v can be aleast-squares solution of Ax = b.

If A is an m × n matrix and b is in ℝm , aleast-squares solution of Ax = b is an x in ℝn such that b − Ax ≤ b − Ax for all x in

a. The generalleast-squares problem is to find an x that makes Ax as close as possible to b.

c. Aleast-squares solution of Ax = b is a vector x such that b − Ax ≤ b − Ax for all x in ℝn .