Linear Algebra

Systems of Linear Equations

and Matrices
 Linear Equation
A linear equation is an equation in which
each term is either a constant or the
product of a constant times the first power
of a variable. Such an equation is
equivalent to equating a
first-degree polynomial to zero. Some
examples of linear equations are as
follows:
•
•
•

2
Linear System
A linear system (or system of linear
equations) is a collection of linear
equations involving the same set of
variables. For example,

is a system of three equations in the

three variables  .
3
A linear system in three
variables determines a
collection of planes.

4
A linear system in two
variables determines
a collection of
straight lines.

5
Matrices
 Variety of engineering problems lead to the
need to solve systems of linear equations
Ax  b
where
 a11 a12 a13 
a1n   x1   b1 
     
 a21 a22 a23 a2 n   x2   b2 
A   x    b   
     
     
 am1 am 2 am3  amn   xn   bn 

matrix column vectors

6
Matrix
 MATRIX - a rectangular array of numbers
arranged into rows and columns. A matrix
with m rows and n columns can be written as:
 a11 a12 a13 
a1n 
 
Rows,  a21 a22 a23 a2 n 
j = 1, …, m
    
  Columns,
 am1 am 2 am3  amn 
k = 1, …, n

Matrices are useful as they enable us to consider an

array of numbers as single object, denote is by a single
symbol, and perform calculations with these symbols. 7
Examples
 These are valid matrices

1 0  a b 1 2 3 
0 1  c e 0 1 0 
     

 These have problems

 11 
1 2  4   7 
3  5 6  
   
8 9
 10 
 
8
Matrix Notations
 Denote matrices by Capital Boldface Letters
A,B,C, … or by writing the general entry in
brackets like A  [a ] . A matrix A with m rows
jk

and n columns can also be written as:

 a11 a12 a13  a1n 

 
 a21 a22 a23 a2 n 
A  a jk  
   
 
 am1 am 2 am 3  amn 

9
 In the double subscript notation for the
entries, the first subscript always denotes the
row and the second the column in which the
given entry stands. Thus a23 is the entry in the
second row and third column.
 Vectors are represented by lowercase
boldface letters like a, b, c, …

10
Row and Column Matrices
(vectors)
 ROW MATRIX (or row vector) is a matrix with
one row, i.e., the dimension m = 1

r  r1 r2 r3  rn 
 COLUMN VECTOR is a matrix with only one
column
 c1 
c 
c  2

 
 
cm  11
Square Matrix
 When the row and column dimensions of a
matrix are equal (m = n) then the matrix is
called square

 a11 a12  a1n 

 
 a21 a22 a2 n 
A   
 
 
 an1 an 2  ann 

12
Special Matrices
u11 u12 u13 u14 
0 u u23 u24 
 Upper Triangular Matrix U 22

0 0 u33 u34 
 
0 0 0 u44 
 l11 0 0 0
l l22 0 0 
 Lower Triangular Matrix  21
L
l31 l32 l33 0
 
l41 l42 l43 l44 
13
Diagonal Matrices
 a11 0 0 0 
 
 Diagonal Matrix  0 a22 0 0 
A   
0 0  0
 
 0 0 0 ann 

 a11 a12 0 0 0 
 
 a21 a22 a23 0 0 
 Tri-diagonal Matrix A   0 a32  0 
 
 0 0   a45 
 0 0 0 a54 a55 


14
Identity Matrix

 Identity Matrix 1 0 0 0
0 1 0 0
I 
0 0 1 0
0 0 0 1


 The identity matrix has the property that

if A is a square matrix, then

IA  AI  A

15
Matrix Transpose
 The transpose of the (m x n) matrix A is the (n x
m) matrix formed by interchanging the rows and
columns such that row i becomes column i of the
transposed matrix

 a11 a12  a1n   a11 a21  am1 

   
a a22 a2 n   a12 a22 am 2 
A   21 
T
A   
   
   
 am1 am 2  amn   a1n a2n  amn 

16
Example - Transpose

1 3 T 1 2
A  A  
 2 5 3 5 

1 0 
1 3 4 
A  AT  3 1
0 1 0  
  4 0

17
Matrix Equality
 Two (m x n) matrices A and B are equal if
and only if they have the same size and each
of their elements are equal. That is

A=B
if and only if
ajk = b j k for j = 1,...,m; k = 1,...,n

18
Vector Addition
 The sum of two (m x 1) column vectors a and
b is

 a1   b1   a1 + b1 
     
 a2   b2   a2 + b2 
a  b        
   
     
 am   bm   am + bm 

19
Examples - Vector Addition
1 3
 3 5
u  v 
2   1
4   2
   

 1   3   1  3   4
 3  5   3  5 2
uv        
 2    1   2  1  1 
 4    2  4  2   2
       
20
Matrix Addition
 a11 a12  a1n   b11 b12  b1n 
   
 a21 a22 a1n   b21 b22 b1n 
AB 
       
   
 am1 am 2  amn   bm1 bm 2  bmn 

 a11  b11 a12  b12  a1n  b1n 

 
 a21  b21 a22  b22 a1n  b1n 

    
 
 am1  bm1 am 2  bm 2  amn  bmn 
21
Examples - Matrix Addition
1 2 3   3 2 1
A   2 1 4 B   4 1 2
   
1 4 3  2 3 1 

 1 2 3   3 2 1  4 4 4 
A  B  2 1 4    4 1 2    2 2 6 
     
 1 4 3   2 3 1  3 7 4 

 The following matrix addition is not defined

1 2 2 4 6
5 2  1 3 5  ? (not defined!)
   
22
Scalar – Matrix Multiplication
 Multiplication of a matrix A by a scalar is
defined as
 a11 a12  a1n 
 
 a21 a22 a2 n 
A   
 
 
 am1 am 2  amn 

 Examples
1 2  4 8 
  4, A  , A   0 4 
 0 1   

1 4 1   2  8  2
  2, A    , A    4 0  6 
 2 0 3   23
Matrix – Matrix Multiplication
 The product of two matrices A and B is defined only if
the number of columns of A is equal to the number of
rows of B.
 If A is (m x p) and B is (p x n), the product is an (m x
n) matrix C
C mxn  Amxp B pxn

24
Matrix – Matrix Multiplication
a a11 12
 a  1r

a a  a  b b  b  b 
 21 22
 2r 11 2 1j 1n

      b b  b  b 
C  AB = 
21 12 2j 2n


a ai1 i2
 a    
ir

    
   b b  b  b  r1 r2 rj rn
 
a am1 m2
 a  mr

a b  a b
11 11 1r r1
a b  a b
11 12
 a b  a b 
1r r2 11 1n 1r rn

a b   a b a b  a b  a b  a b 
 21 11 2r r1 21 12 2r
 r2 21 1n 2r rn

    
 
 a b  a b  i1 1j ir rj

    
 
a b    a b
m1 11 mr r1
a b   a b
m1 12
 a b  a b 
mr r2 m1 1n mr rn

c  a b  a b  a b  a b
ij i1 1j i2 2j ir rj
k 1
ik kj
25
 Example –
Matrix Multiplication
 b1 
 
 b2  c1x1  a1xn bnx1 (scalar)
a  (a1, a2 ,, an ) b 

 
 bn 

 b1 
 
 b2 
c  a  b  (a1 , a2 ,, an )   a1b1  a2b2    an bn

 
 bn 

26
 Example –
Matrix Multiplication
 a11 a12  a1n   b1 
a   
21 a 22  a 2n   b2  c nx1  Anxn bnx1
A b 
   
a   
a
 n1 n 2  a nn   bn 

 a11 a12  a1n  b1  a11b1  a12b2   a1n bn

a   
 21 a22  a2n   b2  a11b1  a22b2   a2n bn
c  Ab    
   
a   
 n1 an 2  ann  bn  an1b1  an 2b2   a nnbn

27
Example –
Matrix Multiplication
1 3   2 1
A  B  C 2 x 2  A2 x 2 B2 x 2
 2 4   3 5

1 3 2 1  1  2  3  3 1 1  3  5  11 16 
C  A B         
2 4 3 5 2  2  4  3 2 1  4  5 16 22

28
Example –
Matrix Multiplication
1 2 3  2 1
A  2 1 4 B  1 2 
C 3 x 2  A3 x3 B3 x 2
   
1 4 3 2 1 

1 2 3   2 1 
C  A  B   2 1 4  1 2 
  
1 4 3 2 1 
1  2  2 1  3  2 1 1  2  2  3 1 10 8 
 2  2  1 1  4  2 2 1  1  2  4 1  13 8 
   
1  2  4 1  3  2 1 1  4  2  3 1 12 12
29
Example –
Matrix Multiplication
3 0 
4 7
A  1 1  B  C 3 x 2  A3 x 2 B2 x 2
   6 8 
5 2

3 0  12 21
  4 7  
C  A B  1 1   10 15
  6 8   
5 2 32 51

30
Matrix Inverse
 If A is an (n x n) square matrix and there is a
matrix X with the property that
AX  I

 X is defined to be the inverse of A and is

denoted A-1

AA1  I A1 A  I

31
 Example - Matrix Inverse
AA1  I

• Example (2 x 2) matrix

 a11 a12 
A  
a a
 21 22 

1 1  a22  a12 
A 
a11a22  a12a21  a21 a11 

32
System of Linear Equations
 A finite set of linear equations in the variables
x1 , x2 ,..., xn is called a system of linear equations or
a linear system.
a11 x1  a12 x2  ...  a1n xn  b1
a21 x1  a22 x2  ...  a2 n xn  b2
   
am1 x1  am 2 x2  ...  amn xn  bm
An arbitrary system of m
linear equations in n unknowns

33
Matrix Form of the Linear System

where
Ax  b

 a11 a12 a13 

a1n   x1   b1 
     
a a22 a23 a2 n   x2   b2 
A   21 x    b   
     
     
 am1 am 2 am3  amn   xn   bn 

matrix
column vectors

34
 A sequence of numbers s1 , s2 ,..., sn that satisfies the
system of equations is called a solution of the
system.

 A system that has no solution is said to be

inconsistent ; if there is at least one solution of the
system, it is called consistent.

 A consistent system may have one or infinite

solutions.

35
 Every system of linear equations has either no
solution, exactly one solution, or infinitely many
solutions.
 A general system of two linear equations:
a1 x  b1 y  c1 (a1 , b1 not both zero)
a2 x  b2 y  c2 (a2 , b2 not both zero)

(a) Two lines may be parallel

-> no solution

36
(b) Two lines may intersect at only one point
-> one solution

(c) Two lines may coincide

-> infinitely many solution
37
 A General System of Three Linear
Equations

No Solution of three variable systems.

A picture of three planes that have no
solution.There is no single point at
which all three planes intersect,
therefore this system has no solution.

Note : A General Linear

Equation in Three variables
represents a plane in space

38
 A General System of Three Linear
Equations
The other common example of systems of three variables equations that
have no solution is pictured below. In the case below, each plane
intersects the other two planes. However, there is no single point at
which all three planes meet. Therefore, the system of 3 variable
equations below has no solution.

39
 A General System of Three Linear
Equations

One Solution of three variable

systems
If the three planes intersect as
pictured then the three variable
system has 1 point in common, and
a single solution represented by the
black point .

40
Augmented Matrix
 The location of the +’s, the x’s, and the =‘s in the
linear system can be abbreviated by writing only
the rectangular array of numbers. This is called
the augmented matrix for the system.
 For the given system

a11 x1  a12 x2  ...  a1n xn  b1

a21 x1  a22 x2  ...  a2 n xn  b2
   
am1 x1  am 2 x2  ...  amn xn  bm
41
 The augmented matrix is

 a11 a12 ... a1n b1 

 a a ... a 
b2 

A  21 22 2n

    
 
 am1 am 2 ... amn bm 

42
Example – Augmented Matrix
 The augmented matrix for the system of equations
x1  x2  2 x3  9
2 x1  4 x2  3 x3  1
3x1  6 x2  5 x3  0
is

1 1 2 9 
 2 4 3 1 
 
 3 6 5 0 
43
Elementary Row Operations
 The basic method for solving a system of linear
equations is to replace the given system by a new
system that has the same solution set but which is
easier to solve.
 Since the rows of an augmented matrix correspond
to the equations in the associated system, the new
systems is generally obtained in a series of steps by
applying the following three types of operations to
eliminate unknowns systematically. These are called
elementary row operations.

44
1. Multiply an equation through by a nonzero
constant.
2. Interchange two equations.
3. Add a multiple of one equation to another.

45
Example -
Using Elementary row Operations (1/4)

x  y  2z  9 x  y  2z  9
add -3 times
2 x  4 y  3 z  1 add - 2 times 2 y  7 z  1 7 the first equation
the first equation
to the third
3x  6 y  5 z  0 to the second
   
3x  6 y  5 z  0
   

1 1 2 9  1 1 2 9 
2 4  3 1 add - 2 times 0 2 7 17 
  add -3 times
the first row   the first row
3 6  5 0 to the second  3 6 5 0  to the third
  
   

46
Example -
Using Elementary row Operations (2/4)

x  y  2z  9 x  y  2z  9
2 y  7 z  17 multiply the second y  72 z   172 add -3 times
1 the second equation
3 y  11z  27 equation by
  23 y  11z  0tothe third
 

1 1 2 9  1 1 2 9 
0 2  7  17  0 1  7  17 
  multiply the second  2 2  add -3 times
0 3  11  27  row by 1 0 3  11  27 the second row
to the third
 2 
   

47
Example -
Using Elementary row Operations (3/4)
x  y  2z  9 x  y  2z  9 Add -1 times the
Multiply the third secondequation
y  72 z   172    
equation by - 2
 y  72 z   172 
to thefirst
 12 z   32 z 3

1 1 2 9  Multily the third 1 1 2 9  Add -1 times the

0 1  7  172  0 1
 2
row by - 2
    72  172  second row
to the first
0 0  12 23 
0 0 1 3     
 

48
Example -
Using Elementary row Operations (4/4)
Add - 11
2
times
x  112 z  35
2
the third equation x 1
to the first and 72 times
y  72 z   172 the third equation y  2
to the second
z 3     
 z 3

Add - 11
2
times
the third row
1 0 112 35
2  to the first and 72 1 0 0 1 
0 1  7  
17 times the third row 0 1 0 2 
 2 2
to the second  
0 0 1 3     
 0 0 1 3

 The solution x=1,y=2,z=3 is now evident.

49
Echelon Forms
 A matrix with the following properties is in
reduced row-echelon form, (RREF).
1. If a row does not consist entirely of zeros, then the
first nonzero number in the row is a 1. We call this a
leading 1.
2. If there are any rows that consist entirely of zeros,
then they are grouped together at the bottom of the
matrix.
3. In any two successive rows that do not consist entirely
of zeros, the leading 1 in the lower row occurs farther
to the right than the leading 1 in the higher row.
4. Each column that contains a leading 1 has zeros
everywhere else.
50
 A matrix that has the first three properties (but
not necessarily 4) is said to be in row-echelon
form (REF).

51
Row-Echelon &
Reduced Row-Echelon form
 reduced row-echelon form:
0 1 2 0 1 
1 0 0 4  1 0 0  0
0 1 0 7  , 0 1 0  ,  0 0 1 3 0 0 
, 
    0 0 0 0 0  0 0 
0 0 1 1 0 0 1   
0 0 0 0 0
 row-echelon form:

1 4 3 7  1 1 0  0 1 2 6 0 
0 1 6 2  , 0 1 0  , 0 0 1 1 0 
     
0 0 1 5  0 0 0  0 0 0 0 1 
52
Concluding Remarks on Row-
echelon forms
 Row-echelon form of a matrix is not unique and
depends upon the sequence of elementary row
operations performed on the original matrix yet it
does not affect the solution of the linear system.

 Reduced row-echelon form of a matrix is unique.

 Row-echelon/reduced row-echelon form are well

suited for computer computations because of
their systematic and algorithmic nature.

53
Concluding Remarks on Row-
echelon forms
 Many algorithms modify the basic procedure to
reduce round-off errors, minimize the storage
space and accelerate the computations. (For
example, pivoting is performed to reduce round
off errors.)

54
Example (a)
Suppose that the augmented matrix for a system of
linear equations have been reduced by row operations
to the given reduced row-echelon form. Solve the
system.
1 0 0 5 
(a) 0 1 0  2
0 0 1 4 

Solution x  5
The corresponding system y  -2
of equations is : z 4
55
Example - (b)(1/2)
1 0 0 4  1
(b) 0 1 0 2 6 
0 0 1 3 2 
free variables
Solution
x1  4 x4  - 1
1. The corresponding
system of equations is : x2  2 x4  6
x3  3 x4  2
leading
variables

56
Example - (b)(2/2)
x1  - 1 - 4 x4 2. We see that the free variable can be
assigned an arbitrary value, say t, which
x2  6 - 2 x4
then determines values of the leading
x3  2 - 3 x4 variables.

x1  1  4t ,
3. There are infinitely many
solutions, and the general x2  6  2t ,
solution is given by the x3  2  3t ,
formulas
x4  t
57
Example (c) (1/2)
1 6 0 0 4  2
0 0 1 0 3 1 
(c) 
0 0 0 1 5 2
 
0 0 0 0 0 0 

Solution
1. The 4th row of zeros leads to x1  6 x2  4 x5  - 2
the equation places no
restrictions on the solutions x3  3 x5  1
(why?). Thus, we can omit x4  5 x5  2
this equation.

58
Example (c) (2/2)
Solution x1  - 2 - 6 x2 - 4 x5
2. Solving for the leading
variables in terms of the free
x3  1 - 3 x5
variables: x4  2 - 5 x5

3. The free variable can be x1  - 2 - 6 s - 4t ,

assigned an arbitrary
value,there are infinitely x2  s
many solutions, and the x3  1 - 3t
general solution is given by
the formulas. x4  2 - 5t ,
x5  t
59
Example (d)
1 0 0 0 
(d) 0 1 2 0
0 0 0 1

Solution
the last equation in the corresponding system of equation is
0 x1  0 x2  0 x3  1
Since this equation cannot be satisfied, there is no solution
to the system.

60
Elimination Methods (1/7)
 We shall give a step-by-step elimination procedure that can be
used to reduce any matrix to reduced row-echelon form.

0 0  2 0 7 12 
2 4  10 6 12 28
 
2 4  5 6  5  1

61
Elimination Methods (2/7)
 Step1. Locate the leftmost column that does not consist entirely
of zeros.
0 0  2 0 7 12 
2 4  10 6 12 28
 
2 4  5 6  5  1
Leftmost nonzero column

 Step2. Interchange the top row with another row, to bring a

nonzero entry to top of the column found in Step1.

2 4  10 6 12 28
0 0  2 0 7 12  The 1st and 2nd rows in the
  preceding matrix were
2 4  5 6  5  1 interchanged.

62
Elimination Methods (3/7)
 Step3. If the entry that is now at the top of the column found in Step1 is
a, multiply the first row by 1/a in order to introduce a pivot 1.

1 2  5 3 6 14 
0 0  2 0 7 12  The 1st row of the preceding
 
2 4  5 6  5  1 matrix was multiplied by 1/2.

 Step4. Add suitable multiples of the top row to the rows below so that
all entries below the pivot 1 become zeros.

1 2  5 3 6 14 
0 0  2 0 7  -2 times the 1st row of the
 12  preceding matrix was added to
0 0 5 0  17  29 the 3rd row.

63
Elimination Methods (4/7)
 Step5. Now cover the top row in the matrix and begin again with Step1
applied to the sub-matrix that remains. Continue in this way until the
entire matrix is in row-echelon form.

1 2  5 3 6 14 
0 0  2 0 7 12 
 
0 0  5 0  17  29 Leftmost nonzero
column in the submatrix

1 2  5 3 6 14 
0 0 1 0  7  6  The 1st row in the sub-matrix
 2  was multiplied by -1/2 to
0 0 5 0  17  29 introduce a pivot 1.

64
Elimination Methods (5/7)
 Step5 (cont.)
1 2  5 3 6 14  -5 times the 1st row of the sub-
0 0 1 0  7  6  matrix was added to the 2nd row
 2  of the sub-matrix to introduce a
0 0 0 0 12 1  zero below the pivot 1.

1 2  5 3 6 14  The top row in the sub-matrix was

0 0 1 0  7  6  covered, and we returned again
 2  Step1.
0 0 0 0 2 1 
1
Leftmost nonzero column in
the new sub-matrix
1 2  5 3 6 14 
0 0 1 0  7  6  The first (and only) row in the
 2  new sub-matrix was multiplied
0 0 0 0 1 2  by 2 to introduce a pivot 1.

 The entire matrix is now in row-echelon form. 65

Elimination Methods (6/7)
 Step6. Beginning with last nonzero row and working upward, add suitable
multiples of each row to the rows above to introduce zeros above the pivot
1’s.
1 2  5 3 6 14 7/2 times the 3rd row of the
0 0 1 0 0 1  preceding matrix was added to
 
the 2nd row.
0 0 0 0 1 2 

1 2  5 3 0 2 
0 0 1 0 0 1  -6 times the 3rd row was added
  to the 1st row.
0 0 0 0 1 2

1 2 0 3 0 7 
0 0 1 0 0 1  5 times the 2nd row was added
  to the 1st row.
0 0 0 0 1 2

 The last matrix is in reduced row-echelon form. 66

Elimination Methods (7/7)
 Step1~Step5: the above procedure produces a row-
echelon form and is called Gaussian elimination.
 Step1~Step6: the above procedure produces a
reduced row-echelon form and is called Gaussian-
Jordan elimination.
 Every matrix has a unique reduced row-echelon
form but a row-echelon form of a given matrix is not
unique.

67
Example -
Gauss-Jordan Elimination(1/4)
 Solve by Gauss-Jordan Elimination
x1  3 x2  2 x3  2x 5  0
2 x1  6 x2  5 x3  2 x4  4 x5  3x6  1
5 x3  10 x4  15 x6  5
2 x1  6 x2  8 x4  4 x5  18 x6  6

 Solution:
The augmented matrix for the system is
1 3 -2 0 2 0 0
2 6 -5 -2 4 -3 - 1

0 0 5 10 0 15 5
 
2 6 0 8 4 18 6 68
Example -
Gauss-Jordan Elimination(2/4)
 Adding -2 times the 1st row to the 2nd and 4th rows
gives
1 3 -2 0 2 0 0
0 0 -1 -2 0 -3 - 1

0 0 5 10 0 15 5
 
0 0 4 8 0 18 6

 Multiplying the 2nd row by -1 and then adding -5

times the new 2nd row to the 3rd row and -4 times
the new 2nd row to the 4th row gives
1 3 -2 0 2 0 0
0 0 1 2 0 3 1
 
0 0 0 0 0 0 0
  69

0 0 0 0 0 6 2
Example -
Gauss-Jordan Elimination(3/4)
 Interchanging the 3rd and 4th rows and then multiplying the 3rd
row of the resulting matrix by 1/6 gives the row-echelon form.
1 3 -2 0 2 0 0
0 0 1 2 0 3 1
 
0 0 0 0 0 1 1
3
 
0 0 0 0 0 0 0
 Adding -3 times the 3rd row to the 2nd row and then adding 2
times the 2nd row of the resulting matrix to the 1st row yields the
reduced row-echelon form.
1 3 0 4 2 0 0
0 0 1 2 0 0 0 
 
0 0 0 0 0 1 3  1

  70
 0 0 0 0 0 0 0 
Example -
Gauss-Jordan Elimination(4/4)
 The corresponding system of equations is
x1  3 x2  4 x4  2 x 5 0
x3  2 x4 0
x6  13
 Solving for the leading variables in terms of the free
variables
x1  3 x2  4 x4  2 x 5
x3  2 x4
x6  1
3

 We assign the free variables arbitrary values, and

the general solution is given by the formulas:
x1  3r  4 s  2t , x2  r , x3  2 s, x4  s, x5  t , x6  13 71
Back-Substitution
 It is sometimes preferable to solve a system of linear
equations by using Gaussian elimination to bring the
augmented matrix into row-echelon form without
continuing all the way to the reduced row-
echelon form.
 When this is done, the corresponding system of
equations can be solved by a technique called back-
substitution.

72
solved by Back-
substitution(1/3)
 From the computations in previous example, a row-
echelon form from the augmented matrix is
1 3 -2 0 2 0 0
0 0 1 2 0 3 1
 
0 0 0 0 0 1 1
3
 
0 0 0 0 0 0 0

 To solve the corresponding system of equations

x  3 x - 2x
1 2 3
 2x 5
0
x  2x
3 4
+3x  1
6

x 
6
1
3 73
Example – Previous Example solved
by Back-substitution(2/3)
 Step1. Solve the equations for the leading variables.

x1  3 x2  2 x3  2 x 5
x3  1  2 x4  3 x6
x6  1
3

 Step2. Beginning with the bottom equation and

working upward, successively substitute each equation
into all the equations above it.
 Substituting x6=1/3 into the 2nd equation

x1  3 x2  2 x3  2 x 5
x3  2 x4
x6  1
3 74
solved by Back-
substitution(3/3)
 Substituting x3=-2 x4 into the 1st equation
x1  3 x2  4 x3  2 x 5
x3  2 x4
x6  1
3

 Step3. Assign free variables, the general solution is

given by the formulas.
x1  3r  4 s  2t , x2  r , x3  2 s, x4  s,
x5  t , x6  1
3

75
Gaussian Elimination

One of the most popular techniques for solving

simultaneous linear equations of the form
AX  b

Consists of 2 steps
1. Forward Elimination of Unknowns.
2. Back Substitution

76
Forward Elimination

The goal of Forward Elimination is to transform the

coefficient matrix into an Upper Triangular Matrix

 25 5 1 25 5 1 
 64 8 1   0  4.8  1.56
   
144 12 1  0 0 0.7 

77
Forward Elimination

Linear Equations
A set of n equations and n unknowns

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1

a21 x1  a22 x2  a23 x3  ...  a2 n xn  b2
. .
. .
. .
an1 x1  an 2 x2  an 3 x3  ...  ann xn  bn
78
Forward Elimination

Transform to an Upper Triangular Matrix

Step 1: Eliminate x1 in 2nd equation using
equation 1 as the pivot equation
 Eqn1
   (a21 )
 a11 
Which will yield
a21 a21 a21
a21x1  a12 x2  ...  a1n xn  b1
a11 a11 a11
79
Forward Elimination

Zeroing out the coefficient of x1 in the 2nd equation.

Subtract this equation from 2nd equation
 a21   a21  a21
 a22  a12  x2  ...   a2 n  a1n  xn  b2  b1
 a11   a11  a11
Or Where
' ' ' a21
a x  ...  a x  b
22 2 2n n 2
'
a22  a22  a12
a11

' a21
a  a2 n 
2n a1n
a11 80
Forward Elimination

Repeat this procedure for the remaining equations to

reduce the set of equations as

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1

' '
a22 x2  a23 x3  ...  a2' n xn  b2'
' ' ' '
a32 x2  a33 x3  ...  a3n xn  b3
. . .
. . .
. . .

' ' ' '

a x  a x  ...  a x  b
n2 2 n3 3 nn n n
81
Forward Elimination

Step 2: Eliminate x2 in the 3rd equation.

Equivalent to eliminating x1 in the 2nd equation
using equation 2 as the pivot equation.

 Eqn 2 
Eqn3     (a32 )
 a22 

82
Forward Elimination

This procedure is repeated for the remaining

equations to reduce the set of equations as

a11 x1  a12 x2  a13 x3  ...  a1n xn  b1

' '
a22 x2  a23 x3  ...  a2' n xn  b2'
"
a33 x3  ...  a3" n xn  b3"
. .
. .
. .

an" 3 x3  ...  ann

"
xn  bn"

83
Forward Elimination

Continue this procedure by using the third

equation as the pivot equation and so on.
At the end of (n-1) Forward Elimination steps, the
system of equations will look like:
a11 x1  a12 x 2  a13 x3  ...  a1n x n  b1
' '
a22 x2  a23 x3  ...  a2' n xn  b2'
" " "
a x  ...  a x  b
33 3 n n 3
. .
. .
. .

n 1 n 1 
ann xn  bn
84
Forward Elimination

At the end of the Forward Elimination steps

a11 a12 a13  a1n   x1   b1 

 a '
a ' '    
 a2 n   x2   b2  ' 
 22 23

 a "
33
"
 a3n x3      b3 
"

    
       
 ( n 1)    
ann   xn  bn  (n-1 ) 

85
Back Substitution

The goal of Back Substitution is to solve each

of the equations using the upper triangular
matrix.
a11 a12 a13   x1   b1 
0 a22    
a23   x 2   b2  

 0 0 a33   x 3  b3 
Example of a system of 3 equations

86
Back Substitution

Start with the last equation because it has only

one unknown

( n 1)
b n
xn  ( n 1)
a nn

Solve the second from last equation (n-1)th

using xn solved for previously.
This solves for xn-1.
87
Back Substitution

Representing Back Substitution for all

equations by formula

n
i 1 i 1
bi   aij xj
j i 1 For i=n-1, n-2,….,1
xi  i 1
aii
and
( n 1)
b n
xn  ( n 1)
a nn
88
Example -
Gaussian Elimination(1/4)
 Solve x  y  2z  9 by Gaussian elimination
and 2 x  4 y  3z  1
3x  6 y  5 z  0 back-substitution.

 Solution
 We convert the augmented matrix
1 1 2 9
2 4  3 1
 

 3 6  5 0 
89
Example -
Gaussian Elimination(2/4)
 Solution
 to the row-echelon form
1 1 2 9 
0 1  72  172 

0 0 1 3 

 The system corresponding to this matrix is

x  y  2 z  9, y  72 z   172 , z  3

90
Example -
Gaussian Elimination(3/4)
 Solution
 Solving for the leading variables
x  9  y  2 z,
y   172  72 z ,
z 3
 Substituting the bottom equation into those
above
x  3  y,
y  2,
z 3 91
Example -
Gaussian Elimination(4/4)
 Solution
 Substituting the 2nd equation into the top

x  1, y  2, z  3

92
Example: Rocket Velocity
The upward velocity of a rocket is
given at three different times

Time, t Velocity, v
s m/s
5 106.8
8 177.2
12 279.2
The velocity data is approximated by a polynomial as:
vt   a1t 2  a 2 t  a3 , 5  t  12.
93
Find: The Velocity at t=6,7.5,9, and 11 seconds.
Example: Rocket Velocity
Assume
vt   a1t  a2t  a3 , 5  t  12.
2

Results in a matrix template of the form:

t12 t1 1  a1   v1 
 2   a   v 
t 2 t2 1  2  2
t32 t3 1 
 a3 
  v3 

 
Using date from the time / velocity table, the matrix
becomes:  25 5 1  a  106.8 
1
 64 8 1 a   177.2 
   2  
144 12 1  a3  279.2 94
Example: Rocket Velocity
Forward Elimination: Step 1

 Row1
Row2     (64) 
 25 
Yields

 25 5 1   a 1   106.81 
 0  4.8  1.56 a    96.21
   2  
144 12 1  a 3   279.2 

95
Example: Rocket Velocity
Forward Elimination: Step 1

 Row1
Row3     (144) 
 25 
Yields

25 5 1   a 1   106.8 
 0  4.8  1.56  a     96.21
   2  
 0  16.8  4.76 a 3   336.0

96
Example: Rocket Velocity
Forward Elimination: Step 2

 Row2 
Row3     (16.8) 
  4.8 
Yields

25 5 1   a 1   106.8 
 0  4.8  1.56 a    96.21
   2  
 0 0 0.7  a 3   0.735 
This is now ready for Back Substitution
97
Example: Rocket Velocity
Back Substitution: Solve for a3 using the third equation

0.7a3  0.735
0.735
a3 
0.7
a3  1.050

98
Example: Rocket Velocity
Back Substitution: Solve for a2 using the second equation

 4.8a2  1.56a3  96.21

 96.21  1.56a3
a2 
 4.8
-96.21  1.561.050
a2 
-4.8

a2  19.70
99
Example: Rocket Velocity
Back Substitution: Solve for a1 using the first equation

25a1  5a 2  a3  106.8
106.8  5a 2  a3
a1 
25
106.8  519.70   1.050
a1 
25

a1  0.2900

100
Example: Rocket Velocity
Solution:
 a1  0.2900
The solution vector is a    19.70 
 2  
 a3   1.050 
The polynomial that passes through the
three data points is then:
vt   a1t 2  a 2 t  a3
 0.2900 t 2  19.70 t  1.050, 5  t  12

101
Example: Rocket Velocity
Solution:
Substitute each value of t to find the corresponding
velocity
v6   0.29006   19.706   1.050
2

 129.69 m / s.
v7.5  0.29007.5  19.707.5  1.050
2

 165.1 m / s.
v9  0.29009   19.709   1.050
2

 201.8 m / s.
v11  0.290011  19.7011  1.050
2

 252.8 m / s.
102
Pitfalls of Gauss Elimination
Method
Two Potential Pitfalls
-Division by zero: May occur in the
forward elimination steps. Consider the
set of equations:
10 x2  7 x3  7
6 x1  2.099 x2  3 x3  3.901
5 x1  x2  5 x3  6
- Round-off error: Prone to round-off errors.
103
Pitfalls: Example
Consider the system of equations:
Use five significant figures with chopping
 10  7 0  x1   7 
 3 2.099 6   = 3.901
   x2   
 5  1 5  x   6 
 3
At the end of Forward Elimination
10 7 0   x1   7 
 0  0.001  x  =  6.001 
 6   2  
 0 0 15005  x3  15004
 
104
Pitfalls: Example

Back Substitution
10 7 0   x1   7  15004
 0  0.001      x3   0.99993
 6   x 2    6.001  15005
 0 0 15005  x3  15004
6.001  6 x3
x2   1.5
 0.001

7  7 x 2  0 x3
x1   0.3500
10

105
Pitfalls: Example
Compare the calculated values with the exact solution
 x1   0 
X  exact   
  x 2    1
 x3   1 

 x1    0.35 
X  calculated   x 2     1.5 
 x3  0.99993
106
Improvements

Increase the number of significant digits

Decreases round off error
Does not avoid division by zero

Gaussian Elimination with Partial Pivoting

Avoids division by zero
Reduces round off error

107
Homogeneous Systems of
Linear Equations
 A system of linear equations is said to be
homogeneous if all the constant terms
are zero.
a11 x1  a12 x2  .....  a1n xn  0
a21 x1  a22 x2  .....  a2 n xn  0
   
am1 x1  am 2 x2  .....  amn xn  0

108
Homogeneous Systems of
Linear Equations
 A homogeneous system of linear equations is
always consistent, since it has at least one
solution;
x1  0, x2  0,...xn =0
 The above solution is called trivial solution of the
homogeneous system of linear equation.
 There may be solutions other than the trivial one
and they are called non-trivial solutions of the
homogeneous system of linear equations.

109
Homogeneous Systems of
Linear Equations
 For a homogeneous system, Exactly one
of the following is true:
1. The system has only the trivial solution.
2. The system has infinitely many solutions
in addition to the trivial solution.
 A homogeneous system of linear
equations with more unknown than the
equations has infinitely many solutions.

110
Example - Homogeneous
System of Linear Equations
 Solve the following homogeneous system of
linear equations by Gauss-Jordan elimination.

2 x1  2 x2  x 3  x5 =0
 x1  2 x2  2 x 3  3x 4  x5 =0
x1  x2  2 x 3  x5 =0
x 3  x 4  x5 =0
111
Example
 The Augmented Matrix for the system is

 2 2 1 0 1 0 
 1 1 2 3 1 0 
 
 1 1 2 0 1 0 
 
 0 0 1 1 1 0
 Reducing to Reduced Row Echelon form

1 1 0 0 1 0
0 0 1 0 1 0 

0 0 0 1 0 0
 
0 0 0 0 0 0
112
 Example
 The corresponding system of equations is

x1  x2  x5 = 0
x3  x5 = 0
x4 =0
 Solving for leading variables

x1   x2  x5
x 3   x5 x 2 , x5  Free variables 
x 4 =0
113
Example
 Elementary row operations cannot alter the
last column of zeros, hence reduced system
must also be homogeneous.
 For m equations in n variable with m<n, there
exist infinite no. of solutions.
 If no. of non zero rows is r in the reduced row
echelon form then the no. of free variables
are n-r.

114
Matrix Multiplication
 In general, matrix multiplication is not
commutative AB  BA
 It can happen that:
 AB is defined but BA is not defined.
 Both AB & BA are defined but have different
sizes.
 Both AB & BA are defined and have equal
sizes but even then they are not equal.

115
Example
 1 0  1 2 
A  , B 
 2 3 3 0 
 1 2   3 6
AB    , BA   
11 4   3 0 
 Thus AB  BA.

116
Matrix Multiplication
 If order of A, B and C AB
are such that
BC
and are possible, then matrix
multiplication holds the associative law
A( BC )  ( AB )C
 For square matrices A, B and C of same
sizes, matrix multiplication always follows the
associative law.

117
Null or Zero Matrix
 A matrix with all entries equal to zero is called
Null or Zero matrix.

0  0 0 
 0  ,  0 0  , 0 0 0 
   
 Generally, a zero matrix of order mn is
denoted as Om  n .

118
Properties of Zero Matrix
 Zero matrix (of appropriate size) is the
additive identity in matrix arithmetic i.e.
AO  O  A  A
A A  O
 Law of cancellation is not valid for non-zero
matrices.

119
Example
0 1  1 1  2 5 3 7 
A  ,B    ,C    ,D   
0 2 3 4  3 4   0 0 
3 4
 AB  AC    though A  0, it is incorrect
6 8
to cancel A from both sides of equation AB  AC
to get B  C
and
AD  0 does not mean A  0 or D  0
120
Inverse of a Matrix
 If A is an (n x n) square matrix and there is
a matrix X with the property that

AX  XA  I
 X is defined to be the inverse of A and is
denoted by A-1

1 1
AA I A AI

121
 1
Properties of A
 Inverse of a matrix is unique.
 A product of invertible matrices is always
invertible, and inverse of product is the
product of the inverses in the reverse
order. 1 1 1
( AB )  B A
 If A is invertible then A1 is also invertible
and 1 1
(A )  A

122
Row Equivalence
 Two matrices A and B are said to be row
equivalent if and only if they can be obtained
from each other by a sequence of finite no. of
elementary operations.
A............  B
( sequence of elementray operations )
B............  A
( sequence of inverse elementray
operations )
123
Method of Finding Inverse of a
Matrix
 To find the inverse of an invertible matrix
A, we must find a sequence of elementary
operations that turn A to an Identity matrix
and then the same sequence must be
applied on Identity matrix to get inverse of
A.

124
Example (1/4)
 Find the inverse of A

1 2 3 

A   2 5 3 
1 0 8 
1 2 3 1 0 0 Augmenting with
Identity matrix
2 5 3 0 1 0 

1 0 8 0 0 1 
125
Example (2/4)

1 2 3 1 0 0
Adding –2 times 1st row
0 1 3  2 1 0  To 2nd and –1 times the
 first row to 3rd row

0 2 5  1 0 1 
1 2 3 1 0 0
0 1 3  2 1 0  Adding 2 times the
 Sencond row to the third

0 0 1  5 2 1 
126
Example (3/4)

1 2 3 1 0 0  Multiplying third row

0 1 3 2 1 0  By -1


0 0 1 5 2 1
1 2 0  14 6 3  Adding 3 times the
0 1 0 13 5 3  Third row to the second
And –3 times the to
 the first
0 0 1 5 2 1
127
Example (4/4)

1 0 0  40 16 9  Adding –2 times
0 1 0 13 5 3 The row to the
first

0 0 1 5 2 1
 40 16 9 
1  
Hence A   13 5 3 
 5 2 1 
 
128
Determinant Function
 Determinant Function is a real valued
function of matrix variable in a sense that it
associates a real number ƒ(X) with a matrix
X.
 Let A be square matrix. The Determinant
Function is denoted by det. The det(A) is
called the determinant of A.

129
Determinants
 A determinant of second order is defined as
a11 a12
det( A)   a11a 22  a12 a 21
a 21 a 22
 A third order determinant can be defined
a11 a12 a13
det( A)  a 21 a 22 a 23
a 31 a 32 a 33
a 22 a 23 a12 a13 a12 a13
 a11  a 21  a 31
a 32 a 33 a 32 a 33 a 22 a 23 130
 Determinants
 The signs on the right are + - +.Each of the
three terms in last line of the last slide is an
entry in the first column of det(A) times its
“minor,” that is, the second-order
determinant obtained by deleting from det(A)
the row and column of that entry .
 If we write the minors,

a11a23 a32  a11a22 a33  a13a21a32  a12 a21a33  a12 a23a31  a13a22 a31

131
 Determinant of Order n
 Determinant of order n
a11 a12  a1n
a 21 a 22  a2n
det( A) 
  

an1 an 2  ann
 For n=1 det(A)=a11 and
For n>1
det( A)  a j1C j1  a j 2C j 2    a jn C jn j  1, 2 , or n
or
det( A)  a1k C1k  a2 k C2 k    ank Cnk k  1, 2, , or n
132
Determinant of Order n
 Where Cofactor
Cjk=(-1)j+k Mjk
and Mjk is a determinant of order n-1,the determinant of the
submatrix of A obtained from A by deleting the row and
column of the entry ajk (the jth row and the kth column).
 In this way, det is defined in terms of n determinants of order
n-1, each of which is, in turn, defined in terms of n-1
determinants of order n-2 and so on at last second order
determinant occur.
 From the definition it follows that we may expand by any row
or column ,that is to choose the entries in any row or column.

133
 Example
1 3 0
det( A)  2 6 4
1 0 2
6 4 2 4 2 6
det( A)  1 3 0
0 2 1 2 1 0
det( A)  1(12  0)  3(4  4)  0(0  6)
134
Properties of Determinants
 Interchange of two rows or columns multiplies the
value of the determinant by-1.
 Addition of a multiple of a row to another row or
column to another column does not alter the value of
the determinant.
 Multiplication of a row or column by c multiplies the
value of the determinant by c.
 Transposition leaves the value of a determinant
unaltered.
 A zero row or column renders the value of the
determinant zero.
135
Properties of Determinants
 Proportional rows or columns also renders
the value of the determinant to zero.
 The determinants of the upper
triangular ,lower triangular or diagonal matrix
is the product of entries on the main diagonal
that is
det( A)  a11a22  ann

136
Determinants by Row Reduction
 The determinant of a matrix can be obtained
by reducing the matrix to row-echelon form.
This method suffers less computational
complexity.
 Example: Evaluate det(A) using row-reduction

0 1 5
det( A)  3 6 9
2 6 1
137
Determinants by Row Reduction

Interchanging First and Second Row

3 6 9
det( A)   0 1 5
2 6 1

Taking out the common factor of 3

1 2 3
det( A)  3 0 1 5
2 6 1

138
Determinants by Row Reduction

1 2 3
det( A)  3 0 1 5 R 3  R 3  2 R1
0 10 5

1 2 3
det( A)  3 0 1 5 R 3  R 3  10 R 2
0 0 55
139
Determinants by Row Reduction

1 2 3
det( A)  (3)(55) 0 1 5 R3   R3 / 55
0 0 1
det( A)  ( 3)( 55)(1)(1)(1)

det( A)  165

140
Example 2
 Evaluate determinants reducing to triangular
form
2 0 4 6
4 5 9 12
det( A) 
0 2 6 1
3 8 3 10
2 0 4 6
0 5 9 12 R 2  R 2  2 R1
det( A) 
0 0 2.4 3.8 R 4  R 4  1.5 R1
0 0 11.4 29.2 141
Example 2 (2/3)
2 0 4 6
R 3  R 3  0.4 R 2
0 5 9 12
det( A) 
0 0 2.4 3.8 R 4  R 4  1.6 R 2
0 0 0 47.25

2 0 4 6
0 5 9 12
R 4  R 4  4.75 R 3
det( A) 
0 0 2.4 3.8
0 0 0 47.25
142
Example 2 (3/3)
 So following the statement of theorem

det( A)  (2)(5)(2.4)(47.25)
det( A)  1134

143
Linear Combination
 A vector W is called the linear combination of
the vectors a 1, a 2 , ....... an if it can be
expressed in the form

W  k 1a1  k 2 a 2  knan
where k 1, k 2 ,  kn are scalars and
a1, a 2  an have same number of
components.

144
Example 1 (1/3)
 Show that 9 

w  2 
7 
is a linear combination of
1  6 
  
a1   2  a 2   4  
 1  2 
145
Example 1 (2/3)
 By definition of Linear Combination

w  k 1a 1  k 2 a 2
9  1  6 
 2  k 1  2   k 2  4
     
 7   1  2 
146
Example 1 (3/3)
 Equating corresponding components yield

k 1  6k 2  9
2k 1  4k 2  2
 k 1  2k 2  7
which has the solution k1= -3, k2= 2 so that

w  3a1  2a 2
147
Example 2 (1/5)
 Check whether 4 

w   1
8 
is a linear combination of vectors.
1  6 

a1   2   
a 2   4 
 1  2 
148
Example 2 (2/5)
 So by definition

 4  1  6 
 1  k 1  2   k 2  4
     
8   1  2
yields
k 1  6k 2  4
2 k 1  4 k 2  1
 k 1  2k 2  8 149
Example 2 (3/5)
Augmented Matrix is
1 6 4
 2 4  1
 
 1 2 8 
1 6 4
0  8  9  R2  2 R1
  R3  R1
0 8 12 

150
Example 2 (4/5)
1 6 4 
0 1 9 / 8 1
  R2
8
0 8 12 
1 6 4 
0 1 9 / 8 R  8R
  3 2
0 0 3 

151
Example 2 (5/5)
 The System of equations is inconsistent ,so
that no such scalars k1,k2 can be found.
Consequently w is not a Linear combination
of a1 and a 2 .

152
Linear Independence
 Let A be the non empty set of vectors.

A  a1, a 2, a 3...., an 
 Consider the equation in which W is equal to zero.

k 1a1  k 2 a 2   knan  0
has a trivial solution
k 1  0, k 2  0,  kn  0
If this is the only solution ,then A is called the Linear
Independent set. If there are other solutions then A
is called Linear Dependent set.
153
Example(1/3)
 Check the Linear Independence of following vectors.

1  5  3 
 
a1   2  
a 2  6   
a3   2 
3   1 1 

For the vectors to be Linear Independent

k 1a 1  k 2 a 2  k 3 a 3  0
154
Example(2/3)

1  5  3   0 
      
k 1  2   k 2  6   k 3  2    0 
3   1 1   0 

k 1  5k 2  3k 3  0
2k 1  6k 2  2k 3  0
3k 1  k 2  k 3  0 155
Example(3/3)
 The equations also have a non-trivial solution
k 1  (1/ 2) k 3
k 2  (1/ 2) k 3
 The above set is Linearly dependent set.
 The set listed below is Linearly independent
set with only one trivial solution.
1  0 0
a1  0 a 2 
1  a 3   0 
     

0 
0 
 
1  156
Theorems
1) a) A set A with two or more vectors is Linearly
dependent, if and only if at least one of the
vectors in A is expressible as a linear
combination of the other vectors in A.
b) A set with exactly two vectors is Linearly
independent if and only if neither vector is a
scalar multiple of the other.
2) A set that contains the zero vector is Linearly
dependent because x 0  0
1
has many nontrivial solutions

157
Theorems
 3) A set A with two or more vectors is
Linearly independent, if and only if no vector
in A is expressible as a Linear Combination
of the other vectors in A.

158
Theorems
 
4) Let A  a1, a 2, a 3...., an be a set of
vectors in Rq. If n>q then A is linearly
dependent.
For example n=3, q=2

1  5  3 
a1    a2    a3   
 2  6  2
159
Theorems
 The solution in terms of k3 as a free variable is
which has a non-trivial solution and are
Linearly dependent.

k 1  (1/ 2) k 3
k 2  (1/ 2) k 3

160
Vector-Equation of Homogeneous
System
 For a homogeneous system of Linear
equation
Ax  0
 a11 a12 a13  a1n 
 
 a21 a22 a23 a2 n 
where A 
    

a 
 m1 am 2 am 3  amn 
 x1 
x 
and x   2 
 
 
 xn 
  161
Vector-Equation of Homogeneous
System
 a11   a12   a1n 
a  a  a 
 21  x   22  x    2 n  x  0 and
  1   2   n
     
 am1   am 2   amn 

a1 x1  a2 x2   an xn  0
 If unique solution x1, x 2, x 3...., xnexist then
vectors a1, a 2, a 3...., an will always be Linearly
independent.
162
Vector Spaces
 If n is a positive integer ,then an ordered n-
tuple is a sequence of n real numbers
a1, a 2, a 3...., an . The set of all ordered
n-tuples is called n-space and is denoted by
Rn.
 A vector space is a nonempty set V=Rn of
vectors such that with any two vectors a and
b in V all their linear combinations αa+βb (α,
β any real numbers) are elements of V. In
addition these vectors in V satisfy the
following axioms.
163
Vector Spaces
 If a , b and c are vectors in V, then a+b is in
V
 a+b=b+a
 a+(b+c)=(a+b)+c
 There is a vector in V, called a zero vector for
V ,such that 0+a=a+0 for all a in V.
 For each a in V ,there is a vector –a in V,
called the negative of a such that a+(-a)=0.
 If k is any scalar and a is any object in V,
then ka is in V.
164
Vector Spaces
 k(a+b)=ka+kb
 (k+l)a=ka+ la
 k(la)=(kl)a

165
Span
 If a1, a 2, a 3...., an are vectors in vector
space V and if every vector in V is
expressible as a linear combination of these
vectors ,then a1, a 2, a 3...., an span V.
Example
The vectors i=(1,0,0) ,j=(0,1,0) and k=(0,0,1)
spans R3. Every vector z=(a ,b ,c) in R3 can
be written as
(a, b, c)=a i+b j+c k
166
Example 1(1/5)
 Determine whether
1  1   2

a1  1     
a 2  0  and a 3  1  
 2  1  3 

span the vector space R3.

 Solution: Let an arbitrary vector b=(b1, b2, b3)
can be expressed as a linear combination of a 1
, a2 and a3.
167
Example 1(2/5)

b  k1a1  k2a 2  k3a 3

b1  1  1   2
b   k 1   k 0   k 3 1 
 2 1   2    
b3   2  1  3 
168
Example 1(3/5)
 Since they can only
k1  k2  2k3  b1
span the vector space
if the system of linear k1  k3  b2
equations is consistent
for all values of b1,b2
2 k 1  k 2  3k 3  b 3

and b3.
 The coefficient 1 1 2
matrix for the 
A  1 0 1 
above system of
linear equations is  2 1 3 
169
Example 1(4/5)
1 1 2 
det( A)  
1 0 1 
 R3  R3  2 R1

0 1 1

1 1 2 
det( A)  
0 1 1
 R2  R2  R1

0 1 1

1 1 2 
det( A)  
0 1 1
 R3  R3  R2

0 0 0 

170
Example 1(5/5)
 Since the coefficient matrix contains a zero row
it means the determinant of the above matrix
must be equal to zero.
1 1 2 
det( A)  
0 1 1


0 0 0 

det( A)  0

 a1, a2 and a3 doesn’t span the vector space R3.

171
Vector-Equation
 For a non-homogeneous system of Linear
equation
Ax  b
 a11 a12 a13  a1n 
 
 a21 a22 a23 a2 n 
where A 
    

a 

 m1 am 2 am 3  amn 
 x1  b1 
x  b 
and x   2  b   2
   
   
 xn 
  bn 
  172
Vector Equation
 a11   a12   a1n  b1 
a  a  a  b 
 21  x   22  x    2 n  x   2 
  1   2   n  
       
 am1   am 2   amn  bn 

 It means that solving a linear system with

augmented matrix is equivalent to writing b as
a linear combination of the column vectors of
A. x1, x 2, x 3...., xare
n the weights in the
linear combination.
173
Example 2 (1/2)
 It also means that the b is in the space
spanned by the column vectors of A.
 Example : If
 5  3 5 
b   3  and A  1 1 
 6   2 8
is b in the plane R3 spanned by the column
vectors of A.

174
Example 2 (2/2)
3  5   5 
    
x1 1   x2 1    3  
 2   8  6 
which has the solution
x1  5 x2  2

 It means that the b is a vector spanned by

column vectors of A.

175
Basis for a Vector Space
 If V is any vector space and A  a1, a 2, a 3...., an 
is a set of vectors in V, then A is called the
basis for V if the following two conditions
hold
1. A is a linearly Independent.
2. A spans V

176
Basis for vector Space
 If A  a1, a 2, a 3...., an  is a basis for a
vector space V, then every vector a in V can
be expressed in the form a  k1a1  k2a 2    kna n
in exactly one way.
 Example: Show that the set below is the basis
for R3.
1  2 3 
a1   2  a 2  9  and a3  3 
1   0   4 
177
Example 1 (1/3)
 For the basis a1,a2,a3 has to be Linearly
Independent and must span the vector space
 Let b be the vector that is the linear
combination of a1,a2,a3.
b  k1a1  k2a 2  k3a3
b1  1  2 3 
b   k  2   k 9   k 3 
 2 1   2   3  

b3  1   0   4 
178
Example 1 (2/3)
k1  2k 2  3k3  b1
2k1  9k 2  3k3  b2
2k1  4k3  b3
 We can simultaneously prove that a 1,a2,a3
span vector and are Linearly independent. If
the coefficient matrix below is invertible.
1 2 3
A 
2 9 3


1 0 4
 179
Example 1 (3/3)
 Determinant of the coefficient matrix is
calculated
det(A) = -1
So the vectors a1,a2,a3 form basis for R3.

180
Dimensionality of Vector Space
 A non-zero vector space V is called finite-
dimensional if it contains a finite set of vectors
a , a , a ...., a
1 2 3 n that form a basis. If no such
vectors exists, V is called the infinite dimensional
space.
 In a non-zero vector space V the maximum number
of linearly independent vectors in V is called the
dimensionality of vector space V.
 Zero vector space is regarded as the finite
dimensional space even though it has no linear
independent sets and no basis.
181
Rank of a Matrix
 The dimension of the row vectors or
column vectors is called the Rank of the
matrix and is denoted by Rank (A).
Or
 Rank of a matrix is number of leading
entries or pivot positions in Echelon form.
 Let A    a jk 
 be an m×n matrix and
let rank=R(A) has linear independent set of R
row vectors or column vectors.
182
Example 1(1/4)
 Find the rank of the matrix
 1 2 0 4 5 3
3 7 2 0 1 4 
A
2 5 2 4 6 1 
 
4 9 2 4 4 7 
 Solution:
 1 2 0 4 5 3 
0 1 2 12 16 5
A  
0 1 2 12 16 5
 
0 1 2 12 16 5
R2  R2  3R1
R4  R4  4 R1
R3  R3  2 R2 183
Example 1(2/4)
 1 2 0 4 5 3
0 1 2 12 16 5 
A 
0 0 0 0 0 0 
 
0 0 0 0 0 0 
R4  R4  R2
R3  R3  R2
1 2 0 4 5 3 
0 1 2 12 16 5 
A 
0 0 0 0 0 0 
 
0 0 0 0 0 0 
R2   R2
R1   R1 184
 Example 1 (3/4)
1 0 4 28 37 13 
0 1 2 12 16 5
A
0 0 0 0 0 0 
 
0 0 0 0 0 0 
R1  R1  2 R2

 So by using definition given above the rank of the

given matrix is Rank (A) = 2
185
Example 1 (4/4)
 The corresponding system of Linear
Equations.

x1  4 x3  28 x4  37 x5  13x6
x2  2 x3  12 x4  16 x5  5 x6

186
Rank of a Matrix
 Rank (A)=0 if A=0.
 A and AT has the same Rank.

187