MA 106 - Week 1: Linear Algebra

Neela Nataraj

Department of Mathematics,
Indian Institute of Technology Bombay,
Powai, Mumbai 76
neela@math.iitb.ac.in

January 7, 2019
Basic Information

Aim of the course

This core course aims at an introduction to the fundamental
concepts of linear algebra, abstract vector spaces and linear
transformations.
Course contents
See http://www.math.iitb.ac.in/assets/pdf/btech.pdf
Main Text:
E. Kreyszig, Advanced Engineering Mathematics, 8th ed.
Lectures and Tutorials
There will be seven weeks of instruction with 3 hours of lectures
and 1 hour of tutorial every week. The lectures will be uploaded in
the moodle page of the course.
Basic Information (contd..)

Grading Policy
There will be one common quiz and one end-semester exam (which
will coincide with the mid-semester exam of other courses). The
dates of the exams will be announced in the class and the moodle
page later.

The allotted 50 marks will be split as follows:

Common quiz : 15 marks

Endsem exam : 30 marks
TA evaluation : 05 marks

Attendance Policy
Attendance in lectures and tutorials is COMPULSORY.
Students who do not meet 80% attendance will be awarded
the DX grade.
Linear Algebra

Ï One of the basic areas in Mathematics, having at least as

great an impact as Calculus.
Ï Provides a vital arena where the interaction of Mathematics
and machine computation is seen.
Many of the problems studied in Linear Algebra are amenable
to systematic and even algorithmic solutions, and this makes
them implementable on computers.
Ï Many geometric topics are studied making use of concepts
from Linear Algebra.
Ï Applications to Physics, Engineering, Probability & Statistics,
Economics and Biology.
Outline of Week 1

Ï Matrices and properties (Self-study)

Ï Linear transformations and matrices
Ï Linear equations and Gauss’ elimination
Ï Row echelon forms
Ï Reduced REF
Matrix

A matrix is a rectangular array of numbers (or functions) enclosed

in brackets with the numbers (or functions) called as entries or
elements of the matrix.
Matrices are denoted by capital letters A, B, C , · · · .
For integers m, n ≥ 1, an m × n matrix is an array given by
 
a11 a12 ... a1n

 a21 a22 ... a2n 

 . .. .. 
 . 
 . . ... . 
am1 am2 . . . amn

Denoting the matrix above as A = (aij )1≤i ≤m, 1≤j ≤n , (aij ), i = 1, . . . , m

correspond to the m rows of the matrix and and (aij ), j = 1, . . . , n
correspond to the n columns of the matrix.
aij is called as the ij-th entry (or component) of the matrix.
Row and Column vectors

A matrix consisting of a single row is called a row vector.

A row vector (x1 , . . . , xn ) is a 1 × n matrix.
A matrix consistingof a single column is called a column vector.
x1
 .. 
A column vector  .  is an n × 1 matrix.
xn
Row and column vectors are denoted using lower case letters such
as a, b, and so on.
A square matrix is a matrix with the same number of rows and
columns.
For n ≥ 1, an n × n matrix is called a square matrix of order n.
A matrix (aij ) is called the zero matrix if all its entries are zeroes
and is, denoted by O.
The size of the zero matrix should be clear from the context.
Basic Properties

(Addition of Matrices)
If A = (aij )1≤i ≤m, 1≤j ≤n and B = (bij )1≤i ≤m, 1≤j ≤n are two m × n
matrices, their sum, written as A + B, is obtained by adding the
corresponding entries.
That is, A + B = (aij + bij )1≤i ≤m,1≤j ≤n .
For any matrix A, O + A = A + O = A.
(Scalar multiplication)
The product of an m × n matrix A = (aij )1≤i ≤m, 1≤j ≤n and any scalar
c, written as cA, is the m × n matrix obtained by multiplying each
entry of A by c.
That is, c(aij )1≤i ≤m, 1≤j ≤n := (caij )1≤i ≤m, 1≤j ≤n .
Write (−1)A = −A. Then A + (−A) = O. The matrix −A is called
the additive inverse of A.
(−k)A is written as −kA, A + (−B) is written as A − B and is called
the difference of A and B.
Matrix Addition Laws

If A, B and C are matrices of the same size m × n, 0 is the matrix

of size m × n with all entries as zeroes, then
1. A + B = B + A (Commutative Law)
2. (A + B) + C = A + (B + C ) (Associative Law)
3. A + 0 = A (Existence of Identity)
4. A + −A = 0 (Existence of additive inverse)
Scalar Multiplication Laws

If A and B are matrices of the same size m × n, c , k are scalars,

then
1. c(A + B) = cA + cB
2. (c + k)A = cA + kA
3. c(kA) = (ck)A = ckA
4. 1 · A = A
Multiplication of matrices

Let A = (aij ) be an m × n matrix and B = (bjk ) be an n × s matrix.

Define the product AB to be the m × s matrix (cik ), where
X
n
cik = aij bjk .
j =1

If A1 , . . . , Am are row vectors of A and B 1 , . . . , B s are column vectors

of B, then ik-th coordinate of the matrix AB is the product Ai B k .
 B : p rows q columns
 
 b 11 ... b1 j ... b 1q 
 
 
 
 
 .. .. .. .. 
 .. 
 . . . . . 
 
 
 
 b k1 ... bk j ... b kq 
 

1j
 

b
 

×
 

i1
 .. .. .. .. .. 

a
 

+
. . . . .

..
 

.+
j  
bk  
×  
k  b p1 ... bp j ... b pq 
ai

+
..
.+
j
bp
×
p
ai
   

 a 11 ... a 1k ... a 1p 


 c 11 ... c1 j ... c 1q 

   
   
   
 .. .. .. .. ..   .. .. .. .. .. 
 . . . . .   . . . . . 
   
   
   
 ai 1 ... ai k ... ai p   ci 1 ... ci j ... ci q 
   
   
   
   
 .. .. .. ..   .. .. .. .. 
 ..   .. 
 . . . . .   . . . . . 
   
   
   
 a n1 ... a nk ... a np   c n1 ... c nk ... c nq 

A : n rows p columns C = A × B : n rows q columns

Illustrations

Example  
µ ¶ 1 6 0 2
2 1 −1
Let A = 
and B = 2 −1 1 −2 . Then
0 3 1 2×3 2 0 −1 1 3×4
µ ¶
2 11 2 1
AB = .
8 −3 2 −5 2×4

Example
Consider a system of two linear equations in three unknowns
3x − 2y + 3z = 1; − x + 7y − 4z = −5.
 
µ ¶ µ ¶ x
3 −2 3 1
Let A = ; B= ; and X = y . Then the system

−1 7 −4 −5
z
of two equations can be written in the form AX = B .
Matrix Multiplication - 3 ways

Example  
µ ¶ 2 −1
2 −2 4
Let A = 
and B = 4 −2.
1 3 5
6 3
µ ¶
20 14
Then, AB = .
44 8
(i) Each entry of AB is the product of a row of A with a column of
B.
ij th entry of AB = the product of i th row of A & j th column of B.
(ii) Each column of AB is the product of a matrix and a column.
Column j of AB is A times column j of B.
(iii) Each row of AB is the product of a row and a matrix.
Row i of AB is row i of A times B.
Properties of Matrix Multiplication

Ï Let A, B , C be matrices; B , C of same size, such that AB is

defined. Then
A(B + C ) = AB + AC . (Distributive Law)
(kA)B = k(AB) = A(kB), where k is a scalar.
Ï Let A, B , C be matrices such that AB and BC is defined.
Then, A(BC ) = (AB)C . (Associative Law)

Ï AB 6= BA (Commutative law doesn’t hold true in general)

Ï AB = 0 doesn’t necessarily imply that A = 0 or B = 0 or BA = 0.
Ï AC = AD doesn’t necessarily imply that C = D.
Transpose of a matrix

Let A = (aij )1≤i ≤m, 1≤j ≤n be a m × n matrix. The n × m matrix

(bji )1≤j ≤n, 1≤i ≤m , where bji = aij is called the transpose of A and is
denoted by AT .
Taking transpose of a matrix amounts to changing rows into
columns and vice versa.
Example  
µ ¶ 1 −3
1 2 −2
Let A = . Then AT =  2 0 .
−3 0 1
−2 1
Properties of transpose

1. (A + B)T = AT + B T (A and B are matrices of the same size).

2. (AT )T = A
3. (kA)T = kAT (k is a scalar).
4. (AB)T = B T AT (A and B are compatible for multiplication).
If A = AT (resp. A = −AT ), then A is called a symmetric (resp.
skew-symmetric) matrix.
Note: A symmetric matrix is always a square matrix.
Special Matrices

(Upper triangular Matrices)

Upper triangular matrices are square matrices that can have
non-zero entries only on and above the main diagonal, whereas any
entry below the main diagonal must be zero.

Example  
  4 2 2 0
µ ¶ 1 4 2 0
1 3  3 −5 1 

A= , B = 0 3 2 , C =   are examples of
0 2 0 0 0 −6
0 0 6
0 0 0 5
upper triangular matrices.
Special Matrices (Contd..)

(Lower Triangular Matrices)

These are square matrices that can have non-zero entries only on
and below the main diagonal.

Example  
  3 0 0 0
µ ¶ 2 0 0 9 −3
5 0  0 0

A= , B = 8 −1 0, C =   are examples of
2 3 1 0 2 0
7 6 8
1 9 3 6
lower triangular matrices.
Special Matrices (Contd..)

(Diagonal Matrices)
These are square matrices than can have non-zero entries only on
the main diagonal. Any entry below or above the main diagonal
must be zero.

Example
 
2 0 0
A = 0 −3 0 is an example of a diagonal matrix.
0 0 2
Special Matrices (Contd..)

(Scalar Matrices)
Scalar matrices are diagonal matrices with diagonal entries, all
equal, say c.

Example
 
c 0 0
0 c 0 is an example of a scalar matrix.
0 0 c
It is called as scalar matrix because multiplication of any square
matrix A by a scalar matrix S of the same size has the same effect
as multiplication by a scalar.
That is, AS = SA = cA .
Special Matrices (Contd..)

(Identity Matrix (Unit Matrix))

Identity matrices are scalar matrices whose entries in the main
diagonal are all equal to 1.

Example
µ ¶
1 0
I2 = is the identity matrix of size 2 × 2.
0 1
 
1 0 0

I3 = 0 1 0 is the identity matrix of size 3 × 3.
0 0 1
Inverse of a Matrix

Let A be a square matrix of size n × n. The inverse of A (if it

exists), is a matrix B (of the same size) such that AB = BA = In .
Remark : Inverse of a square matrix need not exist always.
Theorem
If the inverse of a matrix exists, it is unique.
Proof : If possible, let B and C be inverses of A.
Then, AB = BA = I and AC = CA = I . Hence,
B = BI = B(AC ) = (BA)C = IC = C . ä
• The inverse of A (if exists) is denoted by A−1 .

Exercise : The transpose of an inverse is the inverse of the

transpose, i.e. (A−1 )T = (AT )−1 .
Example: Dot product as a matrix product
   
v1 w1
 ..   .. 
Let v =  .  and w =  .  be column vectors of the same size n.
vn wn
Their dot product (or inner product or scalar product) is defined as
X
n
v·w = vj wj .
j =1

It is interesting to observe that v · w = vT w as a 1 × 1 matrix, which

being symmetric also equals wT v = w · v.
Question: What about vwT ? Is it defined? Is it also the dot
product?

[1.0]
Matrix vector products- transformations

Ï Let A be an m × n matrix and x be an n × 1 vector in Rn .

Ï When A multiplies x, it transforms the vector to a new vector
Ax in Rm .
Ï This happens at every point x of the n-dimensional space Rn .
Ï The concept generalizes the common notion of a function as a
rule that transforms a real number into another.
Examples of transformations

Matrix Geometric effect of transformation on x ∈ R2

µ ¶
c 0
A= stretches/shrinks a vector x ∈ R2 by c
0 c
µ ¶
cos θ − sin θ
Aθ = rotates (turns) a vector x ∈ R2
sin θ cos θ
through θ
µ ¶
0 −1
A90◦ = rotates(turns) a vector x ∈ R2 through 90◦
1 0
µ ¶
0 1
A= reflection matrix transforms a vector x
1 0
into its image on the opposite side
of a mirror : here y = x
µ ¶
1 0
A= projection matrix takes the whole space onto
0 0
a lower dimensional space, here to x-axis.
Rotation Matrix µ ¶
cos θ − sin θ
The matrix Aθ = represents the rotation of R2
sin θ cos θ
around the origin by an angle θ.
Why?
µ ¶
|p| cos ϕ
Let p = be any vector in R2 , where p is the length of the
|p| sin ϕ
vector p and ϕ is the angle that the vector encloses with the x axis.
Then,
µ ¶µ ¶
cos θ − sin θ |p| cos ϕ
q = Aθ p =
sin θ cos θ |p| sin ϕ
µ ¶
cos θ cos ϕ − sin θ sin ϕ
= |p|
sin θ cos ϕ + cos θ sin ϕ
µ ¶
cos(ϕ + θ)
= |p|
sin(ϕ + θ)
This is indeed a vector of the same length as p and that encloses
the angle ϕ + θ with the x axis.
Matrices to Linear Transformations

Let α, β ∈ R, x,y in Rn . Let A be an m × n matrix. The matrix

multiplication satisfies the linearity property:

A(αx + βy) = α(Ax) + β(Ay).

Every transformation that meets this requirement is a linear

transformation.
Definition: A function/map/transformation T : Rm → Rn is said
to be a linear transformation if

T (αx + βy) = α(T x) + β(T y).

for every x, y ∈ Rm and α, β ∈ R.

Matrix as a linear transformation

Let A = (aij ) be an m × n-matrix with aij ∈ R. For x ∈ Rn , let

T : Rn → Rm be defined by T (x) = Ax for every x ∈ Rn . Note that,
  
a11 a12 ··· a1n x1
 a a22 ··· a2n   
 21   x2 
T (x) =   .
 ··· ··· ··· ···  ··· 
am1 am2 ··· amn xn

Then, T : Rn → Rm is a linear transformation. (Why?)

Conversely, can a linear transformation be associated with a
matrix?
Examples
Are the following transformations linear?
Ï f : R → R defined by f (x) = 2x. Yes
Ï f1 : R → R defined by f (x) = 2x + 1. No
Ï g : R → R defined by g (x) = x 2 − x. No
Ï h : R2 → R2 defined by h(x , y ) = (x − y , 2x). Yes
Ï k : R2 → R2 defined by k(x , y ) = (xy , x 2 + 1). No
Ï T : R2 × R2 → R defined by T (A) = |A|. No
Can you suggest a form for linear transformations from R to R?
Let T : R → R be a linear transformation from R to R.

T (x) = T (x · 1) = x · T (1) = x · a,

where a = T (1).
Hence the linear transformation has the form T = ax.
Can you suggest a form for linear transformations from Rn to R
(resp. Rn ) ? Exercise.
Tut 1, Qn 4

· ¸
1
Show that the range of as a linear map R −→ R2 is a line
−1
through 0.
· ¸ · ¸ · ¸
1 t x(t)
A(t) = [t] = = , say.
−1 −t y (t)
Thus x(t) = t , y (t) = −t which are parametric equations of the line
x + y = 0 through 0.
Tut 1, Qn. 4

 
1 −1
Determine domain and range and also show that −1 2  has a
0 1
plane through 0 as its range.

(i) The matrix is 3 × 2, so that’s a linear transformation R2 −→ R3 .

     
· ¸ 1 −1 · ¸ u −v x
u u
(ii) B = −1 2  = −u + 2v  = y , say.
v v
0 1 v z
Thus x = u − v , y = 2v − u , z = v are the parametric equations of a
plane through 0 whose equation.
On eliminating u , v , is x + y − z = 0 .
Tut 1, Qn. 5
· ¸
1 1
Consider the matrix: . Determine the images of
0 1
(i) Unit square {0 ≤ x ≤ 1, 0 ≤ y ≤ 1},
(ii) Unit circle {x 2 + y 2 = 1} and
(iii) Unit disc {x 2 + y 2 ≤ 1} under the above matrices viewed as
linear maps R2 −→ R2 .
(i) Unit square {0 ≤ x ≤ 1, 0 ≤ y ≤ 1} has vertices
{(0, 0)T , (1, 0)T , (1, 1)T , (0, 1)T }. Therefore the image has
vertices {A(0, 0)T , A(1, 0)T , A(1, 1)T , A(0, 1)T }. The full image
is a parallelogram {(0, 0)T , (1, 0)T , (2, 1)T , (1, 1)T }. Why?
· ¸ · ¸ · ¸ · ¸ · ¸
x x +y u x u −v
(ii) A = = =⇒ = and
y y v y v
x 2 + y 2 = 1 =⇒ u 2 − 2uv + 2v 2 = 1 which is an ellipse.
(iii) Elliptic disc enclosed by (ii).
Example (Exercise) · ¸
1 1
Consider the matrix: Determine the images of
1 1
(i) Unit square {0 ≤ x ≤ 1, 0 ≤ y ≤ 1},
(ii) Unit circle {x 2 + y 2 = 1} and
(iii) Unit disc {x 2 + y 2 ≤ 1} under the above matrices viewed as
linear maps R2 −→ R2 .
(i) Unit square {0 ≤ x ≤ 1, 0 ≤ y ≤ 1} has vertices
{(0, 0)T , (1, 0)T , (1, 1)T , (0, 1)T }. Therefore the image has
’vertices’ {(0, 0)T , (1, 1)T , (2, 2)T , (1, 1)T } which are collinear.
The full image is a line segment (0, 0)T to (2, 2)T .
· ¸ · ¸ · ¸
x x +y u
(ii) A = = and
y x +y v
min and max ofpx + y subject to xp2 + y 2 = 1 is given by
min(x +"yp) =# − " 2,pmax(x
# + y ) = 2 =⇒ which is line segment
2 2
from − p to p .
2 2
(iii) Disc enclosed by (ii). [1.5]
System of linear equations

A system of m linear equations in n variables has the form

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
.. .. .. .. .. .. .
. . . . . . = .. (1)
am1 x1 + am2 x2 + · · · + amn xn = bm

Here, aij ’s, 1 ≤ i ≤ m, 1 ≤ j ≤ n are the coefficients;

x1 , x2 , · · · , xn are the unknown variables,
and [b1 , b2 , , · · · bn ]t is the given right hand side vector.
Matrix form
The linear system
 (1) can be writtenas Ax = B , where
a11 a12 . . . a1n
 
 a21 a22 . . . a2n 
A = [ajk ]m×n = 
 .. .. .. 
 is the m × n coefficient
 . . ... . 
am1 am2 . . . amn
matrix;
 
b1
 .. 
B =  .  ∈ Rm is the given right hand side vector,
bm
 
x1
 .. 
and x =  .  ∈ Rn is a column vector of unknown variables.
xn
(Solution Vector)
A solution vector is a vector x whose components constitute a
solution of the system.
Augmented matrix

The matrix A+obtained by augmenting A by column b, written as

a11 a12 . . . a1n b1
 
.  a21 a22 . . . a2n b2 
A+ := [A .. b] := 
 .. .. .. .. 
,
 . . ... . . 
am1 am2 . . . amn bm
 .. 
a
 11 a 12 . . . a1n . b 1
 .. 
+
 a21 a22 . . . a2n . b2 

or A =  . 
 ..
.. .. .. .. 
 . ... . . . 
..
am1 am2 . . . amn . bm
is called the augmented matrix and it completely describes the
system of equations.
Consistent & Inconsistent Systems

The linear system (1) may have

Ï NO SOLUTION (INCONSISTENT)
Ï PRECISELY ONE SOLUTION (CONSISTENT)
Ï or INFINITELY MANY SOLUTIONS (CONSISTENT)

In order to solve the above system of linear equations, Gauss

proposed 3 kinds of operations each of which gives a new linear
system, (hopefully) simpler and equivalent to the original one.
Ï Interchanging two different equations, say equations (i) and
(j).
Ï Multiplying an equation by a scalar and adding it to some
other equation of the system.
Ï Multiplying an equation by a non-zero number.
How to obtain equivalent systems?

We state the elementary row operations for matrices and

equations which yield row-equivalent linear systems.

Elementary Row Operations for Matrices
1 Interchange two rows
(Ri ↔ Rj ) interchange ith and jth rows
2 Addition of a constant multiple of
one row to another row
Ri → Ri + cRj (j 6= i)
3 Multiplication of a row by
a non-zero constant c (6= 0)
Ri → cRi
Elementary Row Operations for Equations
Interchange two equations
Addition of a constant multiple of
one equation to another equation
Multiplication of an equation by
a non-zero constant c (6= 0)

Definition (Row-equivalent systems)

A linear system S1 is row-equivalent to a linear system S2 (S1 ∼ S2 ),
if S2 can be obtained from S1 by finitely many row operations.
Example (Determined System)

We illustrate the Gauss-elimination method for some examples

before formalizing the method.
Solve

−x1 + x2 + 2x3 =2
3x1 − x2 + x3 =6
−x1 + 3x2 + 4x3 = 4.

Solution : The linear system of equations can be expressed as

    
−1 1 2 x1 2
 3 −1 1 x2  = 6
−1 3 4 x3 4
Echelon Form using Row Operations- Forward elimination
phase
The augmented matrix is given by
 
.
 −1
1
1 2 .. 2
+
 . 
A = 
 3 −1 1 .. 6


. 
-1 3 4 .. 4
 
.
R2 →R2 +(3)R1 −1 1 2 .. 2 
−−−−−−−−−−→  . 
R3 →R3 +(−1)R1 0
 2 7 .. 12
−−−−−−−−−−−→ .
0 2 2 .. 2
 
.
−1 1 2 .. 2 
R3 →R3 +(−1)R2  . 
−−−−−−−−−−−→ 0
 2 7 .. 12  
(Row-Echelon Form)
.
0 0 −5 .. −10
1 The circled element is the pivot element at each stage and the
elements in the boxes need to be eliminated.
Back Substitution
The given system has the same solution as the system

−x1 + x2 + 2x3 =2
2x2 + 7x3 = 12
−5x3 = −10.

and this system can be solved easily as

−5x3 = −10 =⇒ x3 = 2;
2x2 + 7x3 = 12 =⇒ 2x2 + 14 = 12 =⇒ x2 = −1;
−x1 + x2 + 2x3 = 2 =⇒ −x1 − 1 + 4 = 2 =⇒ x1 = 1.

Hence, x1 = 1, x2 = −1, x3 = 2.
THIS IS AN EXAMPLE OF A CONSISTENT SYSTEM
WITH A UNIQUE SOLUTION (DETERMINED SYSTEM).
Example (Determined, Inconsistent System)

Solve

−x2 + 3x3 = −1
x1 + 2x2 − x3 = −8
x1 + x2 + 2x3 = −1

Solution : The linear system of equations can be expressed as

    
0 −1 3 x1 −1
1 2 −1 x2  = −8
1 1 2 x3 −1
Echelon Form using Row Operations

The augmented matrix is given by

 
..
0 −1 3 . −1   no x1 in the first equation;

+  .. 
A = 1 2 −1 . −8 interchange with 2nd equation
 
.. R ↔ R : PARTIAL PIVOTING
1 1 2 . −1 2 1
 
.
 1 2 −1 .. −8
R1 ↔ R2  . 
−−−−−→  0
 −1 3 .. −1 
.
1 1 2 .. −1
Contd...

 . 
1 2 −1 .. −8
 
R3 →R3 +(−1)R1  .. 
−−−−−−−−−−−→ −→ 
0 -1 3 . −1
..
0 -1 3 . 7
 
..
1 2 −1 . −8
R3 →R3 +(−1)R2  . 
−−−−−−−−−−−→ 0 −1 3 .. −1
 
..
0 0 0 . 8

The third equation implies INCONSISTENCY. Hence,

NO SOLUTION .
Example : Consistent, determined system (Infinitely many
solutions)

Solve :

x1 + 2x2 = 2
3x1 + 6x2 − x3 = 8
x1 + 2x2 + x3 = 0

Solution : The linear system of equations can be expressed as

    
1 2 0 x1 2
3 6 −1 x2  = 8
1 2 1 x3 0
Echelon Form using Row Operations
The augmented matrix is given by

.. 
 1 2 0
. 2
+  .. 
A = 3 6 −1 . 8
 
..
1 2 1 . 0
 
..
(
R2 →R2 +(−3)R1
−−−−−−−−−−−→  1 2 0. 2  Candidate for second pivot is 0
.. 
R3 →R3 +(−1)R1 0 0 −1 . 2
−−−−−−−−−−−→  ..
 Go to next column
0 0 1 . −2
 .. 

1 2 0 . 2
 . 
−→ 0 0 -1 .. 2 
 
..
0 0 1 . −2
Contd..


.. 
1 2 0 . 2 n
R3 →R3 +(1)R2  .. 
−−−−−−−−−−→  0 0 −1 . 2
 ROW-ECHELON FORM
..
0 0 0 . 0
Back Substitution
That is,

x3 = −2;
x1 + 2x2 = 2 =⇒ x1 = 2 − 2x2 .

Here, x1 and x3 are basic variables and x2 is the free variable (no
pivot).
If we choose x2 = t, then x1 = 2 − 2t.
The solution vector is
     
2 − 2t 2 −2
 t  =  0 +t  1 
−2 −2 0

THIS IS AN EXAMPLE OF A CONSISTENT

DETERMINED SYSTEM WITH INFINITELY MANY
SOLUTIONS.
Exercises : Tut. Sheet 2- Qns 4,5 [2]
Any matrix after a sequence of elementary row operations gets reduced
to what is known as a Row Echelon form.
(Row-Echelon Form (REF))
A matrix is said to be in a row echelon form (or to be a row echelon
matrix) if it has a staircase-like pattern characterized by the following
properties:
(a) The all-zero rows (if any) are at the bottom.
(b) If we call the left most non-zero entry of a non-zero row its leading
entry, then the leading entry of each non-zero row is to the right of the
leading entry of the preceding row.
(c) All entries in a column below a leading entry is zero.
The first non-zero entry in the j th row is known as the j th pivot.
 
3 2 1 3
 
 0 1 −1 6 
The matrix   is in REF. The three pivots are indicated.
 0 0 0 12 
0 0 0 0
 
3 2 1 3
 
 0 1 −1 6 
The matrix   is NOT in REF.
 0 0 0 0 
0 0 0 12
Row echelon form ctd.

In this form each row, except perhaps the first, starts with a string
of zeroes. Each row

starts with strictly more number of zeroes than the previous row.

Ï The j th pivot is below and strictly to the right of (j − 1)th

pivot. All the entries below a pivot are zeroes.
Ï The no. of pivots in a REF of A
≤ the no. of rows in A.
Reduced REF

Ï A matrix in REF can be further row-operated upon to ensure

that (i) each pivot becomes 1
and (ii) all the entries above each pivot become 0.
This is the reduced REF and it is unique.
Ï Reduced REF is mainly of theoretical interest only.
 
2 3 −2 4
0 −9 7 −8
 
1.   is in REF, whereas
0 0 0 20 
0 0 0 0
 
1 0 0 4
2. 0 1 0 −8 is in RREF.
0 0 1 20
Remarks

1. Any row reduced echelon matrix is also a row-echelon matrix.

2. Any nonzero matrix may be row reduced into more than one
matrix in echelon form. But the row reduced echelon form
that one obtains from a matrix is unique.
3. A pivot is a nonzero number in a pivot position that is used as
needed to create zeros with the help of row operations.
4. Different sequences of row operations might involve a different
set of pivots.
5. Given a matrix A, its REF is NOT unique. However, the
position of each of its pivots is unalterable.
A quick quiz!
Determine whether the following statements are true or false.
1. If a matrix is in row echelon form, then the leading entry of
each nonzero row must be 1. F
2. If a matrix is in reduced row echelon form, then the leading
entry of each nonzero row is 1. T
3. Every matrix can be transformed into one in reduced row
echelon form by a sequence of elementary row operations. T
4. If the reduced row echelon form of the augmented matrix of a
system of linear equations contains a zero row, then the
system is consistent. F
5. If the only nonzero entry in some row of an augmented matrix
of a system of linear equations lies in the last column, then the
system is inconsistent. T
6. If the reduced row echelon form of the augmented matrix of a
consistent system of m linear equations in n variables contains
r nonzero rows, then its general solution contains r basic
variables. T (number of free variables = n − r ).
Elementary Matrices

An m × m elementary matrix is a matrix obtained from the m × m

identity matrix Im by one of the elementary operations; namely,

1. interchange of two rows,

2. multiplying a row by a non-zero constant,
3. adding a constant multiple of a row to another
row.

That is, an elementary matrix is a matrix which differs from the

identity matrix by one single elementary row operation.
1. Row Switching Operation
Interchange of two rows - Ri ↔ Rj .
The elementary matrix Pij corresponding to this operation on Im is
obtained by swapping row i and row j of the identity matrix.

C ... Ci ... Cj ... Cm

 1 
R1 1
..  .. 
.  . 
 
Ri 
 0 ... 1 

.  .. 
Pij = .. 
 . 

Rj 
 1 ... 0 

..  
 .. 
.  . 
Rm 1
 
0 1 0
Example : Consider I3 . R1 ↔ R2 gives P12 = 1 0 0 .
0 0 1
2. Row Multiplying Transformation
Multiplying a row by a non-zero constant - Ri −→ kRi . The
elementary matrix Mi (k) corresponding to this operation on Im is
obtained by multiplying row i of the identity matrix by a non-zero
constant k.

C1 C2 ... Ci ... Cm
 
R1 1
 
R2  1 
..  .. 
.  . 
 
Mi (k) =  
Ri  k 
..  
 .. 
.  . 
Rm 1
 
1 0 0
Example : Consider I3 . R3 → 7R3 gives M3 (7) = 0 1 0 .
0 0 7
3. Row Addition Transformation
Ri −→ Ri + kRj : The elementary matrix Eij (k) corresponding to
this operation on Im is obtained by multiplying row j of the identity
matrix by a non-zero constant k and adding with row i.

C1 ... Ci ... Cj ... Cm

 
R1 1
..  .. 
.  . 
 
Ri 
 1 k 

..  .. 
Eij (k) = .  . 
 
Rj 
 1 

..  
 .. 
.  . 
Rm 1
 
1 0 0
Example : R2 → R2 + (−3)R1 for I3 gives E21 (−3) = −3 1 0 .
0 0 1
Proposition 1

Let A be an m × n matrix. If Ae is obtained from A by an elementary

row operation, and E is the corresponding m × m elementary matrix,
e
then E A = A.
Proof :    T
1 0 ··· 0 e1
   T
0 1 · · · 0   e2 
Let Im =  .
. . . . . = . 
. . ..   
. .   .. 
0 0 ··· 1 eT
m
 
0
.
 .. 
 
 
where ei = 1, with 1 is in the i th position, 1 ≤ i ≤ m.
.
.
.
0
Also, eT
i A = A(i ) , the i th row of A.
Recall that row i of AB is row i of A times B.
 .   .   
.. .. ..
 T  T   . 
ej  ej A A(j ) 
     
     
Ri ↔ Rj (i < j) Pij A =  ...  A =  ...  =  ...  = Ae.
     
eT  eT A A(i ) 
 i   i   
.. .. ..
. . .
     
.. .. ..
 T.  T   . . 
Ri −→ kRi Mi (k)A =     
kei  A = kei A = kA(i )  = A.
 e
.. .. ..
. . .
Ri −→ Ri + kRj
     
.. .. ..
 T .   T . T   . 
Eij (k)A = e
 i + ke T
j  A = (e + ke )A = A(i ) + kA(j )  = Ae.
 i j   
.. .. ..
. . .

[3]
Additional Material
Additional Example (Over Determined System)

Solve :

x1 + 2x2 =2
3x1 + 6x2 − x3 =8
x1 + 2x2 + x3 =0
2x1 + 5x2 − 2x3 = 9.

Solution : The linear system of equations can be expressed as

   
1 2 0   2
3 x1
 6 −1  
   8
  x = 
1 2 1 2 0
x3
2 5 −2 9
Echelon Form using Row Operations

The augmented matrix is given by

 .. 
 1 2 0
. 2
 . 
+
 3
 6 −1 .. 8

A =  . 
 1
 2 1 .. 0

.
2 5 −2 .. 9
  
.  no x2 in second equation;
R2 →R2 +(−3)R1 1 2 0 .. 2  



−−−−−−−−−−−→  .. interchange with 4th eqn.
R3 →R3 +(−1)R1 0 0
 −1 . 2  

−−−−−−−−−−−→  .  as 3rd also doesn’t have x2
R4 →R4 +(−2)R1 0 0
 1 .. −2


R2 ↔ R4
−−−−−−−−−−−→ .. 


0 1 −2 . 5  PARTIAL PIVOTING
Contd...

 .. 
1 2 0 . 2
 . 
R2 ↔R4 0
 1 −2 .. 5  
−−−−−→  .. 
0 0 1 . −2
 
..
0 0 −1 . 2
 .. 
1 2 0 . 2
 . 
0 1 −2 .. 5 
−→ 
 ..


0 0 1 . −2
 
..
0 0 -1 . 2
Contd..

 .. 
1 2 0 . 2
 . n
R4 →R4 +(1)R3 0 1
 −2 .. 5  ROW-ECHELON
−−−−−−−−−−→  ..  FORM
0 0 1 . −2
 
..
0 0 0 . 0
Back Substitution

The given system has the same solution as the system

x1 + 2x2 =2
x2 − 2x3 =5
x3 = −2.

and this system can be solved easily as

x3 = −2;
x2 − 2x3 = 5 =⇒ x2 = 5 + 2 × −2 =⇒ x2 = 1;
x1 + 2x2 = 2 =⇒ x1 = 2 − 2 =⇒ x1 = 0.

Hence, x1 = 0, x2 = 1, x3 = −2.
THIS IS AN EXAMPLE OF A CONSISTENT
OVER-DETERMINED SYSTEM.
Additional Example : Consistent, determined system
(Infinitely many solutions) (EXERCISE)

Exercise : Solve

2x1 + 3x2 − 2x3 + 4x4 =2

−6x1 + −9x2 + 7x3 − 8x4 = −3
4x1 + 6x2 − x3 + 20x4 = 13

The solution vector is

       
3
4 − t − 6s 4 −3/2 −6
 2     1  0
  0    
 t  =  +t  +s  
  3  0  − 4 
 3 − 4s 
s 0 0 1
Solution :

The linear system of equations can be expressed as

 
  x1  
2 3 −2 4   2
x2 
−6 −9 7 −8   = −3
x3 
4 6 −1 20 13
x4

The augmented matrix is given by

 
..
 2 3 −2 . 24
+  . 
A =  -6
 −9 7 −8 .. −3
..
4 6 −1 20 . 13
Echelon Form using Row Operations

 
..
R2 →R2 +(3)R1 2 3 −2 4 . 2
−−−−−−−−−−→  . 
R3 →R3 +(−2)R1 0 0 1 4 .. 3 
−−−−−−−−−−−→  ..

0 0 3 12 . −2



 Candidate for second pivot is 0,
x2 is a free variable


Go to next column and choose the pivot

.. 
2 3 −2 4 . 2 (
R3 →R3 +(−3)R2  ..  x4 is also a free variable
−−−−−−−−−−−→  0 0 1 4 . 3
 Row-Echelon Form
.
0 0 0 0 .. 0
Back Substitution

Here, x2 & x4 are free variables and x1 & x3 are the basic variables.
(Free variables do not occur at the beginning of any equation when
the system has been reduced to echelon form).
If we choose x2 = t & x4 = s, then x3 = 3 − 4s and

2x1 = 2 − 3x2 + 2x3 − 4x4

= 2 − 3t + 2(3 − 4s) − 4s
3
= 8 − 3t − 12s =⇒ x1 = 4 − t − 6s .
2
Solution Vector

Hence, the solution vector is

       
3
4 − t − 6s 4 −3/2 −6
 2     1  0
  0
 t  =  +t 

  
+s  
  3  0  −4
 3 − 4s 
s 0 0 1

THIS IS AN EXAMPLE OF A CONSISTENT

DETERMINED SYSTEM WITH INFINITELY MANY
SOLUTIONS.
Additional example: Solution given REF
Example
Consider
 the REF form
 of the augmented matrix given by
1 1 2 3 2
 0 1 5 0 1 
 
 .
 0 0 0 1 2 
0 0 0 0 0
In this case r = 3 (number of pivots) and n = 4 (number of
variables).
Note that x3 is a free variable and x1 , x2 , x4 are basic variables.
Back substitution yields x4 = 2,

x2 + 5x3 = 1 =⇒ x2 = 1 − 5x3 = 1 − 5s
x1 + x2 + 2x3 + 3x4 = 2 =⇒ x1 = 2 − (1 − 5s) − 2s − 3 × 2
=⇒ x1 = −5 + 3s .

We get infinitely many solutions (since x3 is a free parameter).

SUMMARIZING STEPS IN GAUSSIAN ELIMINATION

Stage 1: Forward Elimination Phase

The steps in Gaussian elimination can be summarized as follows:
1. Search the first column of [A|b] from the top to the bottom for the first
non-zero entry, and then if necessary, the second column (the case where
all the coefficients corresponding to the first variable are zero), and then
the third column, and so on. The entry thus found is called the current
pivot.
2. Interchange, if necessary, the row containing the current pivot with the
first row.
3. Keeping the row containing the pivot (that is, the first row) untouched,
subtract appropriate multiples of the first row from all the other rows to
obtain all zeroes below the current pivot in its column.
4. Repeat the preceding steps on the submatrix consisting of all those
elements which are below and to the right of the current pivot.
5. Stop when no further pivot can be found.
REF
The m × n coefficient matrix A of the linear system Ax = b is thus
reduced to an (m × n) row echelon matrix U and the augmented
matrix [A|b] is reduced to
[U |c] =
 
0 . . . p1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ . . . ∗ c 1
 0 . . . 0 . . . p2 ∗ ∗ ∗ ∗ ∗ . . . ∗ c2 
 
 0 ... 0 0 0 ... p ∗ ∗ ∗ . . . ∗ c3 
 3 
 . .. .. .. .. .. .. .. 
 .. . . . 0 0 ... . . . . 
 
 .
 0 . . . 0 0 0 0 0 0 pr ∗ . . . ∗ c r 
 
 0 . . . 0 0 0 0 0 0 0 0 . . . 0 cr +1 
 
 .. .. .. .. .. .. .. .. .. .. .. 
 . . . . . . . . . . . 
0 . . . 0 0 0 0 0 0 0 0 . . . 0 cm
The entries denoted by ∗ and the ci ’s are real numbers; they may or may
not be zero. The pi ’s denote the pivots; they are non-zero. Note that
there is exactly one pivot in each of the first r rows of U and that any
column of U has at most one pivot. Hence r ≤ m and r ≤ n.
Consistent & Inconsistent Systems

If r < m (the number of non-zero rows is less than the number of

equations) and cr +k 6= 0 for some k ≥ 1, then the (r + k)th row
corresponds to the self-contradictory equation 0 = cr +k and so the
system has no solutions (inconsistent system) .
If (i) r = m or (ii) r < m and cr +k = 0 for all k ≥ 1, then there exists
a solution of the system (consistent system) .

(Basic & Free Variables)

If the jth column of U contains a pivot, then xj is called a basic
variable; otherwise xj is called a free variable.
In fact, there are n − r free variables , where n is the number of
columns (unknowns) of A (and hence of U).
Stage 2: (Back Substitution Phase)

In the case of a consistent system, if xj is a free variable, then it can

be set equal to a parameter sj which can assume arbitrary values.
If xj is a basic variable, then we solve for xj in terms of xj +1 , . . . , xm ,
starting from the last basic variable and working our way up row by
row.
Additional references

Ï H. Anton, Elementary linear algebra with applications (8th

edition), John Wiley.
Ï G. Strang, Linear algebra and its applications (4th Edition),
Thomson.
Ï David C. Lay, Linear Algebra and Its Applications (4th
Edition), Pearson education.
Ï Jim DeFranza and Daniel Gagliardi, Introduction to linear
algebra with applications, Waveland Press.