Lecture 1:

Matrices and
Systems of
Linear
Equations

Brandon
Behring

Introduction
to the Course
Lecture 1: Matrices and Systems of Linear
Warm Up:
2 × 2 Linear
Equations
Systems

Gaussian
Elimination
Brandon Behring
break!

Matrix
Notation Dynamical Systems Laboratory
Inverses and
Transposes
September 9, 2020
Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
Equations 1 Introduction to the Course
Brandon
Behring
2 Warm Up: 2 × 2 Linear Systems
Introduction
to the Course

Warm Up:
2 × 2 Linear
3 Gaussian Elimination
Systems

Gaussian
Elimination 4 break!
break!

Matrix
Notation 5 Matrix Notation
Inverses and
Transposes

Conclusions
6 Inverses and Transposes
for the week

7 Conclusions for the week

 Lecture 1:
Matrices and
Systems of
Linear
Goals of the course
Equations

Brandon
Behring

Introduction
to the Course

Warm Up:
2 × 2 Linear
Systems

Gaussian
Elimination
We plan to cover a wide variety of important mathematical
break!
topics for modern engineering and data applications.
Matrix
Notation

Inverses and
Transposes

Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
Course overview
Equations

Brandon
Behring

Introduction
to the Course

Warm Up: • Core concepts of Linear Algebra and its modern

2 × 2 Linear
Systems applications. Approximately 4 weeks
Gaussian
Elimination
• Core concepts of Ordinary Differential Equations.
break! Approximately 4 weeks.
Matrix • Introduction to Fourier Analysis, Vector Calculus and an
Notation

Inverses and
introduction to Partial Differential Equations.
Transposes Approximately 4 weeks.
Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
Outline of linear algebra topics
Equations

Brandon
Behring
For the next four weeks, we will cover the following topics. The
first exam will be on October 12th and will only include Linear
Introduction
to the Course Algebra topics.
Warm Up: 1 Matrices and Systems of Linear Equations.
2 × 2 Linear
Systems
2 Vector Spaces, Linear Independence, and Basis.
Gaussian
Elimination 3 Determinants, Eigenvalues and Diagonalization.
break!
4 Applications of Linear Algebra: (To be chosen from)
Matrix
Notation Least-Squares approximation, Linear Regression, Markov
Inverses and Chains, Networks, Singular Value Decomposition, and the
Transposes
Pseudoinverse.
Conclusions
for the week
After the first three classes, all classes will contain integrated
examples. For linear algebra, we will spend the first three
classes quickly going over background material and then spend
a whole class going over modern examples.
 Lecture 1:
Matrices and
Systems of
Linear
Expectations of me
Equations

Brandon
Behring 1 I will post class notes Sunday night. I will post solutions
to the previous weeks’ HW and the questions for next
Introduction
to the Course week’s homework Monday evenings.
Warm Up:
2 × 2 Linear
2 I will be as flexible as possible with being available for
Systems (virtual) office hours to answer questions.
Gaussian
Elimination 3 I have set up a slack channel for students to ask questions
break! about class and hw. I will follow and try to help.
Matrix
Notation 4 This blended format is new for all of us- If you have
Inverses and suggestions for ways to improve the presentation, please
Transposes
let me know, and I will do my best to incorporate your
Conclusions
for the week suggestions.
5 If many people have the same question/a topic isn’t
adequately explained during the lecture, I can record a
video explaining just that question.
 Lecture 1:
Matrices and
Systems of
Linear
Expectations of you: I
Equations

Brandon
Behring 1 All terms which you are expecting to recall are emphasized
Introduction
in red.
to the Course
2 Homework will be given out weekly, and due before the
Warm Up:
2 × 2 Linear following class. Each homework assignment must be typed
Systems
on a word-processor (Microsoft Word or Latex) orcleanly
Gaussian
Elimination written and submitted to NYU Classes. All homework
break! should be written clearly with complete sentences and
Matrix explanations.
Notation

Inverses and 3 Latex is strongly recommended—it is both faster and

Transposes
easier than typesetting equations in word and always
Conclusions
for the week results in a more polished product. To this end, I will
provide the .tex files for the homework, notes, and
solutions to help provide you with tips on how to use
Latex. If you are new to Latex, overleaf is a great resource.
 Lecture 1:
Matrices and
Systems of
Linear
Expectations of you: II
Equations

Brandon
Behring
1 There will be two midterms and a final exam.
Introduction
to the Course 2 The exams will be take-home, not in-class, with a 3-hour
Warm Up:
2 × 2 Linear
window. Students in different time zones may choose a
Systems different start and end time. All work must be scanned
Gaussian
Elimination
(phone is fine) and submitted. Every 10 minutes after the
break!
deadline, your work will be marked down 5 points (out of
Matrix 100). If you believe you will have a problem submitting
Notation
your work, please discuss it with me before the exams.
Inverses and
Transposes 3 You must show all steps on HW and exams- correct
Conclusions
for the week
answers that show no steps or explanations will not receive
credit. Answers that are not right, but show the correct
method can (possibly) receive (near) full credit.
 Lecture 1:
Matrices and
Systems of
Linear
Grade break down
Equations

Brandon
Behring

Introduction
to the Course

Warm Up: Because of the remote nature of the course, homework will be
2 × 2 Linear
Systems weighed more heavily than usual:
Gaussian
Elimination
• Homework (30%)
break! • Midterm Exam 1 (20%)
Matrix
Notation
• Midterm Exam 2 (20%)
Inverses and • Final exam (30%)
Transposes

Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
A system with exactly one solution
Equations
Consider the following system of two equations in two
Brandon
Behring unknowns:
Introduction
to the Course (Equation 1) 3 = 1x + 2y
Warm Up: (Equation 2) 6 = 4x + 5y
2 × 2 Linear
Systems

Gaussian This can be solved in two steps: elimination and

Elimination
back-substitution.
break!

Matrix
• Elimination:Equation 2 − 4 × Equation 1 yields
Notation

Inverses and −6
Transposes −3y = −6 ⇒ y = =2
−3
Conclusions
for the week
• Back-Substitution using Equation 1,

1x + 2(2) = 3 ⇒ x = 3 − 4 = −1.

• The solution is (x, y ) = (−1, 2).

 Lecture 1:
Matrices and
Systems of
Linear
Geometry: Two intersecting lines
Equations

Brandon
Behring
This solution has an interpretation of the intersection of two
lines.
Introduction y
to the Course

Warm Up:
2 × 2 Linear
Systems

Gaussian
Elimination

break!

Matrix
Notation

Inverses and
x
Transposes

Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
A system with no solution
Equations

Brandon
Behring

Introduction The following system has no solution

to the Course

Warm Up:
2 × 2 Linear
(Equation 1) 3 = 1x + 2y
Systems
(Equation 2) 6 = 4x + 8y
Gaussian
Elimination

break! • Elimination: Equation 2 − 4 × Equation 1 yields

Matrix
Notation
0 = −6,
Inverses and
Transposes

Conclusions which is a contradiction

for the week
• Geometrically, we have two parallel lines.
 Lecture 1:
Matrices and
Systems of
Linear
Geometry: Two parallel lines
Equations

Brandon
Behring
(Equation 1) 3 = 1x + 2y
Introduction
to the Course (Equation 2) 6 = 4x + 8y
Warm Up:
2 × 2 Linear
Systems y
Gaussian
Elimination

break!

Matrix
Notation

Inverses and
Transposes
x
Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
A system with infinite solution
Equations

Brandon
Behring
The following system has a whole line of solutions
Introduction
to the Course
(Equation 1) 3 = 1x + 2y
Warm Up:
2 × 2 Linear
Systems
(Equation 2) 12 = 4x + 8y
Gaussian
Elimination • Elimination: Equation 2 − 4 × Equation 1 yields
break!

Matrix
Notation
0 = 0,
Inverses and
Transposes which does not provide any information.
Conclusions The second equation is just four times the first equation!
for the week
• Geometrically, we have two overlapping lines. Any point
on this line satisfies the system.
 Lecture 1:
Matrices and
Systems of
Linear
Geometry: Whole line of solutions
Equations

Brandon
Behring
(Equation 1) 3 = 1x + 2y
Introduction
to the Course (Equation 2) 12 = 4x + 8y
Warm Up:
2 × 2 Linear
Systems y
Gaussian
Elimination

break!

Matrix
Notation

Inverses and
Transposes
x
Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
The general case
Equations

Brandon
Behring

Introduction
to the Course
A system of linear equations has either
Warm Up:
2 × 2 Linear • No solution, or
Systems

Gaussian
• Exactly one solution, or
Elimination
• An infinite number of solutions. For a 2 × 2 system that
break!

Matrix
would be a line. For a 3 × 3 system that may be a line or
Notation a plane, etc.
Inverses and
Transposes In Case 1 we say the equations are inconsistent, otherwise the
Conclusions system is said to be consistent
for the week
 Lecture 1:
Matrices and
Systems of
Linear
A 3 × 3 linear system: 6
Equations

Brandon
possibilities
Behring

Introduction
to the Course

Warm Up:
2 × 2 Linear
Systems

Gaussian
Elimination

break!

Matrix
Notation

Inverses and
Transposes

Conclusions
for the week

What possibility isn’t shown?

 Lecture 1:
Matrices and
Systems of
Linear
Matrix notation
Equations

Brandon
Behring
The system of linear equations
Introduction
to the Course
2x + y − 2z = 10
Warm Up:
2 × 2 Linear 3x + 2y + 2z = 1
Systems

Gaussian 5x + 4y + 3z = 4
Elimination

break! can be written compactly as

Matrix
Notation  
2 1 −2 10
Inverses and
Transposes  3 2 2 1 .
Conclusions
for the week
5 4 4 4

The vertical bar is customary when performing Gaussian

elimination.
 Lecture 1:
Matrices and
Systems of
Linear
Gaussian elimination: Reduction
Equations

Brandon 1 Replace the second row by −a21 R2 + a11 R1 and the third
Behring
row by −a31 R2 + a11 R1 .
Introduction
to the Course 2 Replace the third row by by −ã32 R2 + ã22 R2 .
Warm Up:  
2 × 2 Linear
Systems 2 1 −2 10
3 2 2 1  −3R1 + 2R2
Gaussian
 
Elimination

break! 5 4 4 4 −5R1 + 2R3
 
Matrix
Notation 2 1 −2 10
⇒ 0 1 10 −28 
 
Inverses and 
Transposes

Conclusions
0 3 16 −42 −3R2 + R3
for the week  
2 1 −2 10
⇒ 0 1 10 −28 
 

0 0 −14 42
 Lecture 1:
Matrices and
Systems of
Linear
Gaussian elimination:back
Equations

Brandon
substitution
Behring The reduced system
Introduction
to the Course 2x + y − 2z = 10
Warm Up:
2 × 2 Linear
y + 10z = −28
Systems
− 14z = 42
Gaussian
Elimination

break! can be solved by back substitution

Matrix
Notation −14z = 42 ⇒ z = −3
Inverses and
Transposes

Conclusions
for the week y + 10z = −28 ⇒ y + 10(−3) = −28 ⇒ y = 2

2x + y − 2z = 10 ⇒ 2x + 2 − 2(−3) = 10 ⇒ 2x = 2 ⇒ x = 1
Therefore, the unique solution is (x, y , z) = (1, 2, −3)
 Lecture 1:
Matrices and
Systems of
Linear
Take a Break
Equations

Brandon
Behring

Introduction
to the Course

Warm Up:
2 × 2 Linear
Systems

Gaussian
Elimination

break!

Matrix
Notation

Inverses and
Transposes

Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
Gaussian elimination: example 2
Equations

Brandon
Behring

Introduction
The system of linear equations
to the Course

Warm Up: x + 2y − 3z = 1
2 × 2 Linear
Systems 2x + 5 − 8z = 4
Gaussian
Elimination 3x + 8y − 13z = 7
break!

Matrix can be written compactly written as

Notation
 
Inverses and
Transposes
1 2 − 3 1
Conclusions
 2 5 − 8 4 
for the week
3 8 −13 7
 Lecture 1:
Matrices and
Systems of
Linear
Gaussian elimination: Reduction
Equations

Brandon
Behring

Introduction  
to the Course
1 2 −3 1
Warm Up:
2 5 −8 4  −2R1 + 2R2
 
2 × 2 Linear 
Systems
3 8 −13 7 −3R1 + R3
Gaussian
Elimination
 
2 1 −2 10
break!
⇒ 0 1 −2 2 
 
Matrix 
Notation
0 2 −4 4 −2R2 + R3
Inverses and  
Transposes
2 1 −2 10
Conclusions
⇒ 0 1 −2 2 
 
for the week 
0 0 0 0
 Lecture 1:
Matrices and
Systems of
Linear
Gaussian elimination:back
Equations

Brandon
substitution
Behring The reduced system
Introduction
to the Course x + 2y − 3z = 1
Warm Up:
2 × 2 Linear
y − 2z = 2
Systems

Gaussian can be solved by back substitution. We now only have two

Elimination
equations for three unknowns, so we have a degree-of-freedom.
break!
We parameterize this by letting z = t ∈ R is arbitrary.
Matrix
Notation

Inverses and y − 2z = 2 ⇒ y = 2 + 2t
Transposes

Conclusions
for the week
x + 2y − 3z = 1 ⇒ x + 2(2 + t) − 3t = 1 ⇒ x = −3 − t
Therefore, the solution is the line given by
(x, y , z) = (−3 − t, 2 + 2t, t) where t ∈ R. Geometrically, this
system is the intersection of two planes.
 Lecture 1:
Matrices and
Systems of
Linear
Matrix notation
Equations
• A general m × n matrix is written as
Brandon
Behring
 
a11 a12 · · · a1n
Introduction
 a21 a22 · · · a2n 
m×n
A= . ..  = {aij } ∈ R .
 
to the Course .. ..
 .. . . . 
Warm Up:
2 × 2 Linear am1 am2 · · · amn
Systems

Gaussian it has m rows and n columns.

Elimination
• If A = {aij } and B = {bij } or both m × n matrices and
break!

Matrix
α, β ∈ R, then addition and scalar multiplication are defined by
Notation component:
Inverses and αA + βB = {αaij + βbij }
Transposes

Conclusions
• If A is a m × p matrix and B is a p × n matrix, then the matrix
for the week multiplication AB = C = {cij } where
p
X
cij = aik bkj
k=1

for 1 ≤ i ≤ m and 1 ≤ j ≤ n.
 Lecture 1:
Matrices and
Systems of
Linear
Matrix Multiplication
Equations
• Row times columns: Example (1 × 3) × (3 × 1) is a 1 × 1 matrix
Brandon
Behring (or a scalar)
 
Introduction
to the Course
  1
2 1 1 1 = [2 · 1 + 1 · 1 + 1 · 2] = [5]
Warm Up:
2 × 2 Linear 2
Systems

Gaussian
Next class, we will use this to define the inner product.
Elimination
• Matrix times columns- three inner products:
break!
       
Matrix 2 1 1 1 2·1+ 1·1+1·2 5
Notation  4 −6 0 · 1 =  4 · 1 + (−6) · 1 + 0 · 2 = −2
Inverses and
Transposes
−2 7 2 2 −2 · 1 + 7·1+2·2 9
Conclusions
for the week
• Alternatively, we can think of the product as a combination of
three columns:
           
2 1 1 1 2 1 1 5
 4 −6 0 · 1 = 1  4 + 1 −6 + 2 0 = −2
−2 7 2 2 −2 7 2 9
 Lecture 1:
Matrices and
Systems of
Linear
Matrix multiplication
Equations
Again, recall the definition of matrix multiplication:
Brandon
Behring

Introduction
to the Course

Warm Up:
2 × 2 Linear
Systems

Gaussian
Elimination

break!

Matrix
Notation

Inverses and Matrix multiplication is not commutative. Often, it is not even

Transposes
defined!     
Conclusions
for the week
5 1 2 0 14 3
=
3 −2 4 3 −2 −6
however     
2 0 5 1 10 2
= .
4 3 3 −2 29 −2
 Lecture 1:
Matrices and
Systems of
Linear
Matrix Transpose
Equations

Brandon
Behring If A = {aij }, its transpose is defined as AT = {aji }. For
Introduction
example,
to the Course
   
Warm Up: 2 1 1 2 4 −2
2 × 2 Linear
Systems A =  4 −6 0 ⇒ AT = 1 −6 7
Gaussian −2 7 2 1 0 2
Elimination

break!
If B is a n × m matrix, B T is a m × n matrix
Matrix
Notation  
Inverses and 1 4  
T 1 2 3
Transposes
B= 2 5 ⇒B =
 .
Conclusions 4 5 6
for the week 3 6

In this case, B is 3 × 2, B T is 2 × 3, BB T is a 3 × 3 matrix,

and B T B is a 2 × 2 matrix.
 Lecture 1:
Matrices and
Systems of
Linear
Properties of the transpose
Equations
Some practice using the definition to prove some simple
Brandon
Behring statements:

T
Introduction 1 AT =A
to the Course
Proof: Using the definition
Warm Up:
2 × 2 Linear  T
Systems
A = {aij } ⇒ AT = {aji } ⇒ AT = {aij }.
Gaussian
Elimination

break!
2 (A + B)T = AT + B T
Matrix
Notation Proof: If A and B are the same size, then
Inverses and
Transposes A = {aij }, B = {bij } ⇒ (A + B)T = {aji + bji } = AT + B T
Conclusions
for the week
3 (αA)T = αAT
Proof: Using the definition

A = {aij }, α ∈ R ⇒ (αA)T = {αaji } = α{αaji } = αA)T

 Lecture 1:
Matrices and
Systems of
Linear
Properties of the transpose
Equations

Brandon
Behring
1 If A = {aij } is a m × p matrix and B = {bij } is a p × n
Introduction matrix,, then (AB)T = B T AT
to the Course
Proof: Let AB = C = {cij } where cij = pk=1 aik bkj , then
P
Warm Up:
2 × 2 Linear
Systems p p
X X
Gaussian
Elimination
cji = ajk bki = bki ajk
break!
k=1 k=1
p
Matrix X
T T
Notation = bik akj = B T AT
Inverses and k=1
Transposes

Conclusions
for the week Therefore, C T = {cji } = B T AT .
2 If AT = A, then A is symmetric.
3 If AT = −A, then A is anti-symmetric or skew-symmetric.
 Lecture 1:
Matrices and
Systems of
Linear
Diagonal matrices and the Identity
Equations

Brandon
matrix
Behring • A square n × n matrix d = {dij } is diagonal if is
Introduction nondiagonal entries are zero. We can compactly denote
to the Course
such a diagonal matrix as D = diag (d11 , d22 , . . . dnn ):
Warm Up:
2 × 2 Linear  
Systems d11 . . . 0
D =  ... . . . ..  .
Gaussian 
Elimination . 
break!
0 . . . dnn
Matrix
Notation
• The n × n identity matrix, In , is a diagonal matrix where
Inverses and
Transposes all diagonal entries are 1, In = diag (1, 1, . . . , 1)
Conclusions
for the week
 
1 ... 0
In =  ... . . . ...  .
 

0 ... 1
 Lecture 1:
Matrices and
Systems of
Linear
Matrix Inverse
Equations

Brandon
The inverse of an n × n matrix, if it exists, is another n × n
Behring matrix.If your multiply by A and then multiply by A−1 , you are
Introduction back where you started. If Ax = b, then x = A−1 b; thus
to the Course
A−1 Ax = x.
Warm Up:
2 × 2 Linear 1 The inverse of A is a matrix B such that BA = In and
Systems

Gaussian
AB = In . There is at most one such B, and it is denoted
Elimination by A−1 :
break! A−1 A = In and AA−1 = In .
Matrix
Notation Gaussian elimination solves Ax = b without explicitly
Inverses and
Transposes
finding A−1 .
Conclusions 2 If A is invertible, the one and only solution to Ax = b is
for the week
x = A−1 b
3 Suppose there is a nonzero vector x such that Ax = 0.
Then A cannot have an inverse. No matrix can bring
0 = [0, 0, . . . , 0]T back to x: A0 = 0 for any matrix A.
 Lecture 1:
Matrices and
Systems of
Linear
Matrix Inverse
Equations

Brandon
Behring 1 A 2 × 2 matrix is invertible if ad − bc 6= 0
Introduction
 −1  
to the Course a b 1 d −b
=
Warm Up: c d ad − bc −c a
2 × 2 Linear
Systems

Gaussian
The quantity ad − bc is known as the determinant; we will
Elimination discuss more the 3rd week.
break!
2 For a diagonal matrix D = diag
(d11 , d22 , . . . dnn ),
Matrix −1 −1
Notation D −1 = diag d11 , d22 −1 .
, . . . dnn
Inverses and
Transposes 3 If A and B are both invertible, then (AB)−1 = B −1 A−1 :
Conclusions
(AB) B −1 A−1 = ABB −1 A−1 = AIn A−1 = AA−1 = In


for the week

B −1 A−1 (AB) = B −1 A−1 AB = BIn B −1 = BB −1 = In

 Lecture 1:
Matrices and
Systems of
Linear
Gauss-Jordan elimination
Equations
• We can consider the equation AA−1 = I and find A−1 by
Brandon
Behring Gaussian elimination to create zeros above and below the pivots.
• This process takes n3 operation, which is the same number of
Introduction
to the Course operations as matrix multiplication!
Warm Up:
2 × 2 Linear
• Forward elimination: Provides a decomposition to upper
Systems triangular matrix U and lower triangular matrix L:
Gaussian
Elimination

break!

Matrix
Notation

Inverses and
Transposes

Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
Gauss-Jordan elimination
Equations

Brandon
Behring Next, use row reduction to create zeros above the pivots, then
Introduction divide the rows by the diagonal entries.
to the Course

Warm Up:
2 × 2 Linear
Systems

Gaussian
Elimination

break!

Matrix
Notation

Inverses and
Transposes

Conclusions
for the week
 Lecture 1:
Matrices and
Systems of
Linear
Summary
Equations

Brandon
Behring

Introduction The following techniques will be used for the rest of the course.
to the Course
You will be expected to know how to do them for all future
Warm Up:
2 × 2 Linear homework and exams.
Systems

Gaussian
1 Understand how to use Gaussian elimination to reduce a
Elimination system of linear equations.
break!
2 Be able to visualize the possible solution sets for systems
Matrix
Notation with two or three unknowns.
Inverses and
Transposes 3 Matrix algebra: how to add, multiply, transpose a matrix.
Conclusions 4 Use Gauss-Jordan elimination to find the inverse of a
for the week
matrix (if it exists).
 Lecture 1:
Matrices and
Systems of
Linear
References
Equations

Brandon
Behring

Introduction
to the Course
If you need more review and examples:
Warm Up: • Linear Algebra and Its Applications, 5th edition, David
2 × 2 Linear
Systems Lay.
Gaussian
Elimination
• Schaum’s Outline of Linear Algebra, Lipschutz & Lipson.
break! If today’s lecture was review for you, the following are great
Matrix books on applied linear algebra:
Notation

Inverses and • Linear Algebra and Its Applications, 4th edition, Gilbert
Transposes
Strang.
Conclusions
for the week • Numerical Linear Algebra, Trefethen & Bau.