- Fall 2021 -

ENGINEERING
PROGRAMMING
AND NUMERICAL
METHODS
ASST PROF. ESSAID MERYAM
DEPARTMENT OF ROBOTICS ENGINEERING
 Description

 AIMS
 This chapter is aimed to study linear algebra.

 EXPECTED OUTCOMES
 Students should be able to build blocks of linear algebra: matrices and vectors..
 Students should be able to develop a systematic procedure (the Gauss–Jordan method) for solving linear equations.
Content:

Linear Algebra
 What is Linear Algebra?
 Matrices and Vectors
 Products
 Matrices and Systems of Linear Equations
 Understanding the Dimension of the Solution Space
 The Gauss–Jordan Method for Solving Systems of Linear Equations
 What is Linear Algebra?

 Linear algebra is a branch of mathematics that studies vector spaces, also called linear
spaces, along with linear functions that input one vector and output another.

 It has extensive applications in engineering, physics, natural sciences, computer science,

and the social sciences (particularly in economics). Nonlinear mathematical models can
often be approximated by linear ones.

 The study of linear algebra and matrices has a long history. It first emerged from
determinants, which were used to solve systems of linear equations. Determinants were
used by Leibnitz in 1693, and subsequently by Cramer in 1750 for solving linear systems.
 Matrices and Vectors

  Matrices
 A matrix is any rectangular array of numbers.
 For example:
are all matrices.

 If a matrix A has and , we call A an matrix.

 We refer to as the order of the matrix. A typical matrix A may be written as
 Matrices and Vectors

 Matrices

   For example, if

 then
 Matrices and Vectors

  Matrices
 Sometimes we will use the notation to indicate that is the matrix whose th element is .

 For example, if

 then if and only if , and .

 Matrices and Vectors
  Vectors
 Any matrix with only one column (that is, any matrix) may be thought of as a column vector. The
number of rows in a column vector is the dimension of the column vector. Thus,

 may be thought of as a matrix or a two-dimensional column vector. will denote

 the set of all dimensional column vectors.
 In analogous fashion, we can think of any vector with only one row (a matrix as a row vector. The
dimension of a row vector is the number of columns in the vector.
 Thus, may be viewed as a matrix or a three-dimensional row vector.
 Matrices and Vectors

  Vectors
 vectors appear in boldface type: for instance, vector v. An m-dimensional vector (either row or column) in which
all elements equal zero is called a zero vector (written 0). Thus,

are two-dimensional zero vectors.

 Any m-dimensional vector corresponds to a directed line segment in the m-dimensional plane. For example, in the
two-dimensional plane, the vector corresponds to the line segment joining the point to the point
 Matrices and Vectors

  Vectors
 The directed line segments corresponding to , ,
 The Scalar Product of Two Vectors

   important result of multiplying two vectors is the scalar product. To define the scalar product of two vectors,
An
suppose we have a row vector and a column vector:

 of the same dimension. The scalar product of u and v (written ) is the number .
 For the scalar product of two vectors to be defined, the first vector must be a row vector and the second vector must
be a column vector. For example, if

 then u ∙ v = 1(2) + 2(1) + 3(2) = 10. By these rules for computing a scalar product, if
and

  is not defined, because the vectors are of two different dimensions.

 Matrix Operations

  The Scalar Multiple of a Matrix

 Given any matrix A and any number c (a number is sometimes referred to as a scalar),the matrix is obtained from
the matrix A by multiplying each element of A by c. For example:

 For scalar multiplication of the matrix is sometimes written as .

 Matrix Operations
  Addition of Two Matrices
 Let and be two matrices with the same order (say). Then the matrix is defined to be the m n matrix whose element
is . Thus, to obtain the sum of two matrices A and B, we add the corresponding elements of A and B. For example,
if

 
 This rule for matrix addition may be used to add vectors of the same dimension. For example, if and then Vectors
may be added geometrically by the parallelogram law (Figure below).
 Matrix Operations

 Addition of Two Matrices

Addition of Vectors
 Matrix Operations


  For any two points u and v in the m-dimensional
plane, the line segment joining u and v (called the
line segment ) is the set of all points in the m-
dimensional plane that correspond to the vectors ,
where (next Figure).

 For example, if and then the line segment consists

of the points corresponding to the vectors ], where .
and , we obtain the endpoints of the line segment ;
for , we obtain the midpoint of the line segment .
 Matrix Operations

  The Transpose of a Matrix

 Given any matrix

   the transpose of A (written ) is the matrix

 Matrix Operations

  The Transpose of a Matrix

 Transpose of a matrix swaps rows with columns

  Observe that = A.

 What is the for each of the following matrices:

 Matrix Operations

  The Transpose of a Matrix

 Transpose of a matrix swaps rows with columns

 What is the for each of the following matrices:

Matrix Operations
 Matrix Operations

  Matrix Multiplication
 Given two matrices A and B, the matrix product of A and B (written ) is defined if and only if

 For the moment, assume that for some positive integer has columns and B has rows. Then for some m and n, A is
an matrix and B is an matrix.
 Matrix Operations

 Matrix Multiplication

  Because A is a matrix and B is a matrix, is defined, and

matrix. From Equation (2),
 Matrix Operations
E X AM P L E 2
 Column Vector Times Row Vector:
 Matrix Operations
E X AM P L E 2
 Column Vector Times Row Vector:
 Matrix Operations
E X AM P L E 3
 Row Vector Times Column Vector
 Matrix Operations
E X AM P L E 3
 Row Vector Times Column Vector
Matrix Operations
E X AM P L E 4
 Undefined Matrix Product
 Show that AB is undefined if
Matrix Operations
Matrix Operations
E X AM P L E 4
 Undefined Matrix Product
 Show that AB is undefined if
Matrix Operations
Matrix Operations
Matrix Operations
 Matrices and Systems of Linear Equations

 Consider a system of linear equations given by

  In Equation (3), , are referred to as variables, or unknowns, and the ’s and ’s are constants. A set of equations such
as (3) is called a linear system of m equations in n variables.
 Matrices and Systems of Linear Equations

   denote a possible solution to Equation (3) by an n-dimensional column vector x, in which the ith element of x is
We
the value of . The following example illustrates the concept of a solution to a linear system.
 E X AM P L E 5: Solution to Linear System
Matrices and Systems of Linear Equations
 Matrices and Systems of Linear Equations

 Using matrices can greatly simplify the statement and solution of a system of linear equations. To show how
matrices can be used to compactly represent Equation (3), let
Matrices and Systems of Linear Equations
Matrices and Systems of Linear Equations
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 The Gauss–Jordan method is an efficient method for solving a system of linear equations.

 Using the Gauss–Jordan method, we show that any system of linear equations must satisfy one of the following
three cases:
 Case 1 The system has no solution.
 Case 2 The system has a unique solution.
 Case 3 The system has an infinite number of solutions.

 The Gauss–Jordan method is also important because many of the manipulations used in this method are used
when solving linear programming problems by the simplex algorithm
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Elementary Row Operations
 Before studying the Gauss–Jordan method, we need to define the concept of an elementary row operation
(ERO). An ERO transforms a given matrix A into a new matrix A via one of the following operations.
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Elementary Row Operations
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Elementary Row Operations
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Elementary Row Operations
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Elementary Row Operations
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Elementary Row Operations
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 Finding a Solution by the Gauss–Jordan Method
 The Gauss–Jordan method solves a linear equation system by utilizing EROs in a systematic fashion. We illustrate
the method by finding the solution to the following linear system:

 The augmented matrix representation is

 The Gauss–Jordan Method for Solving
Systems of Linear Equations
  Finding a Solution by the Gauss–Jordan Method
 Suppose that by performing a sequence of EROs on () we could transform () into

 We note that the result obtained by performing an ERO on a system of equations can also be obtained by
multiplying both sides of the matrix representation of the system of equations by a particular matrix. This
explains why EROs do not change the set of solutions to a system of equations.
 Matrix (9) corresponds to the following linear system:
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
  Finding a Solution by the Gauss–Jordan Method
 System (9) has the unique solution . Because (9’) was obtained from (8’) by a sequence of EROs, we know that
(8) and (9) are equivalent linear systems.
 Thus, must also be the unique solution to (8).
The Gauss–Jordan Method for Solving Systems of Linear Equations

 Finding a Solution by
the Gauss–Jordan
Method
The Gauss–Jordan Method for
Solving Systems of Linear
Equations

 Finding a Solution by
the Gauss–Jordan
Method
The Gauss–Jordan Method for
Solving Systems of Linear
Equations

 Finding a Solution by
the Gauss–Jordan
Method
The Gauss–Jordan Method for
Solving Systems of Linear
Equations

 Finding a Solution by
the Gauss–Jordan
Method
The Gauss–Jordan Method for
Solving Systems of Linear
Equations

 Special Cases: No Solution or an Infinite Number of Solutions:

 Some linear systems have no solution, and some have an infinite number of solutions. The following two
examples illustrate how the Gauss–Jordan method can be used to recognize these cases.
The Gauss–Jordan Method for Solving Systems of Linear
Equations
E X AM P L E 6 Linear System with No Solution
 The Gauss–Jordan Method for Solving
Systems of Linear Equations

E X AM P L E 6 Linear
System with No Solution

 
Example 6 illustrates the following
idea: If you apply the Gauss–
Jordan method to a linear
system and obtain a row of the
form then the original linear
system has no solution.
 The Gauss–Jordan Method for Solving Systems of Linear
Equations

E X AM P L E 7 Linear System with Infinite Number of Solutions

 The Gauss–Jordan Method for
Solving Systems of Linear
Equations

E X AM P L E 7 Linear System
with Infinite Number of
Solutions
 The Gauss–Jordan Method for Solving Systems of Linear
Equations

E X AM P L E 7 Linear System with Infinite Number of Solutions

 The Gauss–Jordan Method for Solving
Systems of Linear Equations
How to do Gauss Jordan Elimination

Gauss-Jordan Elimination is an algorithm that can be used to solve systems of linear equations and to find the inverse of
any invertible matrix. It relies upon three elementary row operations one can use on a matrix:

1. Swap the positions of two of the rows

2. Multiply one of the rows by a nonzero scalar.
3. Add or subtract the scalar multiple of one row to another row.
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
 How to do Gauss Jordan Elimination
Steps for Gauss-Jordan Elimination
1. To perform Gauss-Jordan Elimination:
2. Swap the rows so that all rows with all zero entries are on the bottom
3. Swap the rows so that the row with the largest, leftmost nonzero entry is on top.
4. Multiply the top row by a scalar so that top row's leading entry becomes 1.
5. Add/subtract multiples of the top row to the other rows so that all other entries in the column containing
the top row's leading entry are all zero.
6. Repeat steps 2-4 for the next leftmost nonzero entry until all the leading entries are 1.
7. Swap the rows so that the leading entry of each nonzero row is to the right of the leading entry of the row
above it.
 The Gauss–Jordan Method for Solving
Systems of Linear Equations
  
 How to do Gauss Jordan Elimination

 To obtain the inverse of a  matrix  :

1. Create the partitioned matrix , where I is the identity matrix.
2. Perform Gauss-Jordan Elimination on the partitioned matrix with the objective of
converting the first part of the matrix to reduced-row echelon form.
3. If done correctly, the resulting partitioned matrix will take the form 
4. Double-check your work by making sure that.
Questions?