Part IX

Systems of Linear Equations

13 Systems of Linear Equations
13.1 Basics
Definition 13.1.1. A system of equations with the form
a11 x1 + a12 x2 + . . . + a1n xn = b1
a21 x1 + a22 x2 + . . . + a2n xn = b2
.. .. (13.1)
. .
am1 x1 + am2 x2 + ... + amn xn = bm
is called a system of linear equations. Here x1 , . . . , xn are the unknowns to be solved for, the coefficients aij
with 1 ≤ i ≤ m and 1 ≤ j ≤ n and the bi with 1 ≤ i ≤ m on the right hand side are from an underlying field
K. The sigma symbol can also be used to write even more concisely as
n
X
aij xj = bi für i = 1, . . . , m
j=1
.

• We are interested in the solution set of the SLE, i.e. in

L := {(x1 , . . . , xn ) ∈ K n mit (13.1)} .

• Of course this solution set can also be empty.

• Since such systems of linear equations appear in many applications and often with hundreds of variables
and equations (or more) and it is convenient to leave the work up to the computer, we also have to
formulate an effective algorithm so that we can handle them.

• In order to solve the SLE we use equivalence transformations, i.e. transformations that do not change the
solution set of the system.
• This is evident in the fact that these transformations can be completely reversed without the loss of
information.
• The following transformations are sufficient for the solution:
(i) Swapping two equations,
(ii) Multiplying both sides of an equation with a given number 6= 0,
(iii) Adding any desired number of multiples of an equation to another equation.

13.2 Solution Strategy

• When the unknown x1 appears at all in (13.1), then one of the coefficients a11 , . . . , am1 6= 0.
• If a11 = 0 and a`1 6= 0, then in the first step we swap the 1-te and the `-te equations (transformation of
the type (i)).
• This results in
1st row → a`1 x1 + a`2 x2 + ... + a`n xn = b`
.. ..
l . .
`th row → a12 x2 + ... + a1n xn = b1
.. ..
. .
am1 x1 + am2 x2 + ... + amn xn = bm

• In order to avoid confusion of the indices, we still begin with the form of the equation (13.1) and assume
that the numbering has been done such that a11 6= 0.

43
 • Since a11 6= 0 we can divide the first equation by a11 (transformation (ii)). All other equations remain the
same:
x1 + aa11
12
x2 + . . . + aa1n 11
xn = ab11 1

a21 x1 + a22 x2 + . . . + a2n xn = b2

.. ..
. .
am1 x1 + am2 x2 + . . . + amn xn = bm
• Now we use several transformations of the type (iii) in order use the first equation to eliminate the variable
x1 from the other equations. To do so proceed as follows:
– Subtract the a21 fold of the 1st line from the 2nd line
– Subtract the a31 fold of the 1st line from the 3rd line
– ...
– Subtract the am1 fold of the 1st line from the m. line
• This puts the SLE in the following form:
x1 + a012 x2 + ... + a01n xn = b01
a022 x2 + ... + a02n xn = b02
.. .. (13.2)
. .
a0m2 x2 + ... + a0mn xn = b0m

mit a012 = a11 ,

a12
. . . , a01n = a11 ,
a1n
b01 = b1
a11 und a022 = a22 − a21 · aa11
12
, a023 = a23 − a21 · aa13
11
, . . . , b02 = b2 − a21 · ab11
1

usw.
• From here the 1st equation of the SLE (and thus also the first variable x1 ) 13.2 is left unchanged. We
continue to process with the subsystem of equations 2 to m in exactly the same manner.
• Then
a022 x2 + a023 x3 + ... + a02n xn = b02
.. ..
. .
a0m2 x2 + a0m3 x3 + ... + a0mn xn = b0m
• becomes (when x2 is present at all, otherwise analogous to x3 )
x2 + a0023 x3 + ... + a002n xn = b002
.. ..
. .
a00m3 x3 + ... + a00mn xn = b00m

• Thus up to now we have

x1 + a012 x2 + a013 x3 + ... + a01n xn = b01
x2 + a0023 x3 + ... + a002n xn = b002
.. ..
. .
a00m3 x3 + ... + a00mn xn = b00m

• If we continue this process accordingly, we then arrive at a echelon form (will be defined more exactly
later) from which the result can easily be derived.

13.3 Matrix Notation

Definition 13.3.1. A rectangular arrangement of numbers with m by n elements of a field K is called a
m × n-matrix. We write  
a11 a12 . . . a1n
 a21 a22 . . . a2n 
A= ..  ∈ K m×n ,
 
 . 
am1 am2 ... amn
or in short form A = (aij ) 1≤i≤m (or simply (aij ), when m and n are clear).
1≤j≤n
K m×n is the set of all m × n-matrices with entries from K (often also Mm,n (K)).

44
Definition 13.3.2. The SLE (13.1) can be abbreviated as Ax = b. This means A = (aij ) 1≤i≤m ∈ K m×n the
1≤j≤n
coefficient matrix, x = (xj )1≤j≤n ∈ K n the vector from the unknown and b = (bi )1≤i≤m ∈ K m the right side
or inhomogeneity3 . The SLE is called a homogenous when b1 = b2 = · · · = bm = 0 (short: b = 0), otherwise
inhomogenous. A particularly economical notations for solving the SLE (13.1) is the notation with the extended
coefficient matrix (A|b) ∈ K m×(n+1) .

13.4 Elementary Line Transformations and Row Echelon Form

• Now it is possible to operate using the extended coefficient matrix exactly the same way as with the
associated SLE.
• The three allowed transformations already formulated are therefore now noted once again for (extended)
coefficient matrices.
• Elementary line transformations
The solution set of an SLE does not change when the following transformations are applied in any desired
sequence to the extended coefficient matrix:
(I) Swapping two lines
(II) Multiplication of a line by a constant α ∈ K \ {0}
(III) Addition of the αfold of any line α ∈ K, to another line.
• These elementary line transformations can now be used as stated above to put any matrix (extended or
not) into echelon form.

Definition 13.4.1. A m × n-matrix A ∈ K m×n is said to be in row echelon form when it has the following
form:  • 
•
 • 
•
 
 
 • 
..
0
 
.
 
 
 • 

↑ ↑ ↑ ↑ ↑ ..........
j1 j2 j3 j4 j5
Here the elements at the positions marked with • (these are called pivot elements, the associated columns are
called pivot columns) are not equal to zero there are exclusively zeros below the step line.
Example 13.4.2.
   
1 1 0 0 0 20 1 1 0 0 0 20
 1
 0 −1 0 0 −20 
 Z2 ←Z2 −Z1  0 −1 −1 0
 0 −40  Z3 ←Z3 +Z2
 0 1 0 −1 0 −10   0 1 0 −1 0 −10 
0 0 1 1 −1 0 0 0 1 1 −1 0
   
1 1 0 0 0 20 1 1 0 0 0 20
 0
 −1 −1 0 0 −40   Z4 ←Z4 +Z3  0 −1 −1 0
 0 −40 
 0 0 −1 −1 0 −50   0 0 −1 −1 0 −50 
0 0 1 1 −1 0 0 0 0 0 −1 −50

13.5 From the Row Echelon Form to the Solution

Example 13.5.1. Let  
10 0 0 0 0 5 6 3
 
 5 2 3 4 5 5
(A|c) =  

 0 4 7


9
3 The vectors from the K n (analogous to K m ) are usually written as column vectors. This can be made clearer by writing more

exactly K n×1 instead of K n

45
The system of equations Ax = c is not solvable because its fourth equation is 0x1 + . . . + 0x8 = 9, i.e. it can
not be satisfied by assigning a value to xi , since 0 · x = 0 for all x in every field.
 
1 0 0 0 0 0 5 6 18
 
5 2 3 4 5 10
Example 13.5.2. Let

(A|b) =  

 0 4 12


0
This does not result in a direct contradiction, as in the last example.
The SLE is solvable, with the solution according to the respective pivot elements.
The pivot columns are j1 = 1, j2 = 4 and j3 = 8.
The associated variables x1 , x4 , x8 are also called bound variables, while the remaining variables x2 , x3 , x5 , x6
and x7 are referred to as free variables.
The following results from the row echelon form (from bottom to top)
x8 = 3
2 3 4
x4 = 2 − 5 x5 − 5 x6 − 5 x7 − x8 =
2 3 4
= −1 − 5 x5 − 5 x6 − 5 x7
x1 = 18 − 5x7 − 6x8 = −5x7

x1 = −5x7
bzw. x4 = (−1) + (− 52 x5 ) + (− 53 x6 ) + (− 45 x7 )
x8 = 3
In detail, this is
x1 = −5x7
x2 = x2
x3 = x3
x4 = (−1) + (− 25 x5 ) + (− 35 x6 ) + (− 54 x7 )
x5 = x5
x6 = x6
x7 = x7
x8 = 3

This can also be grouped together by column

 0   0   −5 
x1 0 0 0
       
 x2   0  1 0 0
 0  0
 0   00 
 x3   0  0 1      
x4  −1  2  3  4
  =   + x2 0 + x3 0 + x5 − 5  + x6 − 5  + x7 − 5 
   
 x5   0  0 0  1   0   0 
x   0  0 0
 0   1   0 
     
 6  
x7 0
 
0
 
0 0 0 1
x8 3 0 0 0 0 0

Usually the remaining variables are renamed

 0   0   −5 
x1 0 0 0
       
x
 2   0 1 0 0
 0  0
 0   00 
 x3   0  0 1
   
     
x4  −1  2  3  4
  =   + λ1 0 + λ2 0 + λ3 − 5  + λ4 − 5  + λ5 − 5 
   
 x5   0  0 0  1   0   0 
x   0  0 0
 0   1   0 
     
 6  
x7 0
 
0
 
0 0 0 1
x8 3 0 0 0 0 0

with λi ∈ K. (here e.g. K = Q). Therefore we also write

 0   0   −5 
x1 0 0 0
       
x
 2  0 1 0 0
 0  0
 0   00 
x3   0  0 1
   
     
x4  −1  2  3  4
  =   + Q0 + Q0 + Q− 5  + Q− 5  + Q− 5 
   
x5   0  0 0  1   0   0 
x   0  0 0
 0   1   0 
     
 6  
x7 0
 
0
 
0 0 0 1
x8 3 0 0 0 0 0

46
 • The process of elementary line transformations up to row ech-
elon form is usually called forwards elimination.

• The process of reading and substituting from the last equation

up to the first is usually called backwards substitution.
• Both processes together collectively are usually referred to
as Gauß algorithm (after Johann Carl Friedrich Gauß (1777-
1855) for the solution of systems of linear equations.
Figure 53: Carl Friedrich Gauß

Example 13.5.3 (Continuation of 13.4.2). In 13.4.2 the row echelon form

 
1 1 0 0 0 20
 0 −1 −1 0 0 −40 
 
 0 0 −1 −1 0 −50 
0 0 0 0 −1 −50

was calculated.
Since there is no contradiction in the last line, the SLE is solvable.
From this echelon form one reads

x5 = 50 , x3 = 50 − x4 , x2 = 40 − x3 , x1 = 20 − x2 .

Backwards substitution results in

x5 =50 , x3 = 50 − x4 , x2 = 40 − (50 − x4 ) = −10 + x4 ,

x1 =20 − x2 = 20 − (−10 + x4 ) = 30 − x4 ,

or with λ := x4 ∈ R together      
x1 30 −1
x2  −10 1
     
x3  =  50  + λ −1
     
x4   0  1
x5 50 0
Since λ ∈ R can have an unlimited number of values, there are an infinite number of solutions to this SLE.

47
48
List of Figures
1 Simple Venn Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Intersection of two sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3 Union of two sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4 Difference of two sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
5 Complement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6 Symmetrical difference of sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
7 mapping f of A to B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8 Injective mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
9 Surjective mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
10 Bijective mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
11 Graph of a relation that is not a function of x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
12 Graph of a function R2 → R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
13 Graph of a function R → R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
14 Graph of x 7→ x2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
15 Graph of x 7→ x3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
16 Graph of x 7→ x3 − x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
17 Graph of x 7→ ex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
18 Injective mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
19 Bijective mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
20 Even degree polynomial, LC>0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
21 Even degree polynomial, LC<0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
22 Odd degree polynomial, LC>0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
23 Odd degree polynomial, LC<0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
24 Graph of f (x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
25 Graph of g(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
26 Absolute value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
27 Graph of x 7→ x2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
28 Graph of g . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
√
29 Square and . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
√
30 Square and − . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
31 On the Radian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
32 Definition of Sine and Cosine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
33 Graphs of Sine and Cosine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
34 Definition Tangent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
35 Graph of tangent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
36 Slope of a line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
37 sin(x) and arcsin(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
38 cos(x) and arccos(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
39 tan(x) and arctan(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
40 Graphs of 2x ,
ex =
exp(x) and 10x . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
x x 1 x
41 Graphs of 12 , 1e = exp(−x) and 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
42 Secant of the graph of f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
43 Local and Global Extrema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
44 Saddle point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
45 Extrema on an interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
46 Newton Iteration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
47 tan(x) = x . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
48 Newton spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
49 Point of inflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
50 Strip method, rough . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
51 Strip method, finer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
52 Definite integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
53 Carl Friedrich Gauß . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

49
50
Index
n-tuples, 3 factor, 18
field, 2, 43
point symmetrical , 14 function, 12
→mapping, 12
absolute value, 19 absolute value, 19
addition theorems, 21 bijective, 13
algorithm, 43 constant, 17
Gauß, 47 exponential, 23
antiderivative, 37 injective, 12
antisymmetrical, 11 linear, 17
arc length, 20 polynomial, 17
associativity, 2, 7 quadratic, 17
asymmetrical, 11 rational, 18
surjective, 12
base, 23 trigonometric, 21, 22
bijective, 13 Fundamental Theorem of Calculus, 39

cardinal, 1 graph
cardinality, 1 of a mapping, 13
circle constant, 20 group, 2
co-domain, 12 abelian, 2
coefficients, 43 commutative, 2
commutativity, 2, 7 identity element, 2
complement, 7 inverse element, 2
composition, 15
concave, 35 homogenous, 45
concave down, 35 Horner scheme, 17, 28
concave up, 35 Horner-Schema, 19
conclusion, 3
convex, 35 idempotence laws, 7
cosine, 21 identical mapping, 15
identity element, 2
decreasing image, 12, 15
monotonic, 13, 28, 33 increasing
strictly monotonic, 13, 33 monotonic, 13, 28, 33
degree, 17 strictly monotonic, 13, 33
derivation, 31 induction
derivative function, 31 complete, 5
difference quotient, 31 inhomogeneity, 45
differentiable, 31 inhomogenous, 45
differential quotient, 31 injective, 12, 33
distributive laws, 7 integral
domain, 12 definite, 39
improper, 40
echelon form, 44, 45 indefinite, 37, 39
element integration
inverse, 2 partial, 37
elements, 1 inverse image of P , 16
elimination inverse mapping, 15
forwards, 47 irreflexive, 11
equivalence transformations, 43 iteration
expansion Newton, 35
constant, 18
extremum, 33 laws
global, 33 De Morgan’s, 4, 8
local, 33 leading coefficient, 17

51
leading term, 17 pivot
linear, 32 columns, 45, 46
logical connectives, 3 elements, 45
point
mapping, 12 of inflection, 35
→function, 12 saddle, 34
bijective, 13 pole, 18
composite, 15 polynomial
even, 13 function, 17
graph of a, 13 root, 27
injective, 12 zero, 27
inverse, 15 premise, 3
odd, 13 product
surjective, 12 n-ary cartesian, 3
Mathematicians cartesian, 2
Cantor, Georg (1845-1918), 1 cross, 2
De Morgan, Augustus (1806-1871), 4 n-ary, 3
Euler, Leonhard (1701-1783), 6 proof
Gauß, Johann Carl Friedrich (1777-1855), 47 by construction, 5
Venn, John (1834-1923), 6 by contradiction, 5
matrix, 44 by induction, 5
coefficient, 45 direct, 5
extended, 45 indirect, 5
maximum proposition, 3
global, 33 Pythagoras, 21
local, 33
minimum quadratic formula, 27
global, 33
local, 33 radian, 20
monoton reflexive, 11
strictly increasing, 33 relation, 11
monotonic binary, 11
decreasing, 13 antisymmetrical, 11
increasing, 13, 33 asymmetrical, 11
strictly decreasing, 13, 33 irreflexive, 11
strictly increasing, 13 reflexive, 11
monotony, 33 symmetrical, 11
transitive, 11
necessary, 3 equivalence, 11
number trivial, 11
cardinal, 1 remainder, 18
Euler’s, 23 restriction, 16
numbers root
complex, 2 square, 19
natural, 2 rule
rational, 2 chain, 32
real, 2 product, 32
whole, 2 quotient, 32

operation set, 1
binary, 2 complement, 7
order difference, 6
partial rules of operation, 7
irreflexive, 11 empty, 2
reflexive, 11 identity, 1
intersection, 6
pair power, 6
ordered, 2 rules of operation, 7
periodic, 21 symmetrical difference, 7

52
 union, 6 symmetrical, 11
sine, 21 axially, 14
SLE, 43 system
slope, 22 of linear equations, 43
statement, 3 system of equations
necessary, 3 linear
sufficient, 3 homogenous, 45
stationary, 33 inhomogenous, 45
subset, 1 right side, 45
real, 1
trivial, 6 tangent, 22
substitution theorems
backwards, 47 addition, 21
sufficient, 3 transformation
superset, 1 equivalence, 27
surjective, 12 transformations
symbol elementary line, 45
(a, b), 2 transitive, 6, 11
(a1 , a2 , . . . , an ), 3 truth table, 3
tuple, 2
A × B, 2
An , 3 unknowns, 43
A1 × A2 × · · · × An , 3
D(f ), 12 variables
K m×n , 44 bound, 46
W (f ), 12 free, 46
C, 2
∆, 7 zero, 27
⇐, 3
N, 2
Q, 2
R, 2
⇒, 3
Z, 2
|M |, 1
∩, 6
◦, 15
{A, 7
∪, 6
∅, 2
∃, 4
∃1 , 4
∀, 4
dx , 31
df

idM , 15
⇐⇒ , 3
¬, 3
A, 7
\, 6
⊂, 1
⊆, 1
(, 1
∨, 3
∧, 3
aRb, 11
f 0 , 31
f (U ), 15
f |U , 16
f −1 , 15, 16

53