Unit I-3

Linear independence & bases

Unit I-3 Linear Independence & Bases 1

 Linear independence

• any vector w Î Sp{u1, u2, ..., ur} can be expressed

as a lc w = k1u1 + k2u2 + ... + krur with scalars ki
• the zero vector can always be expressed this way
with all ki = 0, but...
• ... if there is a non-zero lc which gives the zero
vector then the set {u1, u2, ...ur} is called linearly
dependent
• equivalently {u1, u2, ...ur} is a linearly independent
set of vectors if k1u1 + k2u2 + ... + krur = 0 implies all
the ki = 0

Unit I-3 Linear Independence & Bases 2

 Simple examples: Linear independence
• {0} is linearly dependent
• any set including 0 is linearly dependent
• any set {v} with one single non-zero vector is
linearly independent
• a set of two non-zero vectors {u,v} is linearly
dependent if and only if u = kv
• with ei = (0,...,0,1,0,....,0), i.e. zeros and 1 in the ith
coordinate, the set {e1, e2, ..., en} is linearly
independent in Rn

Unit I-3 Linear Independence & Bases 3

 Simple examples: Linear independence
• the set {1, t, t2, ...., tn} is linearly independent in Pn
• in C(R,R) the set {1, x, cos x} is linearly
independent
• in C(R,R) the set {1, x, cos2x, sin2x} is linearly
dependent
– because 1 - cos2x - sin2x = 0 (identically zero) is a non-
zero lc of the vectors that adds to the zero function
• in the binary vs on {1,2,3,4} the set {123,124, 34} is
linearly dependent
– 123 + 124 + 34 = Ø is a non-zero lc of vectors giving the
zero vector

Unit I-3 Linear Independence & Bases 4

 Connecting linear dependence and span
• a set {u1, u2, ...,ur} is linearly dependent if and only if at
least one ui is in the span of the other r-1 vectors
– dependency implies there is a lc k1u1 + k2u2 + ... + krur = 0 and
at least one ki ≠ 0 (re-label the list so it’s k1)
– we can solve for u1 = - (k2/k1)u2 - (k3/k1)u3 - ... - (kr/k1) ur
a non-zero lc of the other r-1 vectors
– so u1 Î Sp{u2, u3, ... , ur}

– to prove the converse suppose that u1Î Sp{u2, u3, ... , ur}
– we can write u1 = c2u2 + c3u3 + ... + crur with not all ci = 0
– then u1 - c2u2 - c3u3 - ... - crur = 0 is a non-zero lc
– so the vectors are linearly dependent

Unit I-3 Linear Independence & Bases 5

Example. Show that {u,v,w} is linearly dependent with u=(1-2,1),
v=(2,1,-1), w=(7,-4,1).

Unit I-3 Linear Independence & Bases 6

 Basis of a vector space
• let V = Sp{u1, u2, ...,ur}
• the spanning set {u1, u2, ...ur} may or may not be linearly
dependent, but if it’s linearly dependent....
• ...choose a ui which is a lc of the other r-1 vectors [slide 5]
• then V is also spanned by just those r-1 vectors
{u1,..., ui-1, ui+1,...,ur}
• continuing to ‘cast out’ dependent vectors, we eventually
arrive at a linearly independent set that still spans V
• this is called a basis of V (the plural is bases)

Unit I-3 Linear Independence & Bases 7

 Basis of a vector space

• a basis for V is a linearly independent spanning set

• think of a basis as …
– a maximal linearly independent set OR
– a minimal spanning set
• all bases of a vs V have the same number of vectors
called the dimension of V (dim V)
– some vs may have a basis with an infinite number of vectors

Unit I-3 Linear Independence & Bases 8

 Basis of a vector space
• {u1, u2, ... , un} is a basis of V if and only if the expression
for any vector uÎV as a lc of basis vectors is unique
– suppose u = a1u1 + … + anun = b1u1 + … + bnun
– then u – u = 0 = (a1 – b1)u1 + … + (an – bn)un
– linear independence implies ai – bi = 0 all i
– so ai = bi and the lc for u is unique
– the converse follows a similar argument

Unit I-3 Linear Independence & Bases 9

 Standard bases
• the standard basis in Rn is the set of vectors {e1,e2,...,en }
– in R3 these are written i = (1,0,0), j = (0,1,0), k = (0,0,1)
• in Pn the mononomial basis is {1, t, t2, t3,..., tn }
– dim(Pn) = n+1
• the standard basis for the space of m´n matrices Rm,n
consists of the set of matrices {Eij} in which the ijth entry is
1 and all other entries are zero
– dim(Rm,n) = mn
• the binary vs on the set {1,2,...,n} has a basis {{1},{2},...{n}}
– this vs is n-dimensional

Unit I-3 Linear Independence & Bases 10

 Some useful results about bases
• in an n-dimensional vs V
– any n+1 vectors must be linearly dependent
– any set of n linearly independent vectors is a basis of V
– any spanning set with n vectors is a basis of V
• examples
– any four vectors in R3 must be dependent
– any three linearly independent vectors in R3 are a basis

Unit I-3 Linear Independence & Bases 11

 Subspaces of Rn
• a subspace W < R3 can have dimension no more than 3
• the (geometric) possibilities are:
– point - the origin (dim W = 0)
– line through the origin (dim W = 1)
– plane through the origin (dim W = 2)
– all of R3 (dim W = 3)
• a subspace of Rn can have dimension no more than n
• a hyperplane in Rn is an (n-1)-dimensional subspace

Unit I-3 Linear Independence & Bases 12

 Basis finding: (1) row space method

• to find a basis for W spanned by a given set of vectors:

– write them as the rows of a matrix A so W is the row space of A
– row reduce the matrix A to echelon form
– select the non-zero rows of the echelon matrix
– these span the row space of A
– they are linearly independent [echelon form]
– so they give the required basis
• the dimension of the row space of a matrix A is called
the rank of A

Unit I-3 Linear Independence & Bases 13

Example. Is the set S = { (1,1,1),(1,2,3),(2,-1,1) } a basis for R3?

Unit I-3 Linear Independence & Bases 14

Example. Is the set S = { (1,1,2),(1,2,5),(5,3,4) } a basis for R3?

Unit I-3 Linear Independence & Bases 15

 Bases and subspaces
• any spanning set S of a finite dimensional vs W contains
a basis B obtained from S by deleting any vector that is a
linear combination of the preceding vectors in S
• any set S = {u1,...,ur} of linearly independent vectors in a
finite dimensional vs V can be extended to a basis B of V

expand to a basis of V

W = Sp(S) < V

reduce to a basis of W

Unit I-3 Linear Independence & Bases 16

Example. Find a basis for W = Sp{ (1,-2,5,-3),(2,3,1,-4),(3,8,-3,-5) } and
extend it to a basis for all of R4.

Unit I-3 Linear Independence & Bases 17

 Basis finding: (2) column space method
• to extract a basis for Sp{u1, …, un} < Rm you must
identify linear dependencies in the spanning set
• a linear dependency is expressed as c1u1+ … + cnun = 0
• this can be written in matrix form as Ac = 0 where A is
the matrix with columns u1, …, un and c = (c1, …, cn)
• row-reducing A to echelon B does NOT change the
solutions of Ac = 0, so it does NOT change the
dependency between columns
• use B to identify the pivot columns and select the same
columns from A to get the required basis

Unit I-3 Linear Independence & Bases 18

Example. Find a basis for W = Sp{ (1,-2,5,-3),(2,3,1,-4),(3,8,-3,-5) } using
the column space method.

Unit I-3 Linear Independence & Bases 19

 An IMPORTANT result about rank
• for any matrix A: rank(A) = rank(AT)
– the matrix in the row space method is the transpose of the matrix
in the column space method
– either method will give the same number of basis vectors

The row and column space of a matrix

have the same dimension

Unit I-3 Linear Independence & Bases 20

Example. Find the rank of

Unit I-3 Linear Independence & Bases 21

Example. Find the rank of

Unit I-3 Linear Independence & Bases 22

Example. Find bases for the row space and column space of

Unit I-3 Linear Independence & Bases 23

Example. Find a basis for the row space of
consisting only of rows of A.

Unit I-3 Linear Independence & Bases 24

 A question for understanding
Do row-equivalent matrices A and B have the same column
space?

Unit I-3 Linear Independence & Bases 25

 Coordinates
• V is a finite dimensional vs over R
• choose an ordered basis B = {u1,...,un} for V
• express u Î V (uniquely) as u = a1u1 + ... + anun
• the scalars a1,a2,...,an are called the coordinates of u with
respect to the basis B
• this defines an n-tuple [u]B = [a1,a2,...,an]B ÎRn called the
coordinate vector of u with respect to B
• choose a different basis B¢ = {v1,...,vn}
• then u = b1v1 + ... + bnvn and [u]B¢ = [b1,b2,...,bn]B¢ is not the
same as [u]B but u is the same vector

Unit I-3 Linear Independence & Bases 26

 Coordinates of vectors in Rn
• the n-tuple notation for a vector in Rn is implicitly a
coordinate representation with respect to the standard
basis S
• writing v = (a1, ...,an)ÎRn means v = a1e1 + ... + anen so
[v]S = [a1, ...,an ]S
• the same v could be be represented with respect to a
different basis B and [v]B is NOT the same as [v]S
• this is called a change of coordinates
• useful in dynamics for instance when we change frame of
reference in R3

Unit I-3 Linear Independence & Bases 27

Example. Find the coordinates of the vector v = (2,3,4) with respect to the
basis B = { (1,1,1),(1,1,0),(1,0,0) } of R3.

Unit I-3 Linear Independence & Bases 28

 Change of basis matrix
• for a basis B = {u1,…, un} we have [u1]B = [1,0,…,0]
because u1 = 1u1+0u2+…+0un
• the same is valid for all the ui’s

• now write the nxn matrix P = [u1 u2 … un] with the S

coordinates of the ui’s as columns
• combining these observations gives P[ui]B = [ui]S …

Unit I-3 Linear Independence & Bases 29

 Change of basis matrix
• … and the same for any vector v: P [v]B = [v]S
• P is always invertible because the columns are linearly
independent so [v]B = P-1 [v]S
• P is called the change of basis matrix between S and B
• a map diagram as in the previous example is always
helpful!
• change of basis between two non-standard bases is also
possible

Unit I-3 Linear Independence & Bases 30

Example. Find the change of basis matrix to get [v]C from [v]B where bases
B = { (1,2), (2,1) } and C = { (1,1), (-1,1) }. Use S an an intermediary basis.

Unit I-3 Linear Independence & Bases 31

 The coordinate map

• V is an n-dimensional vs over R
• choose an (ordered) basis B = {u1,...,un} of V
• the coordinate map takes a vector vÎV to its coordinates
with respect to B:
• this is a linear map V ® Rn
– let v = a1u1 + ... + anun and w = b1u1 + ... + bnun
– [v] + [w] = [a1,... ,an] + [b1,... ,bn] = [a1+b1,... ,an+bn] = [v+w]
– k[v] = k[a1,... ,an] = [ka1,... ,kan] = [kv]

Unit I-3 Linear Independence & Bases 32

Example. Find a basis for W = Sp{p1,p2,p3,p4} < P3 where
p1(t) = t3 - 2t2 + 4t + 1 p2(t) = 2t3 - 3t2 + 9t + 1
p3(t) = t3 + 6t – 5 p4(t) = 2t3 - 5t2 + 7t + 5

Unit I-3 Linear Independence & Bases 33

 Sums of subspaces
• let V be a vs and U,W are subspaces
• the sum U+W = {u+w | uÎU, wÎW} is also a subspace
– closure is easily checked
• in fact U+W = span{U,W}
• also the intersection of two subspaces UÇW is a
subspace of V
• if U,W are finite dimensional subspaces of V then
dim(U+W) = dim U + dim W - dim(UÇW)

This result is very IMPORTANT

Unit I-3 Linear Independence & Bases 34

 Sums and intersections in R3
• U = {(a,b,0)ÎR3} is the xy-plane and W = {(0,c,d)ÎR3} is
the yz-plane
• the sum U+W = R3
• the intersection UÇW = {(0,c,0)ÎR3} is the y-axis
• dim(U+W) = dim U + dim W - dim(UÇW) = 2 + 2 - 1 = 3
as it should
• a vector (a,b,c) ÎR3 can be written as a sum of vectors
in U and W, but not uniquely, e.g.
(2,16,-12) = (2,4,0) + (0,12,-12)
= (2,-8,0) + (0,24,-12) etc

Unit I-3 Linear Independence & Bases 35

 Direct sums
• V is the direct sum of the subspaces U and W if
V= U+W AND UÇW = {0}
• we write V = UÅW for the direct sum
• the importance of direct sum:
V = UÅW if and only if any vector vÎV can be
written uniquely as a sum v=u+w, uÎU, wÎW
• if V = UÅW then dim(UÇW) = 0 and so
dim(UÅW) = dim U + dim W

Unit I-3 Linear Independence & Bases 36

 Direct sums in R3
• U = {(a,b,0)ÎR3} is the xy-plane and W = {(0,0,d)ÎR3} is
the z-axis
• then the direct sum UÅW = R3
• dim(U+W) = dim U + dim W = 2 + 1 = 3 as it should be
• a vector (a,b,c) ÎR3 can be written uniquely as a sum of
vectors in U and W, e.g.
(2,16,-12) = (2,16,0) + (0,0,-12)
and no other sum of these kinds of vectors will work

Unit I-3 Linear Independence & Bases 37

Example. Find bases for U, W, U+W, and UÇW where
U = { (a,b,c,d)| b=2c-d} and W = { (a,b,c,d}| b=2c, d=a+c}.

Unit I-3 Linear Independence & Bases 38