Lecture 20: 5.

5
Row Space, Column Space, and
Nullspace
Wei-Ta Chu

2008/12/3
Row Space and Column Space

 Definition
 If A is an mn matrix, then the subspace of Rn spanned by
the row vectors of A is called the row space (列空間) of A,
and the subspace of Rm spanned by the column vectors is
called the column space (行空間) of A.
 The solution space of the homogeneous system of equation

Ax = 0, which is a subspace of Rn, is called the nullspace (

零核空間) of A.

2008/12/3 Elementary Linear Algebra 2

Remarks

 In this section we will be concerned with two

questions
 What relationships exist between the solutions of a linear
system Ax=b and the row space, column space, and
nullspace of A.
 What relationships exist among the row space, column
space, and nullspace of a matrix.

2008/12/3 Elementary Linear Algebra 3

Remarks

 It follows from Formula(10) of Section 1.3

 We conclude that Ax=b is consistent if and only if b is

expressible as a linear combination of the column vectors
of A or, equivalently, if and only if b is in the column
space of A.

2008/12/3 Elementary Linear Algebra 4

Theorem 5.5.1

 Theorem 5.5.1
 A system of linear equations Ax = b is consistent if
and only if b is in the column space of A.

2008/12/3 Elementary Linear Algebra 5

Example
1 3 2 x1  1 
1 2 3  
 Let Ax = b be the linear system  x
   
2 9

2 1 2 

x3 
 
3
Show that b is in the column space of A, and express b as a linear
combination of the column vectors of A.

 Solution:
 Solving the system by Gaussian elimination yields
x1 = 2, x2 = -1, x3 = 3
 Since the system is consistent, b is in the column space of A.
 Moreover, it follows that 1  3 2  1 
21 
2 3 
3 
       9 

2 
 
1
 
2 
  
3
 

2008/12/3 Elementary Linear Algebra 6

General and Particular Solutions

 Theorem 5.5.2
 If x0 denotes any single solution of a consistent linear
system Ax = b, and if v1, v2,…,vk form a basis for the
nullspace of A, (that is, the solution space of the
homogeneous system Ax = 0), then every solution of Ax = b
can be expressed in the form
x = x0 + c1v1 + c2v2 + · · · + ckvk
Conversely, for all choices of scalars c1, c2,…,ck, the
vector x in this formula is a solution of Ax = b.

2008/12/3 Elementary Linear Algebra 7

Proof of Theorem 5.5.2

 Assume that x0 is any fixed solution of Ax=b and that x is

an arbitrary solution. Then Ax0 = b and Ax = b.
 Subtracting these equations yields
Ax –Ax0 = 0 or A(x-x0)=0
 Which shows that x-x0 is a solution of the homogeneous
system Ax = 0.
 Since v1, v2, …, vk is a basis for the solution space of this
system, we can express x-x0 as a linear combination of
these vectors, say x-x0 = c1v1+c2v2+…+ckvk. Thus,
x=x0+c1v1+c2v2+…+ckvk.

2008/12/3 Elementary Linear Algebra 8

Proof of Theorem 5.5.2

 Conversely, for all choices of the scalars c1,c2,

…,ck, we
have
Ax = A(x0+c1v1+c2v2+…+ckvk)
Ax = Ax0 + c1(Av1) + c2(Av2)+… +ck(Avk)
 But x0 is a solution of the nonhomogeneous system, and
v 1, v 2,
…, vk are solutions of the homogeneous system, so
the last equation implies that
Ax = b + 0 + 0 +… +0 = b
 Which shows that x is a solution of Ax = b.

2008/12/3 Elementary Linear Algebra 9

Remark

 Remark
 The vector x0 is called a particular solution (特解) of Ax =
b.
 The expression x0 + c1v1 + · · · + ckvk is called the general
solution (通解) of Ax = b, the expression c1v1 + · · · + ckvk is
called the general solution of Ax = 0.
 The general solution of Ax = b is the sum of any particular
solution of Ax = b and the general solution of Ax = 0.

2008/12/3 Elementary Linear Algebra 10

Example (General Solution of Ax = b)
 The solution to the x1   3r 4 s 2t  0   3  4 2
nonhomogeneous system x   r  0  1  0  0 
 
2         
x3   2 s  0  0   2 0 
x1 + 3x2 –2x3 + 2x5 =0          t  
 r s
x
4   s  0  0  1  0 
2x1 + 6x2 –5x3 –2x4 + 4x5 –3x6 = -1
x5   t  0  0  0  1 
5x3 + 10x4 + 15x6 = 5           
2x1 + 5x2 + 8x4 + 4x5 + 18x6 = 6 
x6  
 1/ 3 
 
1 / 3 
  0

0 


0

x0 x

is which is the general solution.

 The vector x0 is a particular
x1 = -3r - 4s - 2t, x2 = r, solution of nonhomogeneous
x3 = -2s, x4 = s,
x5 = t, x6 = 1/3 system, and the linear
combination x is the general
 The result can be written in vector solution of the homogeneous
form as system.

2008/12/3 Elementary Linear Algebra 11

Elementary Row Operation

 Performing an elementary row operation on an

augmented matrix does not change the solution set of the
corresponding linear system.
 It follows that applying an elementary row operation to a
matrix A does not change the solution set of the
corresponding linear system Ax=0, or stated another way,
it does not change the nullspace of A.

The solution space of the homogeneous system of equation Ax = 0, which is a

subspace of Rn, is called the nullspace of A.

2008/12/3 Elementary Linear Algebra 12

Example
2 2 1 0 1 
1 1 2 3 1 
 Find a basis for the nullspace of A  
1 1 2 0 1
 
0 0 1 1 1 
 Solution
 The nullspace of A is the solution space of the homogeneous system
2x1 + 2x2 – x3 + x5 = 0
-x1 –x2 –2 x3 –3x4 + x5 = 0
x1 + x2 –2 x3 –x5 = 0
x3 + x4 + x5 = 0
 In Example 10 of Section 5.4 we showed that the vectors
1 1
1  0 
   
v1 0  and v 2 1
   
0
  0 

0 
 
1 

form a basis for the nullspace.

2008/12/3 Elementary Linear Algebra 13

Theorems 5.5.3 and 5.5.4

 Theorem 5.5.3
 Elementary row operations do not change the nullspace of a
matrix.

 Theorem 5.5.4
 Elementary row operations do not change the row space of a
matrix.

2008/12/3 Elementary Linear Algebra 14

Proof of Theorem 5.5.4

 Suppose that the row vectors of a matrix A are r1,r2, …, rm,

and let B be obtained from A by performing an
elementary row operation.
 We shall show that every vector in the row space of B is
also in that of A, and that every vector in the row space of
A is in that of B.
 If the row operation is a row interchange, then B and A
have the same row vectors and consequently have the
same row space.

2008/12/3 Elementary Linear Algebra 15

Proof of Theorem 5.5.4

 If the row operation is multiplication of a row by a

nonzero scalar or a multiple of one row to another, then
the row vector r1’ ,r2’,
…, rm’ofB are linear combination
of r1,r2,…, rm; thus they lie in the row space of A.
 Since a vector space is closed under addition and scalar
multiplication, all linear combination of r1’ ,
r2’,…, rm’wi
ll
also lie in the row space of A. Therefore, each vector in
the row space of B is in the row space of A.

2008/12/3 Elementary Linear Algebra 16

Proof of Theorem 5.5.4

 Since B is obtained from A by performing a row

operation, A can be obtained from B by performing the
inverse operation (Sec. 1.5).
 Thus the argument above shows that the row space of A is
contained in the row space of B.

2008/12/3 Elementary Linear Algebra 17

Remarks

 Do elementary row operations change the column space?

 Yes!
 The second column is a scalar multiple of the first, so the
column space of A consists of all scalar multiplies of the
first column vector.

Add -2 times the first

row to the second

 Again, the second column is a scalar multiple of the first,

so the column space of B consists of all scalar multiples
of the first column vector. This is not the same as the
column space of A.
2008/12/3 Elementary Linear Algebra 18
Theorem 5.5.5

 Theorem 5.5.5
 If A and B are row equivalent matrices, then:
 A given set of column vectors of A is linearly
independent if and only if the corresponding column
vectors of B are linearly independent.
 A given set of column vectors of A forms a basis for the
column space of A if and only if the corresponding
column vectors of B form a basis for the column space of
B.

2008/12/3 Elementary Linear Algebra 19

Theorem 5.5.6

 Theorem 5.5.6
 If a matrix R is in row echelon form, then the row
ve ctor
swi ththel eading1’ s(i.e
.,thenonz eror ow
vectors) form a basis for the row space of R, and
thec olumnve ctor swi t
hthel eading1’ soft her ow
vectors form a basis for the column space of R.

2008/12/3 Elementary Linear Algebra 20

Bases for Row and Column Spaces
T he m atrix
1 2 5 0 3
0 1 3 0 0
R  
0 0 0 1 0
 
0 0 0 0 0
is in row -echelon form . From T heorem 5.5 .6 the vectors
r1 [1 -2 5 0 3]
r2 [0 1 3 0 0]
r3 [0 0 0 1 0]
form a basis for the row space of R , and the vectors
1 2  0
0  1  0 
c 1  , c 2  , c 4  
0 0  1
   
0 0  0
form a basis for the colum n space of R .

2008/12/3 Elementary Linear Algebra 21

Example

 Find bases for the row and column spaces of

1 3 4 2 5 4 
2 6 9 1 8 2 
A  
2 6 9 1 9 7 
 
 Solution: 1 3 4 2 5 4 

 Since elementary row operations do not change the row space of a

matrix, we can find a basis for the row space of A by finding a basis that
of any row-echelon form of A.
 Reducing A to row-echelon form we obtain

1 3 4 2 5 4 


0 0 1 3 2 6 
R  

0 0 0 0 1 5
 
0 0 0 0 0 0


2008/12/3 Elementary Linear Algebra 22

Example 1 3 4 2 5 4 
2 6 9 1 8 2 
A   R



1 3 4 2 5 4 
0 0 1 3 2 6 

2 6 9 1 9 7  0 0 0 0 1 5
   
1 3 4 2 5 4  0 0 0 0 0 0

 The basis vectors for the row space of R and A

r1 = [1 -3 4 -2 5 4]
r2 = [0 0 1 3 -2 -6]
r3 = [0 0 0 0 1 5]
 Keeping in mind that A and R may have different column
spaces, we cannot find a basis for the column space of A
directly from the column vectors of R.
 From Theorem 5.5.5b, we can find the basis for the column
space of R, then the corresponding column vectors of A will
form a basis for the column space of A. 1 
 
4 
 
5 
  2 9 8
c1  , c 3  , c5  
2  9  9 
     
1 4  5

2008/12/3 Elementary Linear Algebra 23

Example (Basis for a Vector Space
Using Row Operations )
 Find a basis for the space spanned by the vectors
v1= (1, -2, 0, 0, 3), v2 = (2, -5, -3, -2, 6),
v3 = (0, 5, 15, 10, 0), v4 = (2, 6, 18, 8, 6).
 Except for a variation in notation, the space spanned by these
vectors is the row space of the matrix
1 2 0 0
 3 1 2 0 0 3


2 5 3 2 6 
0 1 3 2 0 
   

0 5 15 10 0 
0 0 1 1 0
   
2 6 18 8
 6 0
 0 0 0 0 
 The nonzero row vectors in this matrix are
w1= (1, -2, 0, 0, 3), w2 = (0, 1, 3, 2, 0), w3 = (0, 0, 1, 1, 0)
 These vectors form a basis for the row space and consequently form a
basis for the subspace of R5 spanned by v1, v2, v3, and v4.

2008/12/3 Elementary Linear Algebra 24

Remarks

 Keeping in mind that A and R may have different column spaces, we

cannot find a basis for the column space of A directly from the
column vectors of R.
 However, it follows from Theorem 5.5.5b that if we can find a set of
column vectors of R that forms a basis for the column space of R,
then the corresponding column vectors of A will form a basis for the
column space of A.

 In the previous example, the basis vectors obtained for the column
space of A consisted of column vectors of A, but the basis vectors
obtained for the row space of A were not all vectors of A.
 Transpose of the matrix can be used to solve this problem.

2008/12/3 Elementary Linear Algebra 25

Example (Basis for the Row Space of a
Matrix )
 Find a basis for the row space of  The column space of AT are
1 2 0 0 3 
  1  2  2 
2  5 3 2 6 2  5  6 
A        
0 5 15 10 0  c 1 0 , c 2 3 , and c 4 18
       
2 6 18 8 6  0  2  8 
consisting entirely of row vectors 
3  
6  
6 

from A.
 Thus, the basis vectors for the row
space of A are
 Solution:
r1 = [1 -2 0 0 3]
1 2 0 2  
1 2 0 2 
   r2 = [2 -5 -3 -2 6]
2 5 5 6  0
 1 5 10 
 r3 = [2 6 18 8 6]
AT 0 3 15 18 
0 0 0 1 
   
0 2 10 8  0
 0 0 0 

3 6 0 6   
0
 0 0 0 

2008/12/3 Elementary Linear Algebra 26

Example (Basis and Linear Combinations)

 (a) Find a subset of the vectors v1 = (1, -2, 0, 3), v2 = (2, -5, -3, 6), v3
= (0, 1, 3, 0), v4 = (2, -1, 4, -7), v5 = (5, -8, 1, 2) that forms a basis
for the space spanned by these vectors.
 (b) Express each vector not in the basis as a linear combination of
the basis vectors.

 Solution (a):
1 2 0 2 5  
1 0 2 0 1
  1 1 0 1
2 5 1 1 8 

0
 
0 3 3 4 1  
0 0 0 1 1
   
3 6 0 7 2  0
 0 0 0 0
         
v1 v2 v3 v4 v5 w1 w 2 w 3 w 4 w 5

 Thus, {v1, v2, v4} is a basis for the column space of the matrix.

2008/12/3 Elementary Linear Algebra 27

Example

 Solution (b):
 We can express w3 as a linear combination of w1 and w2, express
w5 as a linear combination of w1, w2, and w4 (Why?). By
inspection, these linear combination are
w3 = 2w1 –w2
w5 = w1 + w2 + w4
 We call these the dependency equations. The corresponding
relationships in the original vectors are
v3 = 2v1 –v2
v5 = v1 + v2 + v4

2008/12/3 Elementary Linear Algebra 28

Exercises

 Sec. 5.3: 3(c), 3(d), 4(b), 4(d), 11, 15 , 20(d) , 20(e) , 24

 Sec. 5.4: 3(d), 4(b), 4(d), 9, 14, 19, 21
 Sec. 5.5: 4, 6(c)(d), 8, 9, 12(c), 16, 18
 Sec. 5.6: 4, 7, 16, 17, 18, 19

2008/12/3 Elementary Linear Algebra 29

Lecture 20: 5.6
Rank and Nullity

Wei-Ta Chu

2008/12/3
Four Fundamental Matrix Spaces

 Consider a matrix A and its transpose AT together, then there are six
vector spaces of interest:
 row space of A, row space of AT
 column space of A, column space of AT
 null space of A, null space of AT
 However, the fundamental matrix spaces associated with A are
 row space of A, column space of A
 null space of A, null space of AT
 If A is an mn matrix, then the row space of A and nullspace of A
are subspaces of Rn and the column space of A and the nullspace of
AT are subspace of Rm
 What is the relationship between the dimensions of these four vector
spaces?

2008/12/3 Elementary Linear Algebra 31

Dimension and Rank

 Theorem 5.6.1
 If A is any matrix, then the row space and column space of A
have the same dimension.
 Proof: Let R be any row-echelon form of A. It follows from
Theorem 5.5.4 that
dim(row space of A) = dim(row space of R).
 It follows from Theorem 5.5.5b that
dim(column space of A) = dim(column space of R)
 The dimension of the row space of R is the number of nonzero
rows=numbe rofleading1’ s=di mensionoft hec ol
umn
space of R

2008/12/3 Elementary Linear Algebra 32

Rank and Nullity

 Definition
 The common dimension of the row and column space of a matrix
A is called the rank (秩) of A and is denoted by rank(A); the
dimension of the nullspace of a is called the nullity (零核維數)
of A and is denoted by nullity(A).

2008/12/3 Elementary Linear Algebra 33

Example (Rank and Nullity)

 Find the rank and nullity of the matrix

1 2 0 4 3 5
3 7 2 0 1 4
A  
2 5 2 4 6 1
 
4 9 2 4 4 7 
 Solution:
 The reduced row-echelon form of A is

1 0 4 28 37 13

0 1 2 12 16 5 
 

0 0 0 0 0 0
 
0
 0 0 0 0 0
 Sincet
he rearet wononz e
ror ows(twol e ading1’ s)
, ther ows
pac
eand
column space are both two-dimensional, so rank(A) = 2.

2008/12/3 Elementary Linear Algebra 34

Example (Rank and Nullity)
 To find the nullity of A, we must find the dimension of the
solution space of the linear system Ax=0.
 The corresponding system of equations will be
x1 –4x3 –28x4 –37x5 + 13x6 = 0
x2 –2x3 –12x4 –16 x5+ 5 x6 = 0
 It follows that the general solution of the system is
x1 = 4r + 28s + 37t –13u, x2 = 2r + 12s + 16t –5u,
x3 = r, x4 = s, x5 = t, x6 = u
or x1   4 28  37   13

x2   12  16   
  2     5 

x3  1 0  0  0  Thus, nullity(A) = 4.
    t  u  
r s
x4  
 0 1  0  0 

x5  0 0  1  0 
        
  
x6 
 
0 0 
  
0  
1 

2008/12/3 Elementary Linear Algebra 35