MATH 235: Inner Product Spaces,

SOLUTIONS to Assign. 7

Questions handed in: 3,4,5,6,9,10.

Contents
1 Orthogonal Basis for Inner Product Space 2

2 Inner-Product Function Space 2

3 Weighted Inner Product in R2 *** 2

3.1 SOLUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

4 Inner Product in S 3 *** 3

4.1 SOLUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

5 Inner Products in M22 and P2 *** 4

5.1 SOLUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

6 Cauchy-Schwarz and Triangle Inequalities *** 5

6.1 SOLUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

7 Length in Function Space 6

8 More on Function Space 7

9 Linear Transformations and Inner Products *** 8

9.1 SOLUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

10 MATLAB *** 9
10.1 SOLUTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
10.1.1 MATLAB Program for Solution . . . . . . . . . . . . . . . . . . . . . . . . . . 12
10.1.2 Output of MATLAB Program for Solution . . . . . . . . . . . . . . . . . . . 13

1
1 Orthogonal Basis for Inner Product Space
R1
If V = P3 with the inner product < f, g >= −1 f (x)g(x)dx, apply the Gram-Schmidt algorithm
to obtain an orthogonal basis from B = {1, x, x2 , x3 }.

2 Inner-Product Function Space

Consider the vector space C[0, 1] of all continuously differentiable functions
R1 defined on the closed
interval [0, 1]. The inner product in C[0, 1] is defined by < f, g >= 0 f (x)g(x)dx. Find the inner
products of the following pairs of functions and state whether they are orthogonal

1. f (x) = cos(2πx) and g(x) = sin(2πx)

2. f (x) = x and g(x) = ex

3. f (x) = x and g(x) = 3x

3 Weighted Inner Product in R2 ***

Show whether or not the following are valid inner products in R2 . If one is a valid inner product,

T
then find a nonzero vector that is orthogonal to the vector y = 2 1 .

1. hu, vi := 7u1 v1 + 1.2u2 v2 .

2. hu, vi := −7u1 v1 − 1.2u2 v2 .

3. hu, vi := 7u1 v1 − 1.2u2 v2 .

3.1 SOLUTIONS

BEGIN SOLUTION: Note that in each case, the inner product can be written as hu, vi =
uT Dv, for an appropriate diagonal matrix D. We see that hu, vi = uT Dv = (uT Dv)T = v T Du =
hv, ui. And, hαu, v + wi = α(uT D(v + w)) = α(hu, vi + hu, wi). Therefore, the first three properties
for an inner product all hold true.
 
7 0
1. For hu, vi := 7u1 v1 + 1.2u2 v2 , the diagonal matrix D = . Therefore, hu, ui :=
0 1.2
7u21 + 1.2u22 ≥ 0, with equality if and only if the vector u = 0, i.e. this is a valid innerproduct.
A vector orthogonal to the given
 y satisfies hu, yi := 7u1 (2) + 1.2u2 = 0, e.g. we could set
−1
u1 = −1 and get that u = 14 . (To understand the geometry, we could draw the ellipse
1.2
kuk = 1 and see which vectors are orthogonal to each other.)
 
−7 0
2. For hu, vi := −7u1 v1 − 1.2u2 v2 , the matrix D = . Therefore, hu, ui := −7u21 −
0 −1.2
1.2u22 < 0, if the vector u 6= 0, i.e. this is not a valid innerproduct.

2
  
7 0
3. For hu, vi := 7u1 v1 − 1.2u2 v2 , the matrix D = . Therefore, hu, ui := 7u21 − 1.2u22 <
0 −1.2
1.2u22
0, if the vector u 6= 0, and u21 < 7 , i.e. this is not a valid innerproduct.

END SOLUTION.

4 Inner Product in S 3 ***

Let S 3 denote the vector space of 3 × 3 symmetric matrices.

1. Prove that hS, T i = tr ST is an inner product on S 3 .

   
1 0 1 1 0 1
2. Let S1 = 0 1 0 and S2 = 0 −3 0. Apply the Gram-Schmidt process in S 3 to
1 0 0 1 0 0
find a matrix orthogonal to both S1 and S2 .

3. Find an orthonormal basis for S 3 using the above three matrices.

 
1 0 1
4. Let T = 0 1 −1. Find the projection of T on the span of {S1 , S2 }.
1 −1 1

4.1 SOLUTIONS

BEGIN SOLUTION:

1. (a) That tr ST = tr TS was proved in class already.

(b) tr S(T + V) = tr (ST + SV) = tr ST + tr SV, where the last equality follows straight from
the definition of taking the sum of the diagonal elements.
P
(c) tr αS = αtr S, again follows from the definition, i.e. both are equal to i (αSii ).
(d) hS, Si = tr SS = tr S2 = ij S2ij ≥ 0 and it is = 0 if an only if each Sij
2 = 0, i.e. if and
P
only if S = 0.

2. We first note that tr S1 S2 = 1 + 1 + 1 − 3 = 0, i.e. S1 ⊥ S2 . Therefore, we can apply one

step of the process. But we need a matrix that is linearly independent to both S1 , S2 . By
observation the matrix  
0 0 0
S3 = 0 0 0
0 0 1
 
−2d a d
works and in fact is orthogonal to both. Note that any matrix of the form  a 0 b
d b c
does the job. Also, note we could also choose a random matrix (linearly independent with

3
  
1 1 0
S1 , S2 ) and then apply one step of the process, e.g. choose R = 1 1 0. Then we can
0 0 0
find the third matrix by subtracting the projection onto the span of the first two, i.e.
hR,S1 i hR,S2 i
Ŝ3 = R − hS 1 ,S1 i
S1 − hS 2 ,S2 i
S2
2 −2
= R − 4 S1 − 12 S2
1 1
= R
 − 2 S1 +6 S2    
1 1 0 1 0 1 1 0 1
= 1 1 0 − 21 0 1 0 + 16 0 −3 0
0 0 0 1 0 0 1 0 0
2/3 1 −1/3
=  1 0 0 
−1/3 0 0

3. An orthonormal basis using S3 is

 
1 1
S1 , √ S2 , S3
2 2 3

Alternatively, using Ŝ3 , we get

( √ )
1 1 2 5
S1 , √ S2 , Ŝ3
2 2 3 3

4. The projection of T is
1 1 1 1
( tr TS1 )S1 + ( √ tr TS2 )S2 = ( 4)S1 + ( √ 0)S2 = 2S1 .
2 2 3 2 2 3

END SOLUTION.

5 Inner Products in M22 and P2 ***

Show which of the following are valid inner products.
   
a1 a2 b1 b2
1. Let A = and B = be real matrices.
a3 a4 b3 b4

(a) Define hA, Bi := a1 b1 + 2a2 b3 + 3a3 b2 + a4 b4 .

(b) Define hA, Bi := tr BT A.

2. Let p(x) and q(x) be polynomials in P2 .

(a) Define hp, qi := p(−1)q(−1) + p( 12 )q( 21 ) + p(−3)q(−3).

(b) Define hp, qi := p(−3)q(−3) + p( 21 )q( 21 ) + p(−1)q(−1).

4
5.1 SOLUTIONS

BEGIN SOLUTION:
 
0 1
1. (a) Note that hA, Bi := a1 b1 + 2a2 b3 + 3a3 b2 + a4 b4 = tr AB. Let A = Then
0 0
 
0 0
hA, Ai = tr AA = tr = 0, i.e. kAk = 0, but A 6= 0. Therefore, this is not a
0 0
valid inner product.
 
A(:, 1)
(b) For any m × n matrix A, define vec (A) = , i.e. vec (A) is the vector formed
. . . A(:, n)
from A using the columns of A. Note that tr BT A = ij Bij Aij = vec (B)T vec (A). We
P
can now show that this is a valid inner product in the usual way that the standard dot
product is an inner product.

2. (a) This follows as does the Example 2 in the text on page 429.
(b) Changing the order of the points does not change the verification of any of the rules for
verifying this is an inner product.

END SOLUTION.

6 Cauchy-Schwarz and Triangle Inequalities ***

 
3 0 0
1. Let D = 0 π 0. Consider the inner product defined on R3 by: hu, vi = uT Dv. Let
0 0 2
   
−2 −4
u =  3  and v =  8 . Verify that both inequalities hold.
−7 9
2. Find the projection of v on the span of u and call it w. Verify that equality holds for the
Cauchy-Schwarz inequality applied to u, w.

6.1 SOLUTIONS

BEGIN SOLUTION: Following are the MATLAB lines and the output for the solution.

D=sym(diag([3 pi 2]));
u=[-2 3 -7]’;
v=[-4 8 9]’;
lhs=abs(double(u’*D*v));
rhs= sqrt(double(u’*D*u))*sqrt(double(v’*D*v))
rhs =
238.4100
disp(’verify C.S. inequality, i.e. that rhs - lhs is nonnegative’)
verify C.S. inequality, i.e. that rhs - lhs is nonnegative

5
rhs-lhs
ans =
211.8082

lhs= (double((u+v)’*D*(u+v)));
rhs= (double((u)’*D*(u))) + (double((v)’*D*(v)));
disp(’verify triangle inequality, i.e. that rhs - lhs is nonnegative’)
verify triangle inequality, i.e. that rhs - lhs is nonnegative
rhs-lhs
ans =
53.2036

disp(’find w, the projection on the span of u’)

find w, the projection on the span of u
w= (double(v’*D*u))/(double(u’*D*u))*u
w =
0.3848
-0.5772
1.3467

lhs=abs(double(u’*D*w));
rhs= sqrt(double(u’*D*u))*sqrt(double(w’*D*w))
rhs =
26.6018
disp(’verify that rhs - lhs is now zero, i.e. C.S. inequality holds with equality’)
verify that rhs - lhs is now zero, i.e. C.S. inequality holds with equality
rhs-lhs
ans =
0

echo off

END SOLUTION.

7 Length in Function Space

Let W = Span{1,
Rπ cos(x), cos(2x)} ∈ C[−π, π] with inner product
< f, g >= −π f (x)g(x)dx. It is known that { √12π , √1π cos(x), √1π cos(2x)} is an orthonormal basis
Rπ
for W . A function f ∈ W satisfies −π f (x)dx = a,
Rπ Rπ
−π f (x) cos(x)dx = b and −π f (x) cos(2x)dx = c. Find an expression for ||f || in terms of a, b and
c.

6
8 More on Function Space
Consider the vector space P2 consisting of polynomials of degree at most 2 together with the inner
product Z 1
< f, g >= f (x)g(x)dx , f, g ∈ P2 .
0
Let W be the subspace of P2 having a basis {1, x}.

1. Find an orthonormal basis for W .

2. Let h(x) = x2 + 1. Find the vector in W that is closest to h(x).

7
9 Linear Transformations and Inner Products ***
Let V be a real inner product space (with the usual notation) and let {v1 , v2 , · · · , vn } be an
orthonormal basis for V (so V is finite dimensional). Fix a linear transformation T : V → R.
Define v0 = T (v1 )v1 + T (v2 )v2 + · · · + T (vn )vn .

1. Show that for all v ∈ V, T (v) =< v0 , v >.

2. Prove that, for vectors u0 , w0 ∈ V , if < u0 , v >=< w0 , v > for all v ∈ V , then u0 = w0 .

3. Deduce from Item 2 that v0 is the only vector in V with the displayed property in Item 1.
NOTE: This question shows how a suitable inner product produces a 1 − 1 correspondence
between vectors in the vector space and linear functionals T from it to the reals. This is used
to identify tangent vectors with what are called cotangent vectors in differential geometry,
general relativity and in other symbol manipulations of mathematical physics, where it is
sometimes referred to as “moving indices up-and-down”. A slightly surprising consequence
of Item 3 is that the vector v0 , despite appearances, depends only on the inner product and
T , but not on the choice of orthonormal basis.

9.1 SOLUTIONS

BEGIN SOLUTION:

1. Since S := {v1 , v2 , · · · , vn } is an orthonormal basis, we get that

v = hv, v1 iv1 + hv, v2 iv2 + . . . , hv, vn ivn , ∀v ∈ V. (1)

Therefore, for all v ∈ V ,

hv0 , vi = hT (v1 )v1 + T (v2 )v2 + · · · + T (vn )vn , vi

= T (v1 )hv1 , vi + T (v2 )hv2 , vi + · · · + T (vn )hvn , vi

and also
T (v) = T (hv, v1 iv1 + hv, v2 iv2 + . . . , hv, vn ivn )
.
= hv, v1 iT (v1 ) + hv, v2 iT (v2 ) + . . . , hv, vn iT (vn )
The result now follows since hu, wi = hw, ui, ∀u, w ∈ V , a real inner product space, and
T (u) ∈ R, ∀u ∈ V .

2. Suppose that hu0 , vi = hw0 , vi for all v ∈ V . Then, in particular, this is true with v replaced
by elements of the orthonormal basis vi , i = 1, 2, . . . , n. Therefore, the expansion in (1) (with
v replace by u0 or by w0 ) is the same. This implies that u0 = w0 .

3. Suppose that u0 ∈ V and for all v ∈ V, T (v) = hf u0 , vi. From Item 1, this means that for
all v ∈ V, T (v) = hu0 , vi = hv0 , vi, i.e. u0 = v0 is the only such vector.

END SOLUTION.

8
10 MATLAB ***
Gram-Schmidt & QR Factorization
Overview of the Gram-Schmidt Process
Given a basis {x1 , . . . , xp } for a subspace W of Rn , define

v1 = x1
x2 · v1
v2 = x2 − v1
v1 · v1
x3 · v1 x3 · v2
v3 = x3 − v1 − v2
v1 · v1 v2 · v2
..
.
xp · v1 xp · v2 xp · vp−1
vp = xp − v1 − v2 − · · · vp−1
v1 · v1 v2 · v2 vp−1 · vp−1

Then {v1 , . . . , vp } is an orthogonal basis for W .

When working on a computer, a basis is often presented as a matrix A with linearly independent
columns. The columns of matrix A are a basis for its column space (or range). The Gram-Schmidt
process can be applied to the column vectors of matrix A so that the output of the process is an
orthogonal set of vectors.
For example, let
     
1 0 0
 1   1   0 
x1 = 
 1  , x2 =  1  , x3 =  1 
    

1 1 1
Then, by inspection, {x1 , x2 , x3 } are linearly independent and thus form a basis for the subspace
W of R4 . Consider the following matrix A formed by using the basis {x1 , x2 , x3 } as column vectors:
 
1 0 0
 1 1 0 
A=  1 1 1 


1 1 1

Matrix A will be used to illustrate the Gram-Schmidt process in MATLAB.

First, enter matrix A.
>> A = [1 0 0; 1 1 0; 1 1 1; 1 1 1] %Create matrix A
Next, calculate the orthogonal basis {v1, v2, v3} by using the Gram-Schmidt algorithm stated
above. Recall that in MATLAB, the notation A(:,1) selects the first column of A, A(:,2) selects
the second column, and so on... Also recall that v1’ is the short form for transpose(A) (changes
a row vector to a column vector, or vice versa).
>> v1 = A(:,1) %Calculate v1
>> v2 = A(:,2) - (A(:,2)’*v1)/(v1’*v1)*v1 %Calculate v2
>> v3 = A(:,3) - (A(:,3)’*v1)/(v1’*v1)*v1 - (A(:,3)’*v2)/(v2’*v2)*v2
%Calculate v3

9
 MATLAB tells us that by using the Gram-Schmidt process on matrix A, the corresponding
orthogonal basis is      
1 −0.75 0
 1   0.25   −0.6667 
 1  , v2 =  0.25  , v3 =  0.3333 
v1 =      

1 0.25 0.3333
To convert from an Orthogonal basis to an Orthonormal Basis:
We must simply divide each orthogonal vector by its norm:
>> vn1 = v1/norm(v1) %Calculate normalized v1
>> vn2 = v2/norm(v2) %Calculate normalized v2
>> vn3 = v3/norm(v3) %Calculate normalized v3
The following theorem provides a method for checking whether the columns of a matrix are
orthonormal:

An m × n matrix U has orthonormal columns if and only if

UT U = I

Using this theorem and MATLAB, we can check whether {vn1, vn2, vn3} forms an orthonormal
basis. First define a new matrix U that uses {vn1, vn2, vn3} as its column vectors.
>> U = [vn1 vn2 vn3] %Create matrix U
>> transpose(U) * U %Apply theorem
Indeed, MATLAB returns the identity matrix I, so by the theorem, {vn1, vn2, vn3} form an
orthonormal basis.

10
 QR Factorization
QR factorization is the matrix version of the Gram-Schmidt method. QR factorization is
particularly useful because computer algorithms exist that implement this factorization method
with good control over computer round-off error. Additionally, there is no simple method for
computing eigenvalues of matrices that are larger than 3 × 3 since numerically it is difficult to
calculate the roots of the characteristic polynomial. Using QR factorization iteratively is the most
successful algorithm that exists today for computing eigenvalues. For these reasons, QR is an
important factorization technique in Linear Algebra.
Overview of QR Factorization
If A is an m × n matrix with linearly independent columns, then A can be factored as

A = QR

where Q is an m × n matrix with orthonormal columns consisting of an orthonormal basis of the

column space, and R is an invertible, upper triangular matrix.
To illustrate this method in MATLAB, consider the following matrix A:
 
1 0 0
 1 1 0 
A=  1 1 1 


1 1 1
This is the same matrix that was used in the section on the Gram-Schmidt process. We have
already used MATLAB to calculate the orthonormal basis for matrix A as:
     
0.5000 −0.8660 0
 0.5000   0.2887   −0.8165 
vn1 = 
 0.5000  , vn2 =  0.2887  , vn3 =  0.4082 
    

0.5000 0.2887 0.4082

As such, according to the description of the QR factorization method, it must be that we find:
 
0.5000 −0.8660 0
 0.5000 0.2887 −0.8165 
Q=  0.5000 0.2887

0.4082 
0.5000 0.2887 0.4082
Let’s use MATLAB to verify this result. MATLAB’s command for QR Factorization is

[Q R] = qr(A)

where A is an m × n matrix, Q is returned as an m × m orthogonal matrix and R is returned as

an m × n upper triangular matrix.
This MATLAB command for QR factorization is very general since it applies to any matrix
and it returns more information than required in some cases.
If A has rank n, then only the first n columns of Q will be an orthonormal basis for the column
space of A.
Enter the following in MATLAB:
>> A = [1 0 0; 1 1 0; 1 1 1; 1 1 1] %Create matrix A
>> rank(A) %Compute rank of matrix A
Since A is a 4 × 3 matrix and its rank is 3, we should expect that the orthonormal basis is the
first 3 columns of Q in result of the following command:

11
 >> [Q R] = qr(A) %Calculate the QR factorization of A
Indeed, the first 3 columns of Q match those of the orthonormal basis that we calculated using
the Gram-Schmidt process.
Notice that the MATLAB qr command is much easier to use than the Gram-Schmidt process.
The qr command in MATLAB is also less computationally demanding on your computer (more
efficient and not prone to rounding errors). The Gram-Schmidt method is numerically unstable
when implemented on a computer–the vectors that are returned are often not quite orthogonal
because of rounding errors in the calculation. Therefore, the QR factorization method is usually
prefered when using a computer.
Consider the following matrix A:
 
4 −5 1
 1 2 1 
A=  −2 5 2 


4 −8 8
Use MATLAB to determine if the columns of A are linearly independent? State which MATLAB
commands you used and explain your answer.
Using MATLAB, find the QR factorization of matrix A (i.e. find Q and R such that A = QR).
What is an orthonormal basis for the column space of matrix A?
Given any m × n matrix B that has already been entered in MATLAB, what single command
could you use to determine if matrix B has orthonormal columns?

10.1 SOLUTIONS

BEGIN SOLUTION: Following is a MATLAB program and the output:

10.1.1 MATLAB Program for Solution

diary off
echo off
!rm temp.txt
diary temp.txt
format compact
echo on

A=[ 4 -5 1; 1 2 1; -2 5 2; 4 -8 8]
disp(’first check if the columns are linearly independent - using RREF’)
rrefA=rref(A)
disp(’add the number of nonzero columns and display the rank’)
rankA=sum(sum(rrefA));
disp([’The rank is ’,num2str(rankA)]);
if rankA==3,
echo off
disp(’The rank is 3 and the columns are linearly independent’)
echo on
end
disp(’now find the qr factorization’)

12
[Q,R]=qr(A)
disp(’since the rank is 3, the first 3 columns of Q form an orthonormal basis for the column sp

m=5;n=3;
B=randn(m,n)
disp(’to determine if B has orthonormal columns we can check: B’’B=I?’);
B’*B
disp(’so B does not have orthonormal columns’)
[Q,R]=qr(B,0); % use the economy version of qr
Q
disp(’check if Q has orthonormal columns’)
Q’*Q
disp(’so Q does have orthonormal columns’)

echo off
diary off

10.1.2 Output of MATLAB Program for Solution

The output now follows.

A=[ 4 -5 1; 1 2 1; -2 5 2; 4 -8 8]
A =
4 -5 1
1 2 1
-2 5 2
4 -8 8
disp(’first check if the columns are linearly independent - using RREF’)
first check if the columns are linearly independent - using RREF
rrefA=rref(A)
rrefA =
1 0 0
0 1 0
0 0 1
0 0 0
disp(’add the number of nonzero columns and display the rank’)
add the number of nonzero columns and display the rank
rankA=sum(sum(rrefA));
disp([’The rank is ’,num2str(rankA)]);
The rank is 3
if rankA==3,
echo off
The rank is 3 and the columns are linearly independent
end
disp(’now find the qr factorization’)
now find the qr factorization
[Q,R]=qr(A)

13
Q =
-0.6576 -0.3267 0.3665 -0.5714
-0.1644 -0.7960 -0.1136 0.5714
0.3288 -0.3861 -0.6452 -0.5714
-0.6576 0.3326 -0.6607 0.1429
R =
-6.0828 9.8639 -5.4252
0 -4.5500 0.7663
0 0 -6.3230
0 0 0
disp(’since the rank is 3, the first 3 columns of Q form an orthonormal basis for the column sp
since the rank is 3, the first 3 columns of Q form an orthonormal basis for the column space of

m=5;n=3;
B=randn(m,n)
B =
-0.3999 0.6686 -1.6041
0.6900 1.1908 0.2573
0.8156 -1.2025 -1.0565
0.7119 -0.0198 1.4151
1.2902 -0.1567 -0.8051
disp(’to determine if B has orthonormal columns we can check: B’’B=I?’);
to determine if B has orthonormal columns we can check: B’B=I?
B’*B
ans =
3.4728 -0.6427 -0.0740
-0.6427 3.3360 0.6024
-0.0740 0.6024 6.4062
disp(’so B does not have orthonormal columns’)
so B does not have orthonormal columns
[Q,R]=qr(B,0); % use the economy version of qr
Q
Q =
-0.2146 -0.3315 -0.6860
0.3703 -0.7351 0.0122
0.4377 0.5863 -0.3374
0.3820 -0.0624 0.5618
0.6924 -0.0458 -0.3159
disp(’check if Q has orthonormal columns’)
check if Q has orthonormal columns
Q’*Q
ans =
1.0000 -0.0000 -0.0000
-0.0000 1.0000 0.0000
-0.0000 0.0000 1.0000
disp(’so Q does have orthonormal columns’)
so Q does have orthonormal columns

14
echo off

END SOLUTION.

15