Hilbert Spaces

Joseph Muscat
2014-5-27
(A revised and expanded version of these notes are now published by
Springer.)

Inner Product Spaces

1.1

Introduction

Definition

An inner-product on a vector space X is a map

h , i:X X C

such that
hx, y + zi = hx, yi + hx, zi,
hx, yi = hx, yi,
hy, xi = hx, yi,
hx, xi > 0;

hx, xi = 0 x = 0.

In words, it is said to be a positive-definite sesquilinear form.

PN
The simplest examples are RN and CN with hx, yi =
n yn ; the
n=1 x
square
PN matrices of size N N also have an inner-product given by hA, Bi =

i,j=1 Aij Bij .

Proposition 1.1

1. hx, xi is real (and positive) and is denoted by kxk2

2. hx, yi = 0 x y = 0 (put x = y)
yi (anti-linear); hx + y, zi = hx, zi + hy, zi;
3. hx, yi = hx,
1

1.1

Introduction

J Muscat

4. kxk = ||kxk.
Definition Two vectors x, y are orthogonal when hx, yi = 0, written
as x y. The angle between two vectors is given by cos = hx, yi/kxk kyk.
Proposition 1.2
1. kx + yk2 = kxk2 + 2Rehx, yi + kyk2 .
2. (Pythagoras) If z = x + y and hx, yi = 0 then kzk2 = kxk2 + kyk2.
3. For any orthogonal vectors x, y, kxk 6 kx + yk.
4. For any non-zero vectors x, y, there is a unique vector z and a
unique scalar such that x = z + y and z y.
Proof. The first three statements follow immediately from the axioms
and the first proposition. For the last statement, let z = x y as required;
then z y holds = hy, xi/hy, yi (check).

Proposition 1.3

Cauchy-Schwarz
|hx, yi| 6 kxkkyk

Proof. Decompose x into orthogonal parts x = (x y) + y where =

hy,xi
hy,yi

(assuming y 6= 0); now use Pythagoras theorem, and deduce that |y| 6 kxk.

Note that Pythagoras theorem and Cauchy-Schwarzs inequality are still
valid even if the inner-product is not positive definite but just semi-definite,
as long as kyk =
6 0.
Corollary
kx + yk 6 kxk + kyk
The proof is simply an application of the Cauchy-Schwarz inequality to
the expansion of kx + yk2.


1.1

Introduction

J Muscat

Hence kxk is a norm, and all the facts about normed vector spaces apply
to inner-product spaces. In particular they are metric spaces with distance
d(x, y) = kx yk, convergence of sequences makes sense as xn x
kxn xk 0, continuity and dual spaces also make sense. Inner-product
spaces are special normed spaces which not only have a concept of length
but also of angle.
Definition A Hilbert space is an inner-product space which is complete as a metric space.
Proposition 1.4
subspace.

The closure of a linear subspace remains a linear

Proof. If xn , yn A, then limn xn +limn yn = limn (xn +yn ) A.


Recall that finite dimensional subspaces are always closed. (exercise)
From the theory of Banach spaces we know that addition and scalar
multiplication are continuous operations.
Proposition 1.5

+, , h, i are continuous in each variable.

Proof. If kx x k < and ky y k < then

k(x + y) (x + y )k 6 kx x k + ky y k < 2.
Similarly,
k(x x )k = ||kx x k < ||.

We can consider y := hy, i to be a functional mapping x hy, xi.

This functional is continuous because |y (x)| = |hy, xi| 6 kykkxk. In fact
ky k = kyk.

It follows that taking limits commutes with the inner-product:
hx, lim yn i = lim hx, yn i.
n

Do all norms on vector spaces come from inner-products, and if not, which
normed vector spaces are in fact inner-product spaces? The answer is given
by

1.1

Introduction

J Muscat

Proposition 1.6 (Parallelogram law)

A norm comes from an inner-product if, and only if, it satisfies
kx + yk2 + kx yk2 = 2(kxk2 + kyk2)
Proof. Expanding and adding kx + yk2 and kx yk2 gives the parallelogram law. Conversely, subtracting the two gives 4 Re hx, yi. Hence, by
noticing that Im hx, yi = Re ihx, yi = Re hix, yi, we get the polarization
identity
hx, yi =

1
ky + xk2 ky xk2 + iky + ixk2 iky ixk2 .
4

The converse is true but harder to prove.


This law can be generalized further. If N = 1 (N > 2),
=
R then hx, yi
P
N
1
n
n
2
2
n=1 ky + xk . Even more generally, hx, yi = 2i S 1 ky + zxk dz.
That is, the inner product hx, yi is a complex average of lengths on a ball
of radius kxk, centred at y.
1
N

Proposition
1.7
R

f g.

L2 (R) is a Hilbert space with inner-product hf, gi =

Proof. We assume L2 (R) is a complete normed space. Verifying the

parallelogram law,
Z
Z
Z
2
2
|f + g| + |f g| = (|f |2 + fg + f g + |g|2 + |f |2 fg f g + |g|2.
Hence find the associated inner-product from the previous proposition.


1.1.1

Exercises

1. Find the angle between sin and cos in the space L2 [, ].

P
2. Show that 2 is a Hilbert space with inner-product hx, yi =
n y n .
n=1 x
1

Show further that l and l are not inner-product spaces by finding

two sequences which do not satisfy the parallelogram law.
3. Similarly show that L and L1 are not inner-product spaces.

1.1

Introduction

J Muscat

4. Show that the Cauchy-Schwarz inequality becomes an equality if, and

only if, x = y .
5. Let d = inf kx + yk; show that
|hx, yi|2 6 (kxk2 d2 )kyk2 .
6. Show that any finite-dimensional subspace of a Hilbert space is closed.
Deduce that the set of polynomials of degree at most m forms a closed
linear subspace of L2 [a, b] with dimension m + 1; find a basis for this
space.
7. Show that the set of continuous functions C[a, b] is a non-closed linear
subspace of L2 [a, b].
8. Starting from a norm k.k that satisfies the parallelogram law, we can
create an inner-product
using Prop. 6, from which we can define a
p
norm k.k1 := h., .i. Show that the two norms are identical.
9. Show that two inner-products, on the same vector space, are conformal
i.e. give the same angles between vectors, if, and only if, hx, yi1 =
hx, yi2 for some positive real .

10. We have shown that an inner-product is determined by a norm that

satisfies the parallelogram law. More generally, show that any bilinear
form A(x, y) that satisfies A(y, x) = A(x, y), is determined by its real
quadratic form q(x) = A(x, x), as A(x, y) = 14 (q(x + y) + q(x y) +
iq(x + iy) iq(x iy)).
11. Show that the real and imaginary parts of an inner-product give symmetric/antisymmetric real-valued inner-products. Thus a complex inner-product
has more structure than a real (symmetric) inner-product e.g. C2 has
more structure than R4 .
12. Let H1 H2 be the product of two inner-product spaces. Show that




h x1 , x2 , y1 , y2 i := hx1 , y1iH1 + hx2 , y2 iH2

defines an inner-product on H1 H2 .
R
13. Show that the formula hf, gi := f (x)g(x)w(x) dx, where w(x) is a
positive real function, defines an inner-product. The resulting space of
functions is called a weighted L2 space.

1.2

Orthogonality

J Muscat

14. * Consider the vectorR space of holomorphic functions f : R R+ C

which satisfy supy>0 |f |2 dx < . Show that the formula hf, gi :=
R
lim0+ f (x + i)g(x + i) dx gives an inner-product.

1.2

Orthogonality

Definition

The orthogonal space of a set A is the set

A = { x : hy, xi = 0 y A }.

1.2.1

Exercises

1. Show that 0 = X, X = 0.
2. Show that if { x } = X then x = 0.
3. Show that if { x } = 0 then X is one-dimensional.
Proposition 1.8

A is a closed linear subspace.

Proof. That A is a linear subspace follows from the linearity of the innerproduct. Let x A . That is, there is a sequence of vectors xn A such
that xn x. Now, for any y A, hx, yi = h limn xn , yi = limn hxn , yi = 0.
Hence x A .

Proposition 1.9
1. A A 0;
2. A B B is a closed subspace of A ;
3. A A .
Proof. (i) is left as an easy exercise. For (ii), let x B i.e. hb, xi =
0 b B. In particular this is true for b A so that x A . For (iii), let
x A, then hy, xi = 0, y A . Hence hx, yi = 0 y A . Hence x A .

Note that A is always a closed linear subspace even if A isnt. Question:
if M is a closed linear subspace is it necessarily true that M = M ?

1.3

Least Distance in Hilbert spaces

1.3

J Muscat

Least Distance in Hilbert spaces

Definition
its points:

A convex set is a set A that includes all line segments between

x, y A, t [0, 1],

tx + (1 t)y A.

Exercise: Show that linear subspaces and balls are convex.

Theorem 1.10
If M is a closed convex set and x is any point of
a Hilbert space H, then there is a unique point x0 M which is
closest to x.
x H !x0 M :

d(x, x0 ) 6 d(x, y) y M.

Proof. Let d be the smallest distance from M to x i.e. d = inf yM d(x, y)

(how do we know that d exists?) Then there is a sequence of vectors yn M
such that kx yn k = d(x, yn ) d. Now, by the parallelogram law,
kyn ym k2 = 2kyn xk2 + 2kym xk2 k(yn + ym ) 2xk2
6 2kyn xk2 + 2kym xk2 4d2
0
as n . Hence (yn ) is a Cauchy sequence. But M is closed (hence
complete) and so yn x0 M. It follows, by continuity of the norm, that
kx x0 k = d.
Suppose y M is another closest point to x, i.e. d(x, y) = d. Then
kx0 yk2 = 2kx0 xk2 + 2ky xk2 k(x0 + y) 2xk2
6 2kx0 xk2 + 2ky xk2 4d2
=0

Note that this theorem, which does not refer to inner-products, is not
true in Banach spaces in general. For example l has a closed subspace c0 ,
but there is no closest sequence in c0 to the sequence (1, 1, 1, . . .). In fact,
the smallest distance is d = 1 and this is achieved by any bounded sequence
(an ) with 0 6 an 6 2.
Exercise: prove these assertions.

1.3

Least Distance in Hilbert spaces

J Muscat

Let us concentrate on the case when M is a closed subspace of H.

Theorem 1.11
For a closed linear subspace M of a Hilbert space
H, x0 M is the closest point to x H if, and only if, x x0 M .

Proof. Let a be any point of M. Then x x0 = a + z with a z, where

= ha, x x0 i/kak2 and z = x (x0 + a). By Pythagoras theorem, we get
kzk 6 kx x0 k making x0 + a even closer to x than the closest point x0 .
This is only possible if = 0 i.e. ha, x x0 i = 0. Since a is arbitrary we get
x x0 M.
Conversely, if x x0 a for any a M, then by Pythagoras theorem,
kx ak2 = kx x0 k2 + kx0 ak2 , so that kx x0 k 6 kx ak, making x0
the closest point to x.

Corollary
If M is a closed linear subspace of a Hilbert space
H, then H = M M .

Proof. Write x = x0 + (x x0 ). These two components are in M and

M as proved previously. Moreover M M = 0.


Note that this is false for Banach spaces e.g. the space l 6= c0 M for
any linear subspace M.
Corollary
If M is a closed linear subspace of a Hilbert space
H, then M = M. More generally, for any set A, A = [[A]].
Proof. Let x M . Then x = a + b where a M and b M .
Then 0 = hb, xi = hb, ai + hb, bi = kbk2 , making b = 0 and x M. For the
second part, note that [[A]] is the smallest closed linear subspace containing A.

Therefore, since A [[A]], we get [[A]] A and hence A [[A]] = [[A]].


[[A]] is dense in H if, and only if, A = 0.
= H. Then A = A = H = 0, since A is a
Proof. Suppose [[A]]
= A = 0 = H.
closed linear subspace. Conversely, [[A]]

Corollary

1.3
1.3.1

Least Distance in Hilbert spaces

J Muscat

Exercises

1. Find the closest points x0 to a closed linear subspace M from (i) a

point x M, (ii) a point x M .
2. In the decomposition x = a + b with a M and b M , show that a
and b are unique.
3. Deduce that if H = M N where M is a closed linear subspace and
M N then N = M .
4. Show that, if M is a closed linear subspace of N, then M (M N) =
N.
1.3.2

Projections

Let us take a closer look at the map from x to x0 M.

Definition A projection is a linear map P : H H such that

P 2 = P . An orthogonal projection is one such that ker P imP .

Theorem 1.12 If M is a closed linear subspace of a Hilbert space

H then the map P : H H defined by P x = x0 is a continuous
orthogonal projection with imP = M and ker P = M .
Proof. By the definition of P , for any a H, (a P a) M and this
property defines P a M.
P is linear since (x + y) (P x + P y) = (x P x) + (y P y) M and
P x + P y M, hence P (x + y) = P x + P y. Similarly P (x) = P x.
P is a projection since P x M implies that P 2 x = P x.
P is onto M since for any x M, P x = x. Moreover x ker P P x =
0 x = x P x M .
P is continuous since kxk2 = kx P xk2 + kP xk2 by Pythagoras theorem
so that kP xk 6 kxk.


1.3.3

Exercises

1. Show that the map (x, y, z) 7 (x, 0, 0) is an orthogonal projection in

R3 . Find a projection which is not orthogonal.
2. Let P, Q be two commuting projections. Show that P Q is also a projection.

1.3

Least Distance in Hilbert spaces

J Muscat

10

3. Show that if P is a projection then H = ker P imP .

4. Show that any orthogonal projection is continuous, and imP ker P .
Hence note that there is a 1-1 correspondence between closed linear
subspaces and orthogonal projections.
5. Show that if P is an orthogonal projection onto M then I P is an
orthogonal projection onto M .
6. Suppose that H = M N. Show that, if P and Q denote the orthogonal
projections onto M and N respectively, then I = P + Q and P Q = 0.
Extend this by induction to the case when H = M1 . . . Mn to get
I = P1 + . . . Pn with Pi Pj = Pi ij .
7. Consider two closed linear subspaces M1 and M2 (with associated projections P1 and P2 ). Show that the iteration xn+1 = P2 P1 xn starting
from x0 = y converges to the closest point y0 M1 M2 .
8. Let M =< y > (a closed linear subspace). Find the orthogonal projection P which maps any point x to its closest point in M. (Ans:
P x = hy, xiy)
P
9. Let M = hy1 , . . . , yN i. Find P again. (Ans: P x =
i i yi where
2
i = hyi , xi/kyi k .
1.3.4

Examples: Least Squares Approximation

1. Find the closest point in the plane 2x + y 3z = 0 to the point x =

(5, 2, 0). Letting n = (2, 1, 3), the equation of the plane is given by
n x = 0. Therefore we are looking for a point x0 in the plane such
that x x0 is perpendicular to the plane i.e. is parallel to n. Hence,
x0 = x + n such that it satisfies n x0 = 0. Substituting gives .
2. Find the best-fitting (closest in the space L2 ) cubic polynomial to the
function sin in the region [0, 2].
The space of cubic polynomials, a+bx+cx2 +dx3 , is a four-dimensional
closed linear subspace of the Hilbert space L2 [0, 2], with basis 1, x, x2 , x3 .
It must therefore have an element p(x) = a + bx + cx2 + dx3 which is
closest to sin x. In fact, p is characterized by the condition p sin
q, q M. In particular we get four equations hp(x) sin(x), xi i = 0
for i = 0, . . . , 3, or, equivalently, hp(x), xi i = h sin(x), xi i. The righthand sides can be worked out, giving four linear equations in the four
unknowns a, b, c, d, which can be extracted by solving.

1.3

Least Distance in Hilbert spaces

J Muscat

11

3. Finding the best-fitting (least-squares) line to a number N of points

(an , bn ) R2 .
One can consider such a sequence of N pairs to be a single point b =
(b1 , . . . , bN ) in RN which is a Hilbert space. We wish to find a line
y = mx + c for which y = (yn ) where yn = man + c) are collectively as
close to bn as possible. These lines form a closed subspace M spanned
by the two vectors e1 = (a1 , . . . , aN ) and e2 = (1, . . . , 1) (check). Hence
the condition hy b, Mi = 0 becomes two equations hy b, ei i = 0,
which translated becomes,
X
X
X
m
an + c
1=
bn ,
n

X
n

a2n + c

X
n

an =

bn an .

Solving for m and c gives the usual regression line as used in statistics.
1.3.5

Exercises

1. Find that quadratic polynomial that is closest to sin x in L2 [0, 2].

2. Find that combination of sin and cos which is closest to 1 x3 in
L2 [0, 1].
3. Implement the above examples as Mathematica programs.
4. Find, and implement, the best-fitting circle to a number N of points
in R2 .

2 Orthonormal Bases

J Muscat

12

Orthonormal Bases

Definition A Hilbert basis, also called a total set of vectors, is a set of

vectors {en } such that
hen i is dense in X,
hen , em i = nm .

Note that the first condition is equivalent to hen , xi = 0 n x = 0.

Note also that the set of basis vectors need not be countable; when it is,
the Hilbert space is called separable. From now on we will assume that the
Hilbert basis is countable.
2.0.6

Example

The space of sequences 2 has a Hilbert basis consisting of en = (0, . . . , 0, 1, 0, . . .).

To prove the first condition, let x = (x1 , x2 , . . .) be a sequence in 2 which
is orthogonal to all the ei . Then for any i, xi = hei , xi = 0; hence x = 0.
2.0.7

Gram-Schmidt orthogonalization

Of the two properties, it is the first one that is crucial; if the span of a
countable number of vectors {an } is dense in X but not orthonormal, then
they can be made so using the usual Gram-Schmidt process:

2.1

b0 := a0 ,
P
bn := an n1
i=0 hei , an i,

e0 := b0 /kb0 k
en := bn /kbn k.

Fourier Expansion

Proposition 2.1 Let M be a closed linear subspace M, with a countable Hilbert basis en ;
2

(an )
Proof. Let xN :=

PN

n=1

an en converges in M.

n=1

an en . Then, assuming N > M wolog,

kxN xM k2 = hxN xM , xN xM i
P
P
=h N
an en , N
n=M
+1
m=M +1 am emi
PN
= n,m=M +1 an am hen , em i
P
2
= N
n=M +1 |an | .

2.1

Fourier Expansion

J Muscat

13

Suppose that (an ) 2 ; then this last sum converges to 0, implying that (xn )
is a Cauchy sequence in M, which must therefore converge to a point x M.
Conversely, suppose
that the series xN converges; then kxN xM k 0,
PN
implying that n=1 |an |2 is a Cauchy sequence in C, and hence must converge
as N .

Theorem 2.2 (Bessels inequality)
If a closed linear subspace M of a Hilbert space, has a countable
Hilbert basis en , then
X
(i)
|hen , xi|2 6 kxk2 ,
n

(ii)

P x = x0 =

X
n

Proof. Let xN :=
we get

hence

PN

n=1

hen , xien .

hen , xien . Then xN M and, letting an = hen , xi,

0 6 kx xN k2 = hx xN , x xN i
= kxk2 hxN , xi hx, xN i + hxN , xN i
P
P
an an + N
= kxk2 2 N
n,m=1 an am hen , em i
n=1
PN
2
2
= kxk n=1 |an | ,
N
X
n=1

|hen , xi|2 6 kxk2 ,

is an increasing series, bounded above by kxk2 . Hence the sum on the lefthand side must converge, proving the Bessel inequality. P
N
By the previous proposition, the series of vectors
n=1 hen , xien converges, say to y M.
Moreover,
P
m hem , x yi = hem , xi
n=1 hen , xihem , en i = 0,
so that x y { em } = M . Hence y must be the closest point in M to x.

Corollary
(Parsevals identity)
If en is a countable Hilbert basis for X, then

X
x X
x=
hen , xien ,
n=1

2.1

Fourier Expansion

J Muscat

and
hx, yi =
where x =

n en and y =

14

n n .

n en . In particular
X
kxk = (
|n |2 )1/2 .
n

Proof. The first part is immediate from the theorem since P x = x when
M = X. The second part is a simple expansion of the two series in the
inner-product making essential use of the linearity and continuity of h, i.

Proposition 2.3

If X has a countable Hilbert basis, then X 2 .

Proof. Let en be a countable Hilbert basis. Consider the map

J : X 2
x 7 (n ) = (hen , xi)

Then Bessels inequality shows that the resulting sequence is truly in 2 .

Linearity is easily checked. The Parseval identity is precisely the fact that the
inner-products are preserved i.e. hx, yiX = hJx, Jyi2 . Note that isometries
P
must be 1-1 (their kernel must be trivial). J is onto 2 since it maps n n en
to (n ) and the former is a well-defined vector for (n ) 2 .


2.1.1

Solving Linear Equations

One of the earliest methods for solving a linear equation Ax = b, for x, b

in a function space, and for A a continuous linear operator, was to write
the unknown element
x and the known
P
P element b in terms of a Hilbert basis
en to get x = n xn en and b = n bn en where xn = hen , xi are unknown
coefficients and bn = hen , bi are known coefficients. Substituting into Ax = b
we get
X
X
xn Aen =
bn en .
n

Moreover we can expand the vectors Aen =

coefficients, we get
X
Anm xn = bn .
n,m

Anm em and, comparing

2.2

Hilbert Bases for L2 .

J Muscat

15

This can be thought of as a matrix equation in 2 with the matrix [Anm ] having a countable number of rows and columns. Effectively, we have transferred
the problem from one on H to one on 2 , via the map J.
If the Hilbert basis elements en are chosen to be eigenvectors of A, then the
equation simplifies because of Aen = n en ; this gives n xn = bn . When n 6=
0, we must choose xn = bn /n ; when n = 0 (i.e. the homogeneous equation
Ax = 0 has solutions), then we get 0 = bn = hen , bi; if this is false, then there
are no solutions, otherwise we are free to choose xn arbitrarily. Thus there
will be a solution if, and only if, b ker A. In this case the eigenvectors
of the zero eigenvalue { em } will span the homogeneous solutions, and the
complete solution will be
X
X
x=
am em +
(bn /n )en ,
m

where the am are arbitrary constants. Note that the latter part is the particular solution and, for the case of L2 , can be rewritten as
!
Z X
X
X
1
(bn /n )en =
hen , bien /n =
en (s)en (x) f (s) ds
n
n
n
n
equivalent to the Greens function formulation of the particular solution.
2.1.2

Applications

Hilbert bases are widely used to approximate functions. The first N large
coefficients can be used to store the function in a useful compressed way.
Regenerating the function, or manipulating functions is easily done using the
Parseval identity. This has been used in JPEG and MPEG compression, as
well as in compressing images for Microsoft Encarta, and by the FBI to store
millions of fingerprints for rapid retrieval.
Such bases are also used to filter out noise or pick out particular features
in a function. First expand the function in an appropriate basis, then remove
those coefficients which are smaller than a given threshold. Regenerate the
function from the remaining coefficients.

2.2

Hilbert Bases for L2.

There are various Hilbert bases suitable for the space of L2 functions on
different domains. Each basis has particular properties that can be utilized
in specific contexts. One should treat these the same way that we treat
bases in finite dimensional linear algebra. They are indispensable for actual

2.2

Hilbert Bases for L2 .

J Muscat

16

calculations, but one has to be careful which basis to choose that makes the
problem amenable. For example, for a problem that has spherical symmetry,
it would make sense to use a Hilbert basis adapted to spherical symmetry.
2.2.1

L2 [a, b]: Fourier series

Theorem 2.4
The functions
Hilbert basis for L2 [, ].

1 einx
2

for n Z form a countable

It follows that 12 , 1 cos(x), 1 sin(x), . . . is a countable Hilbert basis.

R
Proof (NOT FOR EXAM) Suppose that f { einx } , i.e. einy f (y) dy =
R in(xy)
inx
0, n Z. Multiplying
by
e
gives
e
f (y) dy = 0 and, changing
R iny
variables, becomes e
f (x + y) dy = 0. Now (cos(y/2))2n = ( 21 einy/2 +
1 iny/2 2n
e
) is a linear combination of exponentials of various frequencies, so
2
that we have
Z
(cos(y/2))2n f (x + y) dy = 0.
But
we get

Therefore,

(cos(y/2))2n dy = 2/(n + 1) (use a reduction formula), so that

(n + 1)
f (x) =
2

(cos(y/2))2n (f (x) f (x + y)) dy.

R
|f (x)| 6 (n+1)
(cos(y/2))2n|f (x) f (x + y)| dy
2

k(cos(y/2))2n kL2 [,] kf (x) f (x + y)kL2 [,]

6 (n+1)
2
6 ckf (x) f (x + y)kL2[,]
It is a theorem from Lebesgue integration (but probably not-covered in the
introductory course in the third year), that L2 functions are L2 -continuous
i.e. kf (x) f (x + y)kL2 [a,b] 0 as y 0. Hence the function f must be
zero.
Orthonormality of the functions follows from
Z
inx imx
he , e i =
eix(mn) dx = 2nm .


Note that there is nothing special about the interval [, ]. Any other
interval [a, b] will do, except that the basis functions have to be modified
accordingly. For example, e2inx is a Hilbert basis for L2 [0, 1].

2.2

Hilbert Bases for L2 .

J Muscat

17

The polynomials are dense in L2 [a, b].

Corollary

Proof. The exponential functions can be approximated in L2 [a, b] as

closely as needed by Taylor polynomials. This means that the set of polynomials is dense in the set of exponential functions written above which in
turn are dense in L2 .

The Parseval identity for this Hilbert basis becomes

X
1 X inx
inx
f (x) =
he , f (x)ie =
n einx
2 n=
n=
R

1
einx f (x) dx, thus giving the usual Fourier series expanwhere n = 2

sion. This can also be written in terms of the basis consisting of cosines and
sines. Notice that this equation holds if, and only if, the coefficients n are
in 2 . The classical Parseval identity is none other than
Z
X
|f (x)|2 dx = kf k2L2 [,] = k(n )k22 =
|n |2 .

Moreover the Fourier operator F which sends functions in L2 to sequences

of Fourier coefficients (in 2 ) is a continuous isometry between Hilbert spaces.
This Fourier basis has the special property that they are eigenvectors of
the differential operator:
Deinx = ineinx .
It can therefore be used to turn a constant-coefficient differential equation
into a polynomial
equation as follows: expanding
unknown function
P
P the inx
inx
y(x) = n cn e and the known function f (x) = n bn e turns the differential equation
(am D m + . . . + a0 )y = f
into the series
X

(am (in)m + . . . + a0 )cn einx =

Comparing coefficients of the nth term gives

bn einx .

(am (in)m + . . . + a0 )cn = bn ,

which can be solved to give the coefficients cn in terms of the known coefficients bn . Of course this method assumes that both the solution y and the
given function f are in L2 [, ], which need not be the case. Any other
solution will not be found this way.

Hilbert Bases for L2 .

2.2
2.2.2

J Muscat

18

L2 [1, 1]: Legendre polynomials

We have just found that the set of polynomials is dense in L2 [a, b] but they
turn out to be non-orthogonal, as can be easily verified by calculating h1, x2 i.
However we can make them orthonormal using the Gram-Schmidt process.
On the interval [1, 1], the resulting polynomials are called the Legendre
polynomials.
q
q

The first few polynomials are 12 , 32 x, 32 52 (x2 13 ), etc. The general

formula is (!)
q
q


1 int(n/2)
n + 2 X (1)k (2n 2k)! n
n + 21 d
n2k
x
=
( )n (x2 1)n .
Pn (x) =
n
n
k
2 n!
(n 2k)!
2 n! dx
k=0
The Legendre functions are eigenvectors of T = D(x2 1)D = (x2
1)D 2 + 2xD:
T pn = n(n + 1)pn .
2.2.3

L2 [0, ): Laguerre functions

This Hilbert space does not contain any polynomials xn , but we can modify
them to xn ex/2 which do belong. A Gram-Schmidt orthonormalization gives
the Laguerre functions. The first few terms are ex/2 , (1 x)ex/2 , (1
2x + 12 x2 )ex/2 , etc. The general formula is
 
n
1
1 X
n k x/2
k 1
(1)
x e
= ex/2 D n (xn ex ).
ln (x) =
n! k=0
k! k
n!
The Laguerre function are eigenvectors of S = xD 2 + D x/4
1
Sln = (n + )ln .
2

2.2.4

L2 (R): Hermite functions

This time we perform orthonormalization starting with the functions xn ex

1 x2 /2 2
x2 /2
We get the Hermite functions,
,
, etc.
xe
4 e

2 /2

int(n/2)
X (1)k (2k)!  n 
(1)n x2 /2 n x2
2n
n2k x2 /2
p
hn (x) = p
x
e
=
e D e .
2k
4k k!
n! k=0
2n n!
The Hermite functions are eigenvectors of R = D 2 x2 :
Rhn = (2n + 1)hn .

2.2
2.2.5

Hilbert Bases for L2 .

J Muscat

19

L2 (A)

There are many other orthonormal Hilbert bases adapted to specific sets A
or weights. The Jacobi functions, the Chebychev (on the circle), the modified Bessel functions on L2 (0, ), the spherical harmonics on the sphere
(L2 (S 2 )) etc. It is a theorem of Rodriguez that the functions fn (x) =
1
w(x) 2 D n (w(x)p(x)n ) for any polynomial p and weight function w L2 (A)
are orthogonal: the Legendre, Laguerre, Hermite, Jacobi, Chebychev functions are all of this type.
2.2.6

Exercises

1. Suppose that en and em form a Hilbert basis for X and Y respectively.

Show that (en , em ) form a Hilbert basis for X Y .
2. Write down the Parseval identity for the basis

1 , 1

cos(nx),

sin(nx).

3. Show that the Fourier operator F sends functions in L1 [, ] to

continuously.
4. This exercise is to prove that the Legendre polynomials are orthonormal. Define un (x) = (x2 1)n . Note that D k un (1) = 0. Show
that hD n un , D m um i = hD n1 un , D m+1 um i and hence show that this
inner-product must be zero unless n = m.
5. (extra) Prove that the Legendre, Laguerre and Hermite functions are
indeed eigenvectors of the corresponding differential operators.
6. Find all L2 [1, 1] solutions of the equation (x2 1)y + 2xy + y =
sin(x).
7. Solve y (4) 2x2 y 4xy + (x4 2)y = g in series form, for functions
in L2 (R). (Hint: expand R2 )
8. Let u be aPsolution of t u = x2 u on x [0, 1] with u(0) = u0 ; show that
u(t, x) = n n (t)en (x) (justifies the separation of variables method).

9. Show that the Bessel inequality is still valid even if the orthonormal
set of vectors en is not countable; deduce that hen , xi = 0 except for a
countable number of en .
10. Suppose that en are a set of vectors such that khen , xik2 = kxk for all
x X. Show that the vectors must be dense in X and orthogonal.

Hilbert Bases for L2 .

2.2
2.2.7

J Muscat

20

Frequency-Time Hilbert Bases

A recent development in Hilbert bases are those bases for functions f (t) that
give information in both frequency and time. In contrast the coefficients that
result from the Fourier operator, for example, only give information about
the frequency content of the function. A large nth coefficient means that
there is a substantial amount of the term einx i.e. of frequency n, somewhere
in the function f (x). The aim of frequency-time bases is to have coefficients
amn depending on two parameters m and n, one of which is a frequency
index, the other a time index.
Windowed Fourier Bases
The simplest way to achieve this is to define the basis functions by
hm,n (x) = e2inx h(x m),
where h is a carefully chosen function, with khkL2 = 1, such that hmn are
orthonormal. The most common choices are the window-function h = [0,1]
2
and the gaussian h(x) = ex /2 . The m gives position (time) information,
while the n gives frequency information.
Note that summing the coefficients in n gives the windowed function:
P
P
2
2inx
h(x m), f (x)i|2
n |hhmn , f i| = Pn |he
= n |he2inx , h(x m)f (x)i|2
= kh(x m)f (x)kL2
= |hh(x m), |f (x)|i|2
Similarly summing the coefficients in m gives a windowed fourier transform of the function.
Wavelet Bases
The basis in this case consists of the following functions in L2 [0, 1]
mn (x) := 2m/2 (2m x n),
together with (x) = [0,1] . The classical Haar basis is generated by the case
(x) = [0, 1 ] [ 1 ,1] . More recently, wavelets generated by a continuous
2
2
function have been used. Such functions are necessarily nowhere differentiable.
2.2.8

Other Bases

There are many other bases used specifically for compression etc. A common
one is the Walsh basis which consists of step functions that are the normalized

2.2

Hilbert Bases for L2 .

J Muscat

21

column vectors of the matrix MN where





1 1
Mn Mn
M1 =
,
Mn+1 =
.
1 1
Mn Mn
2.2.9

Exercises

1. Prove that the windowed Fourier basis with the window function h(x) =
[0,1] (x) are orthonormal.
2. Show that the Haar basis is orthonormal.
3. Look up information on applications of wavelet and other Hilbert bases.

3 Dual Spaces

J Muscat

22

Dual Spaces

3.1

X X

In any inner-product space, every vector x gives rise to a continuous functional

X C
x = hx, i :
y 7 hx, yi

This is indeed linear, while continuity follows from the Cauchy-Schwarz inequality |x (y)| = |hx, yi| 6 kxkkyk.
Are there any other functionals besides these? In normed spaces this is
generally the case e.g. for the Banach space of continuous bounded functions
C[a, b], the dual space is L1 [a, b] which contains many other functions besides
the continuous ones. In Hilbert spaces however this is not the case:
Theorem 3.1 (Riesz theorem)
Every continuous functional of X is of the form hx, i for some
unique vector x i.e.
X !x X

= hx, i.

Proof. First notice that for any z and y, (y)z (z)y ker . Assuming
6= 0, pick z ker , non-zero, to get
0 = hz, (y)z (z)yi = (y)kzk2 (z)hz, yi.
Hence
(y) =

(z)
hz, yi = hx, yi,
kzk2

where x = (z)z/kzk2 . To show that it is unique, suppose x is another such

x then,
hx x, yi = hx, yi h
x, yi = (y) (y) = 0, y
so that x = x.

Let us study this map, called the Riesz map, from X to X more closely:

3.1

X X

J Muscat

Proposition 3.2

23

The map
J : X X
x 7 x

is a bijective conjugate-linear isometry between X and X .

Proof. Let x and y be two vectors in X. Then J(x + y) = (x + y) =
x +y = Jx+Jy since (x+y) (z) = hx + y, zi = hx, zi+hy, zi = x (z)+y (z)

zi =
for any z X. Similarly J(x) = Jx
since (x) (z) = hx, zi = hx,

(z).
x
To show that J is isometric, for any y X,

kx kX = sup
y

|hx, yi|
|x (y)|
= sup
= kxk,
kyk
kyk
y

using the Cauchy-Schwarz inequality, in particular with x = y. Hence

kJxkX = kxkX .
For bijectivity, every linear isometry is 1-1 (ker J = 0 since the lengths of
vectors is preserved); J is onto by Riesz theorem.

This shows that X and X are essentially isomorphic as Hilbert spaces.
In Banach spaces in general the most that can be said is that X is isomorphic
to a closed subspace of X . Note further that Hilbert spaces are therefore
reflexive Banach spaces i.e. X X.
3.1.1

Exercises

1. Use the Riesz map to show that the two equations kk = supx ((x)/kxk)
and kxk = sup ((x)/kk) become the same for Hilbert spaces.
2. Show that the norm of an operator is given by kT k = supx,y (|hy, T xi|/kxkkyk).
3. Show that, under the Riesz bijection, the annihilator A of a set corresponds to A .
4. Prove the Hahn-Banach theorem for Hilbert spaces as follows. Start
with any functional on a closed subspace M. Show that it must
correspond to a vector in M, and hence find an extension of on X.
Prove the rest of the theorem.

3.2

Adjoint Map

3.2

J Muscat

24

Adjoint Map

Recall that for Banach spaces, for every continuous operator T : X Y there
is a continuous adjoint or dual map T : Y X defined by T (x) =
T (x). For Hilbert spaces, the duals X and Y are essentially isomorphic
to X and Y respectively, so we should get an operator T : Y X. In fact,
it is defined by T (y) = Jhy, T.i where J is the Riesz map i.e.,
hT y, xi = hy, T xi

x, y X.

T is well-defined by the Riesz correspondence.

3.2.1

Example

Consider the left-shift map L : 2 2 defined by L(xn ) = (xn+1 ). Let

L (yn ) = (an ), then hL y, xi = hy, Lxi means
X
X
X
yn xn+1 =
yn1 xn .
an xn =
n

In particular, taking x to be a basis vector em = (nm ), we get am = ym1 so

that L (yn ) = (yn1 ) i.e. L = R the right-shift operator.
Proposition 3.3
T = T,

T is unique, linear and continuous, and satisfies

(S+T ) = S +T ,

,
(T ) = T

(ST ) = T S ,

kT k = kT k

Proof. Uniqueness: Suppose A and B are adjoints of T i.e. hAy, xi =

hx, T yi = hBy, xi for any x, y. Then Ay = By for all y so that A = B. We
are therefore justified in using the notation T .
Linearity:
hT (y1 + y2 ), xi = hy1 + y2 , T xi = hy1 , T xi + hy2 , T xi = hT y1 + T y2 , xi;
T xi = hT
y, xi = hT y, xi.
hT (y), xi = hy, T xi = hy,
Moreover,
hT x, yi = hx, T yi = hT y, xi = hy, T xi = hT x, yi;
h(S + T ) y, xi = hy, (S + T )xi = hy, Sxi + hy, T xi = h(S + T )y, xi;
y, xi;
h(T ) y, xi = hy, T xi = hT y, xi = hT

3.2

Adjoint Map

J Muscat

25

h(ST ) y, xi = hy, (ST )xi = hS y, T xi = hT S y, xi.

Continuity:
kT k = sup
x,y

|hT y, xi|
|hy, T xi|
= sup
= kT k.
kxkkyk
kxkkyk
x


Proposition 3.4
Let T be a continuous operator and M a closed
linear subspace. Then
(i) ker T = (imT ) ;

(imT ) = ker T ;

(ii) M is T -invariant M is T -invariant.

Proof. y ker T hT y, xi = 0 x X, hy, T xi = 0 y imT .
Moreover, ker T = ker T = (imT ) .
The second part is basically a translation of the fact that for Banach
spaces, M is T -invariant if, and only if, M is T -invariant. In detail, for
any y M and x M, hT y, xi = hy, T xi = 0 since T x M, so that
T y M .
Conversely, if M is T -invariant then M is T -invariant; but T = T
and M = M for a closed subspace M.


3.2.2

Application

When an operator T does not have an inverse, the equation T x = b need not
have a solution. The next best thing is to ask for a vector x which minimizes
kT x bk.
Proposition 3.5

A vector x minimizes kT x bk if, and only if

T T x = T b.

Proof. Consider the closed linear subspace M = imT . The problem is

equivalent to finding a vector y = T x M which is closest to b. A necessary
and sufficient condition is b y M = ker T . Hence T (b y) = 0 i.e.
T T x = T b.

3.2

Adjoint Map

J Muscat

26


Note however that kT x bk may have no minimum when imT is not

closed. Moreover such an x need not be unique, when ker T 6= 0. However,
there will be a unique such x with smallest norm. The mapping from b to
this x is then well-defined and is denoted by T , called the Moore-Penrose
generalized inverse. In the simple case when T is 1-1, it is given by T =
(T T )1 T (even if T is not onto).
3.2.3

Exercises

(ii) if T : H H is invertible then (T 1 ) =

1. Show (i) (I) = I,
1
(T ) .
2. For T : H H, prove that kT T k = kT k2 , and deduce from this that
kT k = kT k.
3. Find the adjoint map of the Fourier map F : L2 [, ] 2 .
4. Let X be a real Hilbert space and let T be a continuous operator, a a
fixed vector and b a fixed real number. Show that the minimum of the
quadratic function q(x) = hx, T xi + ha, xi + b occurs when (T + T )x +
a = 0. (Hint: consider q(x + h) > q(x) for all h).
5. Show that imT is dense in X if, and only if, T is 1-1.
6. Show that if T and T are onto, then T has a continuous inverse (you
need to use the open mapping theorem).
7. Let a + M be a coset of a closed linear subspace M. Show that there is
a unique vector x a+M with smallest norm. (Hint: this is equivalent
to finding the closest vector in M to a.)
8. Find the best approximation for x that solves

4
147
2 5 8 x = 1 .
369
0

4 Normal Operators

J Muscat

27

Normal Operators

In this chapter and the next we will concentrate on operators from X to

itself. These form a so-called C -algebra B(X) i.e there are the usual binary
operations of addition and multiplication on B(X) and a unary operation of
taking the adjoint.
Before we delve into the properties of these operators, let us note that an
operator is identified by the numbers hy, T xi for all x, y X. For complex
Hilbert spaces, we can even say that they are identified by the numbers
hx, T xi for all x X. This is so because hx, T xi = 0, x T = 0 (Prove!)
Definition
T T = T T .

A normal operator T is a continuous operator such that

A special case is that of self-adjoint operators which satisfy T = T .

Proposition 4.1

For T a normal operator,

(i) kT xk = kT xk,

(ii) ker T = ker T ;

kT 2 k = kT k2;

imT is dense in X T is 1-1.

Proof. (i) follows from

kT xk2 = hT x, T xi = hx, T T xi = hx, T T xi = hT x, T xi = kT xk2 .
Hence kT 2 xk = kT T xk, and
kT xk2 = hx, T T xi 6 kT T kkxk2 ,
from which follows kT k2 6 kT 2 k.
(ii) follows from the fact that T x = 0 kT xk = 0 kT xk = 0
T x = 0, using (i). Also, (imT ) = ker T = ker T which is 0 if, and only if,
T is 1-1.


4.0.4

Exercises

1. Prove that for complex Hilbert spaces hx, T xi = 0, x T = 0, by

substituting the vectors x + y and x + iy. Deduce that if hx, T xi =
hx, Sxi, x then T = S.
2. Show that a continuous operator which satisfies kT xk = kT xk is normal.

4.1

Spectrum

J Muscat

28

3. Prove that for a normal operator T , (i) T , T , and p(T ) for a

polynomial p, are also normal, (ii) ker T is T -invariant and (ker T ) is
T -invariant.
4. Use (ii) of the proposition to show that ker T = ker T 2 = ker T k . Deduce that the only normal nilpotent operator (i.e. T k = 0 for some k)
is 0.
5. Prove that for a separable Hilbert space, there can be at most a countable number of distinct eigenvalues.
6. Suppose T is diagonalisable i.e there is a Hilbert basis of eigenvectors.
Show that T is normal. (Hint: first show that T en = n en .)

4.1

Spectrum

Definition The spectrum of an operator T B(H) is the set (T ) :=

{ C : T does not have an inverse }.

(Note: The standard definition requires that T does not have a continuous inverse. But, by the open mapping theorem, every bijective continuous
linear operator has a continuous inverse, so the two definitions are equivalent.)
From the theory of Banach spaces, it follows that the spectrum of any
operator is a closed bounded non-empty subset of C. But the proof requires
complex analysis. For our purposes, we can only show that the spectrum of
a self-adjoint operator is non-empty.
Proposition 4.2
empty.

The spectrum of any (self-adjoint) operator is non-

Proof. First notice that for a self-adjoint operator A, hx, Axi is a real
number since
hx, Axi = hAx, xi = hx, Axi.
Claim: If A is self-adjoint such that hx, Axi > 0, then
x, y,

|hx, Ayi|2 6 hx, Axihy, Ayi.

The proof is as follows: for any z = tei C,

06
=
i
2
(Re e hx, Ayi) 6
|hx, Ayi|2 6

hx + zy, A(x + zy)i

hx, Axi + 2t Re ei hx, Ayi + t2 hy, Ayi
hx, Axihy, Ayi
hx, Axihy, Ayi,

4.1

Spectrum

J Muscat

29

since can be chosen to rotate hx, Ayi into |hx, Ayi|.

Now, let := inf kxk=1 hx, T xi. This implies, for all x,
hx, (T )xi = hx, T xi kxk2 > 0.
Using y = (T )x and A = T ,
kAxk4 = |hx, A2 xi|2 6 hx, Axihy, Ayi 0
as hx, T xi . Thus, belongs to (T ).
Proposition 4.3

For a normal operator T ,

(i) eigenvalues of T are conjugates of eigenvalues of T ,

(ii) eigenvectors of distinct eigenvalues of T are orthogonal.
(iii)

0
/ (T ) c > 0 kxk 6 ckT xk.

Proof. (i) is a direct application of the previous proposition on ker(T

For eigenvalues and with corresponding eigenvectors
) = ker(T ).
x and y, we have
hy, xi = hy, T xi = hT y, xi = h
y, xi = hy, xi,
implying that either = or hy, xi = 0.
(ii) Suppose the spectrum (T ) does not contain 0. Then T has a continuous inverse T 1 and kT 1 yk 6 ckyk, implying kxk 6 ckT xk.
Conversely, suppose the right inequality is true for all x. Then T x = 0
implies x = 0 so that T is 1-1. The image of T is closed since suppose
yn = T xn is a Cauchy sequence in the image of T . Then kxn xm k 6
ckT xn T xm k = ckyn ym k 0 as n, m . Hence xn x implying
that yn = T xn T x since T is continuous. These two facts imply that the
image of T must be X itself since imT = ker T = ker T = 0. Thus T is
invertible, and continuous:
kT 1 yk 6 ckT T 1yk = ckyk.

That is, a value is in the spectrum iff there are unit vectors xn such
that (T )xn 0.

4.1

Spectrum

J Muscat

30

Proposition 4.4 The spectrum of T has spectral radius kT k, contains

no residual spectrum, and is a subset of the closure of the set
{hx, T xi/hx, xi}.

Proof. The spectral radius of (T ) is given by rT = limn kT n k1/n =

k
k
limk kT 2 k2 = kT k.
Either T is 1-1, in which case its image is dense in X and forms
part of the continuous spectrum, or it is not 1-1, in which case there must
be an eigenvector and is an eigenvalue (point spectrum).
Let A := {hx, T xi/hx, xi}. Then
/ A implies that
> 0 x |

hx, T xi
| > ,
kxk2

hx, (T )xi
| > > 0,
kxk2

kxk2 6 |hx, (T )xi| 6 kxkk(T )xk,

kxk 6 k(T )xk,

/ (T ).

Notice that since |hx, T xi| 6 kT kkxk2 we have that the set A is contained
in the ball of radius kT k. Moreover, since the spectral radius of T is kT k,
the radius of A itself is kT k.

Theorem 4.5 (Spectral Theorem in Finite Dimensions)
A normal operator on a finite-dimensional vector space is diagonalisable.
Proof. In finite dimensions, B(X) has dimension (dim X)2 and so every
operator T must satisfy a polynomial mT (x) = (x )k . . .. This induces
a decomposition of the vector space as X = ker(T )k + . . .. For normal
operators ker(T )k = ker(T ), so that the minimal polynomial consists
of simple factors. This is equivalent to the diagonalizability of T .
4.1.1

Exercises



 
1 0
1
1. By taking T =
and x =
, show that the spectrum of a
0 1
1
normal operator need not equal the closure of the set { hx, T xi/kxk2 }.
2. Show that kp(T )k = max(T ) kp()k when T is normal and p is a
polynomial.

4.2

4.2

Unitary Operators

J Muscat

31

Unitary Operators

Definition A unitary operator is a Hilbert-space automorphism i.e. a

linear bijection which preserves the inner-product,
hT x, T yi = hx, yi x, y X.

Proposition 4.6

The following are equivalent

hT x, T yi = hx, yi

x, y T T = I kT xk = kxk

x.

Proof. hx, yi = hT x, T yi = hx, T T yi x, y T T = I. By the

comment at the beginning of this chapter, we can also say that hx, xi =
hT x, T xi = hx, T T xi x T T = I.

This proposition is basically saying that preserving the inner-product
(lengths and angles) is equivalent to preserving lengths. Recall also that
norm-preserving linear operators must be 1-1. Hence we conclude that a
linear operator is unitary if, and only if, it is onto and T T = I.
Proposition 4.7

Let T be a unitary operator, then

1. the adjoint T = T 1 is also unitary;

2. T is continuous with norm kT k = 1;
3. T is normal;
4. the set of unitary operators form a group i.e. if S, T are
unitary then so is ST .
Proof. If T is unitary then it is invertible and T T = I; hence T =
T 1 . T is itself unitary since (T )1 = (T 1 ) = T . For continuity,
kT k = supx (kT xk/kxk) = 1. T is normal since T always commutes with
its inverse, which for the case of unitary operators, is its adjoint. Finally,
(ST ) (ST ) = T S ST = I and ST is invertible when S, T are.


4.2

Unitary Operators

Proposition 4.8
circle,

J Muscat

32

The spectrum of a unitary operator lies in the unit

T eiR .

Proof. The spectrum must lie in the unit ball since kT k = 1. Take
|| < 1, then T = T (1 T ) and kT k = ||kT k = || < 1. Therefore
(1 T ), and so T , are invertible, with continuous inverse.


4.2.1

Exercises

1. Let T be an invertible normal operator. Show that T T 1 is unitary.

2. Show that the right shift operator on 2 satisfies R R = I but is not
unitary.
3. Let T : 2 2 be defined by T (xn ) = (in xn ). Show that T is unitary
with eigenvalues { 1, i }. Find the set { hx, T xi/kxk2 } and deduce
that the spectrum of T is precisely the set of eigenvalues.
4. Show that an operator is unitary if, and only if, it maps Hilbert bases
to Hilbert bases.
5. Show directly that eigenvalues of unitary operators must satisfy || = 1.
6. In the proof of the equivalence of the first proposition of this section,
use was made that the Hilbert space is over the complex field. Show
that for real Hilbert spaces, T T = I kT xk = kxk is also valid.
7. Let Tg be a group of unitary operators for g G a group. Suppose M
is Tg -invariant for all g G. Show that M is also invariant.
8. (*) Show that a function on a Hilbert space which preserves the innerproduct must be linear. Deduce that isometries (i.e. preserve distances)
on a Hilbert space must be of the type P (x) = Ux + a where U U = I.
(Hint: show hf (x + y), f (z)i = hf (x) + f (y), f (z)i, so f (x+y)f (x)
f (y) h im f i h im f i .)

4.3

Self-Adjoint Operators

4.3

J Muscat

33

Self-Adjoint Operators

Definition A self-adjoint operator is a continuous linear operator such

that T = T .
Self-adjoint operators are obviously normal.
Proposition 4.9 Every operator in B(X) can be written uniquely as
T = S + iR for S, R self-adjoint.
Proof. Simply check that S = (T + T )/2 and R = (T T )/2i are selfadjoint and add to T . Uniqueness follows from the fact that A + iB = 0 for
A, B self-adjoint if, and only if, A = B = 0 (since then A = A = (iB) =
iB = A.)

Note that in this case, T = S iR; and T is normal if, and only if,
SR = RS; T is unitary if, and only if, T is normal and S 2 + R@ = 1; T is
self-adjoint if, and only if, R = 0.
T is self-adjoint iff the set { hx, T xi/kxk2 } is real.

Proposition 4.10

Proof. Since T is self-adjoint, hx, T xi = hT x, xi = hT x, xi = hx, T xi is

a real number. Hence the set { hx, T xi/kxk2 : x X } is real.
Conversely, if the given set is real, then
hx + y, T x + T yi = hx, T xi + hx, T yi + hy, T xi + hy, T yi
implies that hx, T yi + hy, T xi is real. Similarly, starting with x + iy gives
that hx, T yihy, T xi is purely imaginary. Hence hx, T yi = hy, T xi = hT x, yi.
Thus T = T and T is self-adjoint.
Corollary

The spectrum of a self-adjoint operator is real.

Note that the converse is also true i.e. a normal operator whose spectrum
is real must be self-adjoint.
4.3.1

Exercises

1. Prove that a continuous projection P is normal iff it is orthogonal iff

it is self-adjoint.
2. By taking = + i with 6= 0 and expanding k(T )xk, show that
k(T )xk > ||kxk and hence that 6 (T ).

4.4

Compact Normal Operators

J Muscat

34

3. Let T y = p1 (py ) + qy on the Hilbert space L2 [a, b]. Show that

hT u, vi = hu, T vi + [p(u v uv )]ba . Deduce that when we restrict to
the subspace of those functions which also satisfy (i) the periodic condition y(a) = y(b), or (ii) the unmixed conditions 1 u(a) + 2 u (a) = 0,
1 u(b) + 2 u (b) = 0, then T is self-adjoint.
4. A self-adjoint operator is called positive definite when hx, T xi > 0 fora
all x 6= 0. Show that in this case the map hhx, yii = hx, T yi defines an
inner-product on X.

4.4

Compact Normal Operators

Definition T is called a compact operator when for every bounded set

B, T B is compact.
In particular, if xn are unit vectors then T xn has a convergent subsequence. Compact operators are obviously continuous.
For example, matrices T : CN CM are compact since T B has to be
finite-dimensional, so its closure must be closed and bounded, hence compact.
But in infinite dimensions, such as 2 , the identity operator I is not compact
(take any orthonormal sequence).
Example. The map T : 2 2 defined by T (an ) := (an /n) is compact.
Consider the linear maps TN : 2 2 , TN (an ) := (a1 , a2 /2, . . . , aN /N, 0, . . .);
TN is compact since its image is a subspace of CN . Now let xn be a bounded
sequence, say kxn k2 6 M. For each N, there is a subsequence xn,N such
that
1
kTN xn,N TN xm,N k 6 N .
2
Hence
kT xn,n T xm,n k = k(T Tn )xn,n + (Tn Tm )xn,n + Tm (xn,n xm,n ) + (Tm T )xm,n k
6 k(T Tn )xn,n k + k(Tn Tm )xn,n k + kTm (xn,n xm,n )k + k(Tm T )xm,n k
0
Thus every bounded sequence xn gives a convergent subsequence of T xn .
Theorem 4.11
For T compact normal, the non-zero part of the
spectrum consists of a countable number of eigenvalues whose only
possible limit point is 0. Each non-zero eigenvalue has a finitedimensional eigenspace.
Proof. Let (T ), 6= 0. Then there are unit vectors xn such that
(T )xn 0. But when T is compact, there is a convergent subsequence

4.4

Compact Normal Operators

J Muscat

35

T xni y. This implies

xni = T xni (T )xni y
so xni y/ and T xni T y/. Since limits are unique, T y = y, and is
an eigenvalue. (Note that y is not zero since xni are unit vectors.)
Suppose there are an infinite number of unit eigenvectors en with nonzero eigenvalues n . The bounded sequence n must have a subsequence
converging to some . From T en = n en and the fact that T en y for
some subsequence, we get n en y. Yet, as T is normal, its eigenvectors en
of distinct eigenvalues are orthogonal,
and these cannot have a convergent

subsequence since ken em k = 2.

Keeping this in mind, if 6= 0, and there are an infinite number of
distinct eigenvalues n , then their orthonormal eigenvectors would converge
to en y/, a contradiction. The only limit point must then be 0, and there
can only be a countable number of non-zero eigenvalues. Similarly, if has
an infinite number of linearly independent eigenvectors, choose a countable
number of them en , make them orthonormal by the Gram-Schmidt algorithm,
and this furnishes another contradiction.

Corollary Spectral Theorem for Compact Normal operators
Every compact normal operator has a countable orthonormal set
of eigenvectors en with non-zero eigenvalues n , and
X
Tx =
n hen , xien .
n

Proof. T restricted to the orthogonal space of M := [[en ]] is still compact

normal. Yet it cannot have any non-zero eigenvalues because these are already accounted for in en . Its spectrum must therefore be { 0 }, that is the
T restrictedPto M is the zero operator. Hence M = ker T . Writing any
vector x = n hen , xien + v where v ker T , we find
X
X
Tx =
hen , xiT en =
n hen , xien .
n

Thus compact normal operators are diagonalizable. In terms of their representation in 2 , every vector x (an ) is mapped to T x (n an ).