MATH 353 LECTURE NOTES

WEEK 9
INTRODUCTION TO FOURIER SERIES

J. WONG (FALL 2017)

Topics covered
• Introductory notes
◦ Motivating example: Heat conduction in a metal bar
◦ Connection to linear algebra
• Orthogonality, inner products
◦ Inner products (dot product)
◦ Orthogonal bases, eigenvectors/eigenvalues
◦ Extending to functions: L2 (the infinite version of the dot product)
◦ Orthogonality in L2 and the norm (again, infinite version)
• Fourier Series
◦ Definition of Fourier series
◦ Basic structure
− complete set
− Orthogonality, mean-square convergence
− As an approximation for periodic functions
◦ Calculating Fourier series for periodic functions
− Two examples

1. Motivation
1.1. Linear algebra analogy. Suppose we have an n × n matrix A and wish to solve
Ax = b.
Assume A is invertible and symmetric. You know many ways of solving this problem, but
let’s go through the details of one approach that will be a useful analogy. Rather than solve
a linear system, we would like to choose the right basis so that the n equations to solve are
independent.

To do so, recall that an n × n real symmetric matrix has distinct eigenvalues

λ1 , λ2 , · · · λn
and eigenvectors
v1 , v 2 , · · · v n
that form a basis for Rn . Not only that, but they are orthogonal:
vi · v j = 0 if i 6= j.
1
2 J. WONG (FALL 2017)

Now both x and b can be written in terms of this basis:

X n n
X
x= ci v i , b = di vi .
i=1 i=1

Now since Avi − λi vi , plugging into Ax = b gives

n
X n
X
λi ci vi = di vi .
i=1 i=1

Since the vi ’s form a basis, it must be true that the coefficients of each vi are equal, so
ci = di /λi .
This leaves only the problem of writing b in terms of the vi ’s. Here is where orthogonality
is crucial. Take the dot product of the equation for b,
n
X
b= di vi ,
i=1

with vj to get
n
X
b · vj = di (vi · vj ).
i=1
Now all the terms in the sum are zero except for i = j:
b · vj
b · vj = di vj · vj =⇒ di ] = .
vj · v j
The solution to Ax = b is then
n
X 1 b · vi
x= ci vi , ci = .
i=1
λi vi · vi
Assuming we have the eigenvectors and eigenvalues, notice that at no point did we need to
solve any coupled linear systems. All the equations (for di and then ci ) only had one term
divided by another term. The orthogonality of the basis allowed us to decompose b in terms
of the basis by solving for the di ’s one by one, and the eigenvalue property Av = λv let us
solve for ci in terms of di . In the coming weeks, we will see how to generalize these ideas to
differential equations.

1.2. Heat conduction in a metal bar. A metal bar with length L = π is initially heated
to a temperature of u0 (x). The temperature distribution in the bar is u(x, t). Over time, we
expect the heat to diffuse or be lost to the environment until the bar is evenly heated.
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 3

Physicist Joseph Fourier, around 1800, studied this problem and in doing so drew attention
to a novel technique that has since become one of the cornerstones of applied mathematics.
The approach outlined below hints at some of the deep structure we will uncover in the
remainder of the course.
We will show later that the temperature can be modeled by the heat equation
∂u ∂ 2u
= , t > 0 and x ∈ (0, π).
∂t ∂x2
Assume that the temperature is held fixed at both ends. This condition is imposed in the
model through boundary conditions
u(0, t) = 0, u(π, t) = 0 for all t.
Notice that unlike initial conditions, there are boundary conditions at two different values
of x. Thus we do not have an ‘initial value problem’ where we start at one point.

Lastly, the initial heat distribution is t = 0 is

u(x, 0) = u0 (x)
which is the initial condition. The temperature should decrease as heat leaks out of the
bar through the ends; eventually it all dissipates. The solution u(x, t) should predict this.

In summary, our goal is to find a function u(x, t) defined on [0, π] satisfying

∂u ∂ 2u
(1a) = t > 0 and x ∈ (0, π),
∂t ∂x2
(1b) u(0, t) = u(π, t) = 0 for t ≥ 0
(1c) u(x, 0) = u0 (x).
We call (1) (the three equations together) an initial boundary value problem for u(x, t).

To solve the equation, we guess at an exponentially decaying solution

(2) u(x, t) = e−λt φ(x).
Our objective here is just to find a solution to the first two parts, (1a) and (1b) and worry
about the initial condition later.

Substituting into the PDE (1a), we find that

−λφ(x) = φ00 (x).
4 J. WONG (FALL 2017)

Now substitute into the boundary conditions (1b) (note that e−λt cancels out here) to get
φ(0) = 0, φ(π) = 0.
For convenience set λ = µ2 . It follows that (2) satisfies the PDE (1a) and the boundary
conditions (1b) if the function g(x) solves the boundary value problem
(3) φ00 (x) + µ2 φ(x) = 0, φ(0) = 0, φ(π) = 0.
This problem is not an initial value problem, but it is a constant-coefficient ODE, so we can
still solve it explicitly. The general solution is
φ = c1 sin(µx) + c2 cos(µx).
Imposing the condition φ(0) = 0 we find that
φ = c1 sin(µx).
The second condition, φ(1) = 0, requires that
sin(µπ) = 0.
Non-trivial solutions exist whenever µ is a non-zero integer. We have now found an infinite
sequence of solutions to (3):
φn (x) = sin(nx), n = 1, 2, 3, · · ·
Observe that (3) is a linear, homogeneous problem. In particular,
(4) φ1 , φ2 are solutions to (3) =⇒ c1 φ + c2 φ2 is a solution.
This means that for any constant an , the function
2
(5) an e−n t φn (x)
is a solution to the heat conduction problem with initial data
u0 (x) = an sin(nx).
Now the crucial question: what happens when the initial data is not a sine? No single
solution of the form (5) will work. Fourier’s breakthrough was the realization that, using the
superposition principle (4), the solution could be written as an infinite linear combination
of all the solutions of the form (5):
∞
2
X
u(x, t) = an e−n t φn (x).
n=1

Then u(x, t) solves the original problem (1) if the coefficients an satisfy
∞
X
(6) u0 (x) = an φn (x).
n=1

This idea is a generalization of what you know from linear algebra (representing vectors in
terms of a basis) but with basis functions {sin(nx) : n = 1, 2, 3, · · · }.

In fact, this set of functions has the rather remarkable orthogonality property
Z π Z π
(7) φm (x)φn (x) dx = sin(mx) sin(nx) dx = 0, m 6= n.
0 0
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 5

To solve for the coefficient am , we can multiply (6) by sin(mx) and integrate:
Z π Z πX∞
u0 (x) sin(mx) dx = an sin(mx) sin(nx) dx.
0 0 n=1

Now move the integral inside the sum (it is not trivial to show this is allowed!). By the
property (7), only one of the terms in the sum will be non-zero:
Z π Z πX ∞
u0 (x) sin(mx) dx = an sin(mx) sin(nx) dx
0 0 n=1
∞
X Z π
= an sin(mx) sin(nx) dx
n=1 0
∞
!
X Z π
= an · 0 + am sin(mx) sin(mx) dx
n=1,n6=m 0
Z π
= am sin2 (mx) dx.
0
Magically, the infinite sum has been reduced to a simple equation for am :
Rπ
u0 (x) sin(mx) dx
(8) am = 0 R π 2 .
0
sin (mx) dx
This process works for all m, so the solution to the heat conduction problem (5) with arbitrary
initial condition u0 (x) is
∞
2
X
u(x, t) = an e−n t sin(nx)
n=1
with the coefficients given by the formula (8). Of course, all of the manipulations here are
formal and unjustified - it is far from clear whether the series converges, or if it is valid to
swap integrals and sums, and so on (Fourier did not know this either when first applying
the method; it took several decades to settle the issue).
1.3. Observations and goals. The method in the example may seem rather mysterious,
but it hints at some remarkable structure. We have identified eigenfunctions φn that satisfy
φ00n = λφn , φn (0) = φn (π) = 0
and found that they have a special orthogonality property. Then, we exploited superposition
to build an infinite series, which has enough coefficients to match any initial condition. The
functions φn must span all possible initial conditions - they are a basis in some sense. While
building up the theory, we need to address some fundamental questions first:
• What does it mean for functions to be ’eigenfunctions’ and ’orthogonal’ ? What does
it mean for a collection of functions to be a basis, and for what? Think of this as
generalizing eigenvalues and ortthogonal bases for Rn . The fact that the dimension
is infinite leads to serious complications.
• What are the properties of the linear operator
L[φ] = φ00
that arises in the ’eigenvalue’ problem L[φ] = λφ?
6 J. WONG (FALL 2017)

• How can these tools be used to solve PDEs (and what are the limitations)?
• What are the implications of this theory for real problems?
1.4. Outline.
• L2 and orthogonality
◦ functions in L2 (vectors in Rn )
◦ inner product, orthogonality (dot product, orthogonal vectors)
◦ orthogonal basis for L2 (orthogonal basis for Rn with eigenvectors)
• Fourier series, eigenfunctions
◦ Appearance as solution to boundary value problem
◦ operator L and eigenfunctions (A and eigenvectors of A)
◦ Fourier series
• Heat equation, etc.
◦ Introduction to PDEs
◦ Solution using eigenfunctions
2. Inner products, orthogonality of functions
To study eigenfunctions and PDEs and so on, we need to identify the correct space where
the functions of interest reside, and extend notions from linear algebra in Rn (see section 4)
to this space of functions.
2.1. Square-integrable functions (L2 [a, b]). For this section, we will consider real-valued
functions f defined on an interval [a, b]. Such a function is called square-integrable1 if
Z b
|f (x)|2 dx < ∞.
a
2
We now define L [a, b] to be the set of real-valued functions defined on [a, b] that are square
integrable, i.e.
Z b
2
L [a, b] = {f : [a, b] → R such that |f (x)|2 dx < ∞}.
a
This space will turn out to be the right one for studying Fourier series. We will not prove
it, but if f and g are square integrable then f + g is also square integrable, so linear combi-
nations of functions in L2 [a, b] stay in L2 [a, b].

Norm: We define a norm on the space (the ‘L2 norm’) as follows:

s
Z b
kf k = |f (x)|2 dx.
a

When applied to the difference of two functions f and g, i.e.

Z b
2
kf − gk = |f (x) − g(x)|2 dx
a
the norm is a way of measuring the distance between the two functions, analogous to the
Euclidean distance for vectors:
kx − yk2 = (x1 − y1 )2 + · · · + (xn − yn )2 .
1Mathematicians will usually refer to a square integrable function as an ‘L2 function’.
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 7

Notice that this ‘distance’ kf − gk between f and g is a sort of weighted average of the area
between the curves in the interval [a, b]. The quantity (??) is sometimes called the mean-
square distance or mean-square error if g is some approximation to f .

Norm examples: Consider the norm for L2 [−1, 1].

If f (x) = x2 then Z 1
2 1 3 1
2 2
kf (x)k = x dx = x  = .
−1 3 −1 3
If f (x) = |x| and g(x) = x then
Z 1 Z 0
2 2 4
kf (x) − g(x)k = (x − |x|) dx = (2x)2 dx = .
−1 −1 3
Note that f = g when x > 0 but they differ when x < 0.

Notation: For clarity, one sometimes writes kf k2 , kf kL2 or kf kL2 [a,b] . Only this norm
will be used, so we will just write kf k. Be careful to keep track of the interval, because the
norm depends on it; e.g. if f (x) = 1 then kf k = 2 in L2 [−1, 1] but kf k = 4 in L2 [−2, 2].

Inner product: The inner product on L2 [a, b] is

Z b
hf, gi = f (x)g(x) dx.
a

Two functions are called orthogonal ‘with respect to the inner product’ if hf, gi = 0, and
a set of functions {fn } is orthogonal if distinct pairs are all orthogonal (just as with Rn !).
Unfortunately, there is no obvious intuition for orthogonal functions (compared to Rn , where
it means the two vectors are perpendicular).

Notation: Again, because this inner product is the only one we will work with, the ‘with
respect to...’ part will be omitted and the inner product used will be left implied.

Inner product examples: Consider the norm for L2 [−1, 1]. Then
Z 1
h1, xi = x dx = 0.
−1
So the constant function 1 and the function x are orthogonal (in the interval [−1, 1]).

However, Z 1
2 2
h1, x i = x2 dx = ,
−1 3
8 J. WONG (FALL 2017)

so 1 and x2 are not orthogonal. On the other hand, for g(x) = x2 − 1/3,
Z 1
2 2
h1, gi = (x2 − 1/3) dx = − = 0.
−1 3 3
This means that the set
{1, x, x2 − 1/3}
is an orthogonal set in L2 [−1, 1], whereas
{1, x, x2 }
is not.

2.2. L2 [−`, `] as a space of 2`-periodic functions. A function f (x) is T -periodic (or

‘has a period T ’) if

(9) f (x) = f (x + T ) for all x.

That is, after a length T , the function repeats. A periodic function is therefore defined by
its values on any interval of length T . For example, sin x has period 2π and cos(4πx) has
period 1/2.

Note that the period T as defined above is not unique (e.g. sin x is also periodic with
period 4π). The fundamental period refers to the smallest possible T . The distinction is
not particularly important for the discussion here, however.

The point: We will often be interested in periodic functions defined for all x ∈ R. If
the period is 2` then we can identify such a periodic function with its restriction to [−`, `].
Analysis of periodic functions is then done on that interval.

Similarly, any f ∈ L2 [−`, `] corresponds to a 2`-periodic function defined by (9). That

is, we construct a periodic function out of f by copying it on [`, 3`], then again on [3`, 5`]
and so on. Thus we can view L2 [−`, `] also as the space of 2`-periodic functions such that
the integral of f 2 over one period is finite.
For example, consider the function f (x) = |x| in L2 [−1, 1]. This corresponds to the
periodic function drawn below:

The periodic function sin(x), for instance, as a member of L2 [−π, π] is continuous. If in

L2 [−π/2, π/2] however, sin x is not continuous (but is still in the space):
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 9

The values at the discontinuities −π/2, π/2 etc. are ambiguous.

3. Fourier series: fundamentals

In this section we consider the space L2 [−`, `] as the space of 2`-periodic functions in
the sense of the previous section. In terms of the theory/computations, it will not matter
whether we think of f ∈ L2 [−`, `] as really periodic on all of R or just defined on [−`, `] but
it does matter conceptually for applications.
3.1. Definition. Following Fourier’s example, we observe that the set of functions
πx 2πx 3πx πx 2πx
(10) constant, cos( ), cos( ), cos( ), · · · sin( ), sin( ), · · ·
` ` ` ` `
forms an orthogonal set in L2 [−`, `] with respect to the inner product
Z `
hf, gi = f (x)g(x) dx.
−`
Explicitly, the following identities hold (and are not too hard to prove; see homework):
Z `
mπx nπx
(11) cos sin dx = 0, for all m, n,
−` ` `
Z ` (
mπx nπx 0 m 6= n
(12) cos cos dx =
−` ` ` ` m = n and m 6= 0,
Z ` (
mπx nπx 0 m 6= n
(13) sin sin dx =
−` ` ` ` m = n.
Note that the orthogonality of the constant with all the sines and cosines is contained in the
above (since cos mπx
`
= const. when m = 0).

The Fourier series for a function f ∈ L2 [−`, `] is given by

∞
a0 X  nπx nπx 
(14) f = + an cos + bn sin .
2 n=1
` `
Note: we will study the sense in which f and the series are equal later. For now,
we can use the orthogonality relations (11)-(13), to formally solve for the coefficients (i.e.
ignoring any possible technical issues).
10 J. WONG (FALL 2017)

For am , take the inner product of both sides with cos mπx `
and use the linearity property
to put the inner product inside the sum:
∞ 
mπx a0 mπx X nπx mπx nπx mπx 
hf, cos i = h , cos i+ an hcos , cos i + bn hsin , cos i
` 2 ` n=1
` ` ` `

Now by (11)-(13), all the terms on the right vanish except the n = m term in the cosine
sum. If n ≥ 1 then
mπx mπx mπx
hf, cos i = an hcos , cos i.
` ` `
Now employ (12) to evaluate the right side. We end up with the formula
1 `
Z
mπx
(15) an = f (x) cos dx, n ≥ 0.
` −` `
Note that the a0 case has to be checked separately. Keeping the formula valid for n = 0 is
the reason for the choice of 1/2 has the constant basis function. Similarly, we get
1 `
Z
mπx
(16) bn = f (x) sin dx, n ≥ 1.
` −` `

3.2. Periodic functions and the Fourier series. Note that the basis functions sin(nπx/`)
etc. for L2 [−`, `] are periodic with period 2`. Thus the Fourier series for f (x) defined on
[−`, `] is always 2`-periodic as well! This fact makes a Fourier series a natural approximation
for a periodic function.

3.3. The main result. The fundamental result (which has profound implications in math
and physics) is that any function f in L2 [−`, `] has a representation as a Fourier series.

In terms of a basis: Put another way, the set (10) of sines and cosines,
πx 2πx 3πx πx 2πx
constant, cos( ), cos( ), cos( ), · · · sin( ), sin( ), · · ·
` ` ` ` `
forms an orthogonal basis2 for L2 [−`, `], which means that
∞
a0 X  nπx nπx 
(17) Every f ∈ L2 [−`, `] can be written as f = + an cos + bn sin
2 n=1
` `

with an , bn given by (15) and (16). The equality here is not in the sense that at each point
x, the series converges to f (x); we will resolve the ambiguity later.

In terms of periodic functions: The basis functions sin(nπx/`) etc. for L2 [−`, `] are
periodic with period 2`. Thus the Fourier series for f (x) defined on [−`, `] is always 2`-
periodic as well! This fact makes a Fourier series a natural approximation for a periodic
function. Put yet another way, every reasonably nice periodic function is a superposition of
sines and cosines.
2Technical note just for completeness: the term ‘basis’ here is not exactly correct because it allows for
infinite linear combinations; there’s a slightly different term for this.
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 11

3.4. Partial sums; approximation and convergence (briefly). In general, the N -th
partial sum of the Fourier series for a periodic function f (x) is
N
!
a0 X  nπx nπx 
SN (x) = + an cos + bn sin
2 n=1
` `
This partial sum is an approximation to f (x). In fact, it is often a very good approximation.
The main result can therefore be rephrased yet again to say that every reasonably nice pe-
riodic function can be approximated by a sum of sines and cosines.

Notation: ‘N -th partial sum’ might instead be defined to be the first N non-zero terms
of the series rather than terms up to cos N πx/` and sin N πx/`; it depends on the series in
question.

This notion of approximation lets us describe the equality in precisely; it means that
kf − SN k → 0 as N → ∞.
That is, the ‘mean square error’ (i.e. the distance between f and SN as measured in the L2
norm)
Z `
2
kf − SN k = |f (x) − SN (x)|2 dx
−`
goes to zero as we increase the number of terms in the approximation.

Because the error takes into account the difference |f (x) − SN (x)| at every point in the
interval, we would like to say that the convergence implies that the error goes to zero uni-
formly, i.e.
max |f (x) − SN (x)| → 0.
x∈[−`,`]
One might care about the above if trying to use a Fourier series to approximate a function
(it should be a good estimate everywhere!). If f (x) is nice enough this is true, but it is not
true in general. The distinction between the various notions of convergence is subtle; we
will consider it in more detail later after becoming better acquainted with Fourier series.
3.5. Examples. Some examples help to illustrate the defintions of the previous section.
3.5.1. Triangular wave. Let
(
−x −1 ≤ x < 0
f (x) =
x 0<x≤1
and f (x) = f (x + 2) when x ∈
/ [−1, 1]. This is the same as before:
12 J. WONG (FALL 2017)

To compute the Fourier series (note that ` = 1), we use (11)-(13). Some integration by parts
is involved; in particular the formula
Z
1 1
x cos ax dx = x sin ax + 2 cos ax.
a a
When n ≥ 1,
1 `
Z
an = f (x) cos(nπx) dx
` −`
Z 0 Z 1
=− x cos(nπx) dx + x cos(nπx) dx
−1 0
Z 1
(change x → −x in first term) =2 x cos(nπx) dx
0
1
h 2 2 i 
= x sin(nπx) + 2 2 cos(nπx) 

nπ nπ 
0
2  1
(since sin(nπ) = 0 for all n) = 2 2 cos(nπx)

nπ 0
2 n
= 2 2 ((−1) − 1).
nπ
Thus for n ≥ 1, (
− n24π2 for odd n
an = .
0 for even n
For a0 just compute Z 1 Z 1
a0 = f (x) dx = 2 x dx = 1.
−1 0
For the sine terms, there is cancellation:
1 `
Z
bn = f (x) sin(nπx) dx
` −`
Z 0 Z 1
=− x sin(nπx) dx + x sin(nπx) dx
−1 0
= 0.
The two terms exactly cancel. The Fourier series representation for f is therefore
∞
1 4 X 1
f (x) = − 2 cos((2n − 1)πx).
2 π n=1 (2n − 1)2
Now let’s define the partial sum containing N non-zero terms as follows (not the same as
the general definition earlier to avoid terms that are zero):
N
1 4 X 1
SN (x) = − 2 cos((2n − 1)πx)
2 π n=1 (2n − 1)2
1 4 1 4
so S1 = 2
− π2
cos πx, S2 = 2
− π2
(cos πx + 19 cos 3πx) and so on.
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 13

A plot of the approximations shows that the agreement is quite good, even with only a
few terms:

1
1 term
2 terms
0.8 3 terms

0.6

0.4

0.2

0
-3 -2 -1 0 1 2 3
x

A plot of the mean-square error

Z 1
2
kf − SN k = |f (x) − SN (x)|2 dx
−1

shows that the mean-square error does indeed go to zero as N → ∞. In fact, it goes to zero
rather quickly (exponentially fast!). The maximum error

max |f (x) − SN (x)|

x∈[−`,`]

also goes to zero as N → ∞, but much slower; the agreement is not so good near the peak
of the triangle where there is a sharp corner.

10 -1
M.S. error
10 -2 max. error

10 -3

10 -4

10 -5

10 -6

10 -7
0 10 20 30 40 50
N
14 J. WONG (FALL 2017)

3.5.2. Square wave. Let (

−1 −1 ≤ x < 0
f (x) =
1 0<x≤1
and f (x) = f (x + 2) when x ∈ / [−1, 1]. Then a0 = 0 and
Z 1 Z 0 Z 1
an = f (x) cos(nπx) dx = − cos(nπx) dx + cos(nπx) dx = 0,
−1 −1 0

Z 1
bn = f (x) sin(nπx) dx
−1
Z 0 Z 1
= sin(nπx) dx + sin(nπx) dx
−1 0
Z 1
=2 sin(nπx) dx
0
2 1
=− cos(nπx)

nπ 0
2
= − ((−1)n − 1)
nπ
so bn = 4/nπ when n is odd and bn = 0 when n is even. Thus the Fourier series for f (x) is
∞
4X 1
f (x) = sin((2n − 1)πx).
π n=1 2n − 1
The N th partial sum SN Is
N
4X 1
SN (x) = sin((2n − 1)πx).
π n=1 2n − 1

Error: A plot of the approximation shows that the partial sums converge nicely where
f is continuous, but do not perform well at all near the discontinuity:

1 2 terms
4 terms
12 terms
0.5

-0.5

-1

-3 -2 -1 0 1 2 3
x
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 15

The partial sums tend to oscillate and overshoot the discontinuity by a significant amount.
Again, define Sm (x) to be the first m terms of the series above, e.g. S2 = π4 (sin πx+ 31 sin 3πx).
Then as in the previous example,
kf − SN k → 0 as N → ∞.
Zooming in on the top part of the discontinuity at x = 1:

1.2
2 terms
1.15
4 terms
1.1 12 terms
24 terms
1.05 100 terms

0.95

0.9

0.85

0.8
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1
x
As the number of terms increases, the overshoot does not decrease in magnitude. The am-
plitude of the bad oscillations stays about the same; the width, however, shrinks. Thus at
each point x 6= 1, the series will converge to f (x), but the maximum error is
max |f (x) − SN (x)| ≈ 0.18 as N → ∞.
x∈[−1,1]
16 J. WONG (FALL 2017)

4. Review: Inner products and orthogonality in Rn

Reviewing the relevant ideas in Rn is helpful. Recall that the 2-norm (or Euclidean norm)
of a vector x is q
kxk = x21 + x22 + · · · + x2n .
The quantity kx − yk measures the distance between two vectors in x, y ∈ Rn (which is
literally the distance between the two points x and y), The inner product (or dot product)
of two vectors is
hx, yi = x1 y1 + · · · + xn yn
also denoted x · y. The inner product on Rn has several important properties:
i) Symmetry:
hx, yi = hy, xi.
ii) Linearity in each argument: If x, y, z are vectors then
hx + y, zi = hx, zi + hy, zi and hx, y + zi = hx, yi + hx, zi.
If c is a scalar then
hcx, yi = chx, yi and hx, cyi = chx, yi.
iii) Positive definiteness:
hx, xi ≥ 0 and hx, xi = 0 =⇒ x = 0.
Note that the Euclidean norm is given by
p
kxk = hx, xi.
Property (iii) ensures that kxk = 0 if and only if x = 0.

Two vectors x, y ∈ Rn are orthogonal if their inner product is zero:

x, y orthogonal ⇐⇒ hx, yi = 0.
A basis v1 , · · · , vn for Rn is called orthogonal if every pair of distinct basis elements is
orthogonal, i.e.
{vj } is an orthogonal basis if hvj , vk i = 0 for j 6= k.
 MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 17

5. A preview of future topics

In the coming weeks, we will develop a method inspired by the above to solve
ut = −L[u] + h, x ∈ (a, b) and t > 0
with some initial condition u(x, 0) = u0 (x) and boundary conditions at x = a and x = b.
d 2
The differential operator L contains only x-derivatives, e.g L = − dx 2 . It will be the same

sort of operator we encountered in studying second-order linear equations.

We can identify some key step from Fourier’s heat conduction problem:
A) Find ‘eigenfunctions’ that solve a boundary value problem of the form
L[φn ] = λn φn , with some conditions on φ at x = a and x = b.
B) Represent the solution as a sum of coefficients (depending on time) times eigenfunc-
tions:
X∞
u= an (t)φn (x).
n=0

C) Find (simple) equations for the coefficients and solve them to get the solution.
To make (C) work, we will need the functions φn to have specific properties, namely, some-
thing like (7) that will make sure the equation for cn (t) decouples from the others. In
Fourier’s example, the equation is
a0n = −λn an , an (0) = coeff. of φn in the initial condition
and the ‘eigenvalue’ is λn = n2 .

In a sense, we are converting (complicated) linear PDEs into a linear algebra problem plus
an ODE problem. Doing so will require developing substantial theory in several areas:
• Vector spaces of functions, orthogonality: How do we make sense of series like
∞
X
cn φn (x)
n=0

and extend the notion of a basis to functions? Where does the condition (7) come
from? Essentially, we will extend familiar results from linear algebra in Rn to spaces
of functions.
• Boundary value problems / eigenvalue problems: What is the structure of
ODEs when the boundary conditions are at different points (not IVPs)? These
problems are quite different from the IVPs of previous weeks! We will also develop
theory for eigenvalue problems (what properties do the eigenvalues and eigenfunctions
have?).
• Important PDEs (what important equations can be solved this way?):
There are three essential PDEs (the heat equation, wave equation, and Laplace’s
equation) that can be studied using the framework we develop here. These three
equations describe a wide range of phenomena in physics, engineering, and more.
18 J. WONG (FALL 2017)

• Fourier series (an extremely important special case): The case where the
eigenfunctions are sines/cosines is of critical importance - with implications that go
far beyond their use as a tool for solving PDEs. We will look at such series (Fourier
series) in detail.