MA201: Further Mathematical Methods (Linear Algebra) 2002

General Information — Teaching

This course involves two types of teaching session that you should be attending:

Lectures
This is a half unit course with lectures in the Lent Term only. Lectures take place on Tuesdays
12:00-13:00 and Fridays 13:00-14:00 in the New Theatre (E171). There will also be revision lectures
in the Summer Term.
The lecturer for this course is Dr. James Ward. He has office hours on Tuesdays 15:30-16:30
and Fridays 14:30-15:30 in his office, Room B414 (on the fourth floor of Columbia House). If these
times are not convenient for you, then you can e-mail him at j.m.ward@lse.ac.uk to arrange an
appointment.

Classes
Classes start in the second week of the Lent Term and finish in the first week of the Summer Term.
Students will be assigned to a particular group and you should attend the meetings of the group to
which you have been assigned.
The Class Teachers are Dr. James Ward, Dr. Elizabeth Boardman and Dr. David Cartwright.
The object of the classes is to go through the Problem Sheets that will be distributed during lectures.
Anyone having difficulties with the course is strongly encouraged to go and see their Class Teacher
during their Office Hours.

General Information — Assessment

There are two ways in which you will be assessed during this course:

Problem Sheets
Problems will be set each week. Students are expected to attempt selected questions and hand them
in to their Class Teachers by the times that they specify. Work that is handed in will be marked,
graded and returned in the next class. The grades gained for this work will provide a means of
informally assessing the progress of each student.
As with any mathematics course, learning is achieved by doing, and so it is vital that you attempt
the Problems. Most of the questions which are set for the classes will be routine, and their purpose
is to ensure that you have understood the material covered in the lectures. However, some of the
set questions will be harder, and these serve to stretch your understanding. Solution Sheets will be
made available in the week following the classes.
The Problem Sheets will also contain ‘Other Problems’ which you should attempt after the classes
to see if you have learnt from your mistakes, and ‘Harder Problems’ for those of you who find the
standard material a bit too easy. These will not be graded, but full answers will be given on the
Solution Sheets.

The Examination
The formal assessment for this course consists of a two hour examination in the Summer Term. (Note
that there are separate examinations for the Calculus and Linear Algebra components of the Further
Mathematical Methods course.) The examination will have questions on the theory, methods and
applications contained in the course. Consequently, to do well, you will need to have a good grasp
of all of these aspects of the material. At the end of the course, a summary of the material which is
needed for the examination will be provided.

General Information — The Course

Here is some more specific information about the course:

1
Prerequisites
Most of the students who enter into this course have already done Mathematical Methods (MA100),
and the material we shall cover follows on from this course. Students who have taken Quantitative
Methods (MA107) and Further Quantitative Methods (MA207) will also be able to take this course,
but such students should be aware that some of the assumed theory will be less familiar to them
(specifically, the abstract theory of vector spaces and linear transformations). Other students, most
notably those following a General Course programme, should see the lecturer if they are unsure
whether they have the appropriate background.

Course Objectives
The course has three main objectives and these are to develop an understanding of theory, methods
and applications within the context of Linear Algebra. To this end, although we shall mainly be
considering matrices and their properties, we begin by noting that most of our knowledge about
these entities relies on an understanding of vector spaces. As such, the theoretical part of this course
will examine this mathematical structure and use it as a theme which underlies most of the theoretical
material that we shall cover. In particular, we shall see how simple definitions can be combined and
our mathematical knowledge extended via the use of proofs. With this theory in place, we shall
derive some of the more important properties of matrices. However, it is not always convenient to
use such basic notions when dealing with matrices and so we shall also derive methods that will allow
us to analyse matrices without recourse to the underlying theory. These methods, in turn, will allow
us to illustrate the utility of matrices as a mathematical tool. In particular, we shall develop some
applications of matrices to problems in Economics and the Social Sciences. Moreover, we will see
that these methods allow us to obtain qualitative as well as quantitative results.

Textbooks
There are two textbooks that are recommended for this course. For the applications and the earlier
parts of the course I recommend H. Anton and C. Rorres, Elementary Linear Algebra: Applications
Version (7th edition, Wiley, 1994) which was also the course textbook for the Linear Algebra part
of Mathematical Methods (MA100). However, for the later parts of the course, the recommended
textbook is Advanced Mathematical Methods by Adam Ostaszewski, which is also used for the Cal-
culus part of Further Mathematical Methods (MA200). There are many other good Linear Algebra
textbooks about and so if the recommended ones are not to your taste, you can also go to the library
and browse.

General Information — Course Content

The course is divided into three parts. What follows is a rough summary of the material that we will
be covering and how the different topics relate to each other. References to the course textbooks,
Ostaszewski (O) and Anton and Rorres (AR) are given in brackets.

1 Vector spaces I
Vector spaces are a type of mathematical structure, and you were introduced to them in Mathematical
Methods (MA100). As they are fundamental to many of the topics that will be covered in this course,
we shall use them to motivate some of the mathematical ideas that we will encounter. In particular,
we shall introduce three fundamentally different vector spaces, examine their properties and then use
these in some applications. With this in mind, the first part of the course is a revision (and slight
extension) of some things that you should know about vector spaces. (An overview of some of the
more basic aspects of this material can be found in Chapters 3 and 4 of Anton and Rorres.)

1.1 Vector spaces, subspaces, bases and dimension

We start by formally introducing the concepts of vector space and subspace. Three examples of
vector spaces will be given (real, complex and function space), and some subspaces will be examined.
Some properties of linear combinations of vectors will be revised, including the notions of linear

2
independence and linear span. The fact that linear spans are vector spaces will then be explored,
and with the concept of linear independence, bases will be introduced. The dimension of a vector
space will also be defined. Where possible, the more theoretical material will also be explained in a
more intuitive geometrical manner. (O §§1.1-4, AR §§5.1-4 and 10.1-4)

1.2 Linear transformations and related vector spaces

Linear transformations will be introduced along with their matrix representation. Associated with
such matrices are several vector spaces, namely:

• Row and column spaces.

• Ranges and null-spaces.

These will be examined, and relationships between them will be emphasised. In particular, we shall
see that:

• The column space and the range of a matrix are identical.

• The dimensions of the range and the null-space of a matrix will be related by the rank-nullity
theorem.

• Certain linear transformations can represent a change of basis.

• Solutions to linear simultaneous equations will be analysed in terms of the range and null-space.

A geometrical interpretation of some of these results will also be given where appropriate. (O §§1.5-7
and 3.1-5, AR §§5.5-6, 6.5 and 8.1-5)

1.3 Tests for linear independence

Some tests for linear independence will be introduced. In particular, using determinants to test for
linear independence in a real space, and the use of the Wronskian in function spaces. (O §1.8 , AR
§5.3)

1.4 An introduction to geometry in Rn

We shall see that vector subspaces correspond to the solution set of a matrix equation Ax = 0. We
shall also introduce the idea of an affine set, which is the solution set of a matrix equation Ax = b.
This will allow us to introduce the notion of a hyperplane which is an (n−1)-dimensional geometrical
object in Rn . (O §§2.1-2)

1.5 Inner product spaces

Introducing the notion of an inner product, which we can use to generalise our normal idea of the
‘distance’ between two vectors, we get an inner product space. The defining properties of an inner
product will be stated and some examples will be given for the three main vector spaces being
considered in this course. The Cauchy-Schwarz Inequality will be derived and the notion of ‘angle’
will be introduced, the concept of orthogonality will also be revised. Some other results relating to
inner products will be derived, including the Generalised Theorem of Pythagoras and the Triangle
Inequality. (O §§2.3-7, AR §§6.1-2 and 10.5)

1.6 Orthonormal bases and the Gram-Schmidt procedure

The idea of an orthonormal basis is introduced and the Gram-Schmidt procedure for constructing
such bases is described. (O §§2.7-8, AR §6.3)

3
2 Spectral Theory
In Linear Algebra, spectral theory is mainly concerned with the eigenvalues and eigenvectors of
matrices. Spectral theory can be used in a wide variety of applications, mainly through the use of
diagonalisation, and some of these will be investigated.

2.1 Eigenvalues and eigenvectors

We start by revising the methods for calculating the eigenvalues and eigenvectors of a matrix which
you would have seen in Mathematical Methods (MA100). Some explanation of the vector spaces
that are involved in spectral theory due to the presence of eigenvectors will also be given. We shall
then introduce the notion of similar matrices and revise how to diagonalise matrices. (O §§5.1-2, AR
§§7.1-2)

2.2 Applications of diagonalisation I — systems of differential equations

We shall consider a simple model of population dynamics, and use diagonalisation to solve it. Then,
by introducing systems of differential equations, we will construct a slightly more sophisticated model
(based around the idea of ‘competing species’). This will be used to illustrate the idea of steady state
solutions to systems of differential equations, and the stability of these solutions will be discussed.
An elementary account of the linearisation of such systems of differential equations will also be given.
(AR §9.1)

2.3 Applications of diagonalisation II — [integer] powers of matrices

Multiplying a matrix with itself an arbitrary number of times is a chore, but this task can be massively
simplified using diagonalisation. We revise this technique and use it to create a model for age-specific
population growth. (AR §11.18)

2.4 Complex matrices

To complement our study of complex vector spaces, we now introduce the idea of a complex matrix
— i.e. a matrix that has entries which are complex numbers. We find that these can be divided into
several classes, and examine their spectral properties. In particular, we look at

• Hermitian matrices, showing that they have real eigenvalues and that eigenvectors correspond-
ing to distinct eigenvalues are orthogonal

• Unitary and Normal matrices, showing how these lead to the idea of unitary diagonalisation.

We also show how to calculate the spectral decomposition of a matrix, and use this to calculate
[arbitrary] powers of matrices. The real matrix analogues of these results will also be discussed. (O
§§5.3-4, AR §10.6)

3 Vector spaces II
We now continue our discussion of vector spaces by looking at orthogonal complements, orthogonal
projections and some applications.

3.1 Orthogonal complements

We introduce the idea of the orthogonal complement of an arbitrary set of vectors. In the case where
this set of vectors forms a vector space, we show that ‘the complement of the complement of the set
is the set itself’. We also look at the orthogonal complement of the null-space of a matrix and relate
it to the range of its transpose. (O §§3.7-8)

4
3.2 An application of orthogonal complements — bounds on matrix products
Using the results obtained about the range and null-space of a matrix and its transpose, we derive
some interesting results concerning the rank of certain matrix products. (O §§3.8-9)

3.3 Direct sums

We introduce the idea of the sum and direct sum of two subspaces. In particular, we show that the
sum of two subspaces is itself a subspace. (O §4.1)

3.4 Projections
With reference to the idea of a sum, we introduce the notion of a projection. This is a linear
transformation between two subspaces, and the nature of this mapping is considered. Under certain
circumstances (i.e. when the projection is symmetric and idempotent), we see that the projection is in
fact an orthogonal projection. The properties of this special class of projections are also investigated,
including the idea of ‘projection onto a column space’. (O §§4.1-6, AR §6.3-4)

3.5 An application of orthogonal projections in real space— least squares fits

We deduce the common statistical method of ‘least squares fits’ using an orthogonal projection and
some of its properties. (O §4.7, AR §9.3)

3.6 An application of orthogonal projections in function space — Fourier series

We consider the possibility of approximating functions by using linear combinations of orthonormal
functions. This leads to the derivation of the formulae for Fourier series in the case where the
orthonormal functions are sines and cosines. (AR §9.4)

3.7 Generalised inverses

You are all aware that non-singular square matrices have a unique inverse (of course, singular matrices
have no inverse). We briefly introduce the idea of a generalised inverse, which is [unsurprisingly] a
generalisation of the ‘inverse’ concept to non-square and singular matrices. Some properties of the
generalised inverse are investigated, and we deduce the fact that strong generalised inverses represent
orthogonal projections. (O §§8.3-5)

The web-site
More information about the course and copies of the handouts can be obtained from the Department
of Mathematics web-site. The URL is http://www.maths.lse.ac.uk/Courses/ma201.html.