BSc and MSci EXAMINATIONS (MATHEMATICS)

May 2012

This paper is also taken for the relevant examination for the Associateship of the Royal College of
Science.

M3/4/5 S4

Applied Probability
Date: Thursday, 10th May 2012 Time: 10 am – 12 noon

Credit will be given for all questions attempted but extra credit will be given for complete or nearly
complete answers.
Calculators may not be used.

c 2012 Imperial College London M3/4/5 S4 Page 1 of 5

1. (a) Suppose {Nt }t≥0 is a counting process. Give a formal definition, if it is to be a
homogeneous Poisson process of rate λ > 0.

(b) Let {Nt }t≥0 denote a homogeneous Poisson process of rate λ > 0. Let X1 denote the
corresponding first inter–arrival time. Show that X1 is exponentially distributed with
parameter λ.

(c) Let {Nt }t≥0 be a homogeneous Poisson process of rate λ > 0. Let 0 < t1 < t2 < t3 and
x1 ≤ x2 ≤ x3 for x1 , x2 , x3 ∈ {0, 1, 2, . . . } . Show that

tx1 1 (t2 − t1 )x2 −x1 (t3 − t2 )x3 −x2

P(Nt1 = x1 , Nt2 = x2 , Nt3 = x3 ) = exp(−λt3 )λx3 .
x1 !(x2 − x1 )!(x3 − x2 )!

(d) You go on a summer holiday. Your airline has two check–in desks at the airport. The
service times are independent and exponentially distributed with parameter λ > 0. When
the check–in starts at 10am, you get to the check–in area with two other tourists. You
are generous and let the two other tourists proceed to be served. You will then be served
by the next available check–in desk. What is the probability that, of the three tourists,
you will be the last to leave the check–in area?
Hint: There is a clever (and quick!) solution to the problem. Alternatively, express the
event in question in terms of three independent random variables which are exponentially
distributed with parameter λ.

M3/4/5 S4 Applied Probability (2012) Page 2 of 5

2. (a) Suppose {Nt}t≥0 is a counting process. Give a formal definition, if it is to be a non–
homogeneous Poisson process with intensity function λ(t), t ≥ 0.

For (b)-(d) assume that N = {Nt }t≥0 is a non–homogeneous Poisson process with intensity function
λ(t), t ≥ 0.

(b) Let pn (t) = P(Nt = n) for n ∈ {0, 1, 2, . . . }. Show that the forward equations are given
by

dp0 (t)
= −λ(t)p0 (t),
dt
dpn (t)
= −λ(t)pn (t) + λ(t)pn−1 (t), for n ≥ 1.
dt

(c) Compute p0 (t) and p1 (t).

Hint: Recall that a one–dimensional ordinary differential equation

df (t)
+ α(t)f (t) = g(t), t≥0
dt
with continuous functions α, g and initial condition f (0) = C has solution
Rt
g(u)M(u)du + C
f (t) = 0 ,
M(t)

where M is the integrating factor

Z t 
M(t) = exp α(u)du .
0

(d) Let X1 , X2 denote the first two inter–arrival times of N, i.e. X1 is the time from 0
to the first event, and X2 is the time from the first event to the second event. Define
Rt
m(t) := 0 λ(s)ds.
(i) Derive the probability density function of X1 ;
(ii) Derive the conditional probability density function of X2 |X1 ;
(iii) Hence, find the marginal probability density function of X2 , leaving your answer as
an integral.

M3/4/5 S4 Applied Probability (2012) Page 3 of 5

3. (a) Let {Xn }n∈{0,1,2,... } denote a discrete–time stochastic process taking values in a state
space E ⊆ Z. Under which condition is {Xn }n∈{0,1,2,... } a Markov chain on E?

(b) Let Xn be the minimum observation obtained in the first of n rolls of a fair die (for
n ∈ {1, 2, 3, . . . }). Show that {Xn }n∈{1,2,3,... } is a Markov chain, and give the transition
probabilities.

(c) Consider a Markov chain with state space E = {1, 2, 3, 4, 5} and transition matrix

 
1/2 1/2 0 0 0

 1 0 0 0 0 

P= 0 1/3 1/3 1/3 0 .
 
 
 0 0 0 0 1 
0 0 0 1/2 1/2

(i) Specify the communicating classes and determine whether they are transient, null
recurrent or positive recurrent.
(ii) Is the Markov chain irreducible?
(iii) Let π denote a 5–dimensional row vector. Which conditions have to be satisfied in
order for π to be a stationary distribution of the Markov chain?
(iv) Find a stationary distribution for the Markov chain.
(v) Decide whether or not the stationary distribution is unique and justify your answer.

M3/4/5 S4 Applied Probability (2012) Page 4 of 5

4. (a) Let {Xt}t≥0 denote a continuous–time stochastic process taking values in a state space
E ⊆ Z. Under which condition is {Xt }t≥0 a Markov chain on E?

(b) Let X = {Xt }t≥0 denote a continuous–time Markov chain on the state space E =
{0, 1, . . . }. Suppose X is a birth–death process with birth rates λ0 , λ1 , . . . and death
rates µ0 , µ1 , . . . satisfying
λi , ≥ 0 µi ≥ 0, for all i ∈ E, µ0 = 0.
State the infinitesimal transition probabilities
P(Xt+δ = n + m|Xt = n)
for n, m ∈ {0, 1, 2, . . . } and t ≥ 0, δ > 0.

(c) What is the generator G of the birth–death process defined in (b)?

(d) Let N = {Nt }t≥0 denote a pure birth process with N0 = 0 and
P(one event happens in (t, t + δ]|Nt is odd) = αδ + o(δ),
P(one event happens in (t, t + δ]|Nt is even) = βδ + o(δ),
for t ≥ 0, δ, α, β > 0.
(i) Derive the forward equations for pn (t) := P(Nt = n) for n ∈ {0, 1, 2, . . . }.
(ii) Find the following probabilities:
Pe (t) := P(Nt is even), Po (t) := P(Nt is odd).
Hint 1: Derive the following differential equations
Pe′(t) = αPo (t) − βPe (t), Po′ (t) = −αPo (t) + βPe (t),
and solve them by using the identity Pe (t) + Po (t) = 1.

Hint 2: Recall that a one–dimensional ordinary differential equation

df (t)
+ α(t)f (t) = g(t), t≥0
dt
with continuous functions α, g and initial condition f (0) = C has solution
Rt
g(u)M(u)du + C
f (t) = 0 ,
M(t)
where M is the integrating factor
Z t 
M(t) = exp α(u)du .
0

M3/4/5 S4 Applied Probability (2012) Page 5 of 5