23

3. Week 3

Remark 3.1 (Shorthand notation for Pre-images). In the setting of Notation 2.39, we shall suppress
the symbols ω and use the following notation for convenience, viz.

X −1 (A) = {ω ∈ Ω : X(ω) ∈ A} = (X ∈ A).

For specific sets A, other notations, again for convenience, may be used. For example for

(a) If A = (−∞, x], then we write

X −1 (A) = (X ∈ A) = {ω ∈ Ω : X(ω) ∈ (−∞, x]} = {ω ∈ Ω : X(ω) ≤ x} = (X ≤ x).

For A = (−∞, x), (x, ∞), [x, ∞), we shall write X −1 (A) to be equal to (X < x), (X >
x), (X ≥ x) respectively.
(b) If A = {x}, then we write

X −1 (A) = (X ∈ A) = {ω ∈ Ω : X(ω) ∈ {x}} = {ω ∈ Ω : X(ω) = x} = (X = x).

Remark 3.2 (Properties of pre-images). Let X : Ω → R be a function. The following are some
properties of the pre-images under X, which follow from the fact that X is a function.

(a) X −1 (R) = Ω.
(b) X −1 (∅R ) = ∅Ω , where ∅R and ∅Ω denote the empty sets under R and Ω, respectively. When
there is no chance of confusion, we simply write X −1 (∅) = ∅.
(c) For any two subsets A, B of R with A ∩ B = ∅, we have X −1 (A) ∩ X −1 (B) = ∅.
c
(d) For any subset A of R, we have X −1 (Ac ) = (X −1 (A)) .
(e) Let I be an indexing set. For any collection {Ai : i ∈ I} of subsets of R, we have
! !
−1 −1 −1
X −1 (Ai ).
[ [ \ \
X Ai = X (Ai ), X Ai =
i∈I i∈I i∈I i∈I

The above properties shall be used frequently throughout the course.

24

Note 3.3. As discussed in Note 2.38, we now look at real valued functions defined on Ω, where Ω
is the sample space of a random experiment E. We shall also assume that (Ω, F, P) is a probability
space.

Definition 3.4 (Random variable or RV). Let (Ω, F, P) be a probability space. Any real valued
function X : Ω → R shall be referred to as a random variable or simply, an RV. In this case, we
shall say that X is an RV defined on the probability space (Ω, F, P).

Note 3.5. If F is taken to be 2Ω , we immediately have

X −1 (A) = (X ∈ A) ∈ F

for any subset A of R. If F is taken to be a smaller collection of subsets of Ω, then the above
observation may not hold for any arbitrary function X. Given such F, we then restrict our
attention to the class of functions X satisfying the above property and refer to them as RVs. It is
therefore important to specify F before we discuss RVs X.

Note 3.6. The probability function/measure P has not been used in the definition of an RV X.
We now discuss the role of P in analysis of RVs X.

Notation 3.7. We write B to denote the power set of R.

Notation 3.8. Let X be an RV defined on a probability space (Ω, F, P). Then for all A ∈ B, we
have X −1 (A) ∈ F and hence P(X −1 (A)) is well defined. We denote this in terms of a set function
P ◦ X −1 : B → [0, 1] given by P ◦ X −1 (A) := P(X −1 (A)) = P(X ∈ A), ∀A ∈ B. A shorthand
notation PX shall also be used to refer to P ◦ X −1 .

Notation 3.9. Similar to the discussion in Remark 3.1, we shall write P(X ≤ x), P(X = x) etc.
for P ◦ X −1 (A) where A = (−∞, x], {x} etc. respectively.

Proposition 3.10. Let X be an RV defined on a probability space (Ω, F, P). Then, the set function
P ◦ X −1 is a probability function/measure defined on the collection B.

Proof. We verify the axioms/properties of a probability function/measure as mentioned in Defini-

tion 1.33.
 25

We have P ◦ X −1 (R) = P(X −1 (R)) = P(Ω) = 1. Since P is a probability measure on F, we also

have P ◦ X −1 (A) = P(X −1 (A)) ≥ 0, ∀A ∈ B.
If {An }n is a sequence of pairwise disjoint sets in B, then {X −1 (An )}n is a sequence of pairwise
disjoint events in F. Hence,
∞ ∞ ∞ ∞
! !
−1 −1
P(X −1 (An )) = P ◦ X −1 (An ).
[ [ X X
P◦X An = P X (An ) =
n=1 n=1 n=1 n=1

This proves countable additivity property for P ◦ X −1 and the proof is complete. 

Definition 3.11 (Induced Probability Space and Induced Probability Measure). If X is an RV

defined on a probability space (Ω, F, P), then the probability function/measure P ◦ X −1 on B is
referred to as the induced probability function/measure induced by X. In this case, (R, B, P ◦ X −1 )
is referred to as the induced probability space induced by X.

Example 3.12. Recall from Remark 2.37, that if we toss a fair coin twice independently, then the
sample space is Ω = {HH, HT, T H, T T } with P({HH}) = P({HT }) = P({T H}) = P({T T }) = 41 .
Consider the RV X : Ω → R which denotes the number of heads. Here,

X(HH) = 2, X(HT ) = X(T H) = 1, X(T T ) = 0.

Consider the induced probability measure P ◦ X −1 on B. We have

1
P ◦ X −1 ({0}) = P(X −1 ({0})) = P({T T }) = ,
4
1
P ◦ X −1 ({1}) = P(X −1 ({1})) = P({HT, T H}) = ,
2
1
P ◦ X −1 ({2}) = P(X −1 ({2})) = P({HH}) = .
4
More generally, for any A ∈ B, we have

P ◦ X −1 (A) = P({ω : X(ω) ∈ A}) = P ◦ X −1 ({i}).

X

i ∈ {0,1,2}∩A

Remark 3.13. If we know the probability function/measure PX = P◦X −1 for any RV X, then we get
the information about all the probabilities P(X ∈ A), A ∈ B for events X −1 (A) = (X ∈ A), A ∈ B
26

involving the RV X. In what follows, our analysis of RV X shall be through the understanding of
probability function/measure PX = P ◦ X −1 on B.

Definition 3.14 (Law/Distribution of an RV). If X is an RV defined on a probability space

(Ω, F, P), then the probability function/measure PX = P ◦ X −1 on B is referred to as the law or
distribution of the RV X.

We now discuss some properties of a probability function/measure. To do this, we first introduce

a concept involving sequence of sets.

Definition 3.15 (Increasing and decreasing sequence of sets). Let {An }n be a sequence of subsets
of a non-empty set Ω.
(a) If An ⊆ An+1 , ∀n = 1, 2, · · · , we say that the sequence {An }n is increasing. In this case, we
S∞
say An increases to A, denoted by An ↑ A, where A = n=1 An .
(b) If An ⊇ An+1 , ∀n = 1, 2, · · · , we say that the sequence {An }n is decreasing. In this case,
T∞
we say An decreases to A, denoted by An ↓ A, where A = n=1 An .

Remark 3.16. An ↑ A if and only if Acn ↓ Ac .

Proposition 3.17 (Continuity of a probability measure). Let (Ω, F, P) be a probability space.

(a) (Continuity from below) Let {An }n be sequence in F, such that An ↑ A. Then

lim P(An ) = P(A).

n→∞

(b) (Continuity from above) Let {An }n be sequence in F, such that An ↓ A. Then

lim P(An ) = P(A).

n→∞

Proof. To prove the first statement. Since {An }n is an increasing sequence of sets, we have

An ∩ (A1 ∪ A2 ∪ · · · ∪ An−1 )c = An ∩ Acn−1 , ∀n ≥ 2.

Then using a hint from practice problem set 1, we have

∞ ∞
!
Acn−1 )
[ [
An = A1 ∪ (An ∩ .
n=1 n=2
 27

Since the sets A1 , A2 ∩ Ac1 , A3 ∩ Ac2 , · · · are pairwise disjoint, using the countable additivity of P,
we have
∞ ∞ k
!
P(An ∩ Acn−1 ) = P(A1 ) + lim P(An ∩ Acn−1 )
[ X X
P An = P(A1 ) +
k→∞
n=1 n=2 n=2
k
X
= P(A1 ) + lim [P(An ) − P(An−1 )]
k→∞
n=2

= P(A1 ) + lim [P(Ak ) − P(A1 )] = lim P(Ak ).

k→∞ k→∞

This completes the proof of the first statement.

T∞
To prove the second statement. First observe that Acn ↑ Ac with A = n=1 An . Using the first
statement, we have

P(A) = 1 − P(Ac ) = 1 − lim P(Acn ) = lim [1 − P(Acn )] = lim P(An ).

n→∞ n→∞ n→∞

The proof is complete. 

Definition 3.18 (Distribution function of an RV). Let X be an RV defined on a probability space

(Ω, F, P) with law/distribution PX . Consider the function FX : R → R defined by FX (x) :=
P ◦ X −1 ((−∞, x]) = P(X ≤ x), ∀x ∈ R. The function FX is called the cumulative distribution
function (CDF) or simply, the distribution function (DF) of the RV X.

Remark 3.19 (RVs equal in law/distribution). Let X be an RV defined on a probability space

(Ω, F, P) and let Y be an RV defined on a probability space (Ω0 , F 0 , P0 ). If P ◦ X −1 = P0 ◦ Y −1 , i.e.
P ◦ X −1 (A) = P0 ◦ Y −1 (A), ∀A ∈ B, then we say that X and Y are equal in law/distribution. In
this case, FX = FY , i.e. FX (x) = FY (x), ∀x ∈ R.

Remark 3.20. Let X and Y be two RVs, possibly defined on different probability spaces. If FX =
FY , then it can be shown that X and Y are equal in law/distribution. The proof of this statement
is beyond the scope of this course. This statement is often restated as ‘the DF of an RV uniquely
determines the law/distribution of the RV’.
28

Example 3.21 (The DF of a constant RV). Let c ∈ R and let X : Ω → R be given by X(ω) :=
c, ∀ω ∈ Ω. Then, for all x ∈ R, we have

∅, if x < c,


(X ≤ x) = {ω ∈ Ω : X(ω) ≤ x} = 
Ω,
 if x ≥ c.

Therefore, the DF of the RV X is given by

 
if x < c, 0, if x < c,
 
P(∅),
 
FX (x) = P(X ≤ x) = = .
if x ≥ c 1, if x ≥ c.
 
P(Ω),
 

Example 3.22 (The DF of a two-valued RV). Let A ⊂ Ω and let X : Ω → R be given by

1, if ω ∈ A,


X(ω) := 1A (ω) = . Then, for all x ∈ R, we have
0, if ω ∈ Ac







 ∅, if x < 0,


(X ≤ x) = {ω ∈ Ω : X(ω) ≤ x} = A c
, if 0 ≤ x < 1,




Ω, if x ≥ 1.



Therefore, the DF of the RV X is given by

 



 P(∅), if x < 0, 


0, if x < 0,

 

FX (x) = P(X ≤ x) = P(Ac ), if 0 ≤ x < 1, = P(Ac ), if 0 ≤ x < 1, .

 

 
if x ≥ 1 1, if x ≥ 1

P(Ω), 

Example 3.23. Consider X as in Example 3.12. Then for all x ∈ R, we have


0, if x < 0,






PX ({0}), if 0 ≤ x < 1,


X
FX (x) = PX ((−∞, x])) = PX ({i}) = 
i ∈ {0,1,2}∩(−∞,x] 


 PX ({0}) + PX ({1}), if 1 ≤ x < 2,



P ({0}) + P ({1}) + P
X ({2}), if x ≥ 2.

X X
 29

Therefore, 



 0, if x < 0,



1, if 0 ≤ x < 1,


4
FX (x) =
3
4, if 1 ≤ x < 2,







1, if x ≥ 2.

