CHAPTER EIGHT

EXPECTATION
Content
8.1 Expectation of a random variable
8.2 Expectation of a function of a random variable
8.3 Properties of expectation
8.4 Variance of a random variable and its Properties
8.5 Moments and moment generating function
8.6 Chebyshev's inequality
8.7 Covariance and correlation coefficient
 EXPECTATION
 Most probability distributions are characterized by their mean and variance.
 The mean of a random variable is referred to as expectation, provided the
expected value converges to a constant.
 If we want to summarize a random variable by a single number, then this
number should undoubtedly be its expected value.
 The expected value, also called the expectation or mean, gives the center in
the sense of average value of the distribution of the random variable.
 If we allow a second number to describe the random variable, then we look
at its variance, which is a measure of spread of the distribution of the
random variable.
 Expectation of a random variable

 Definition (Expectation of a discrete random Variable)

 Let X be a discrete random variable with possible values x1, x2,

…, xn.

 Let p(xi) i=1, 2, 3…n.

 Then the expected value of X, denoted by E(X), is defined as:

E  X    xi pxi 

 Note: If the series converges absolutely, this number is also called

the mean of X.
• Example 1: A manufacturer produces items such that 10% are
defective and 90% are non-defective. If a defective item is produced
the manufacturer loses $1 and if a non-defective item is produced the
manufacturer gets $5 profit. Find the expected net profit of the
manufacturer per item.
 Solution : Expected value of a random variable is
E X    xi pxi 
 loss  pdeffetive    profit  pnondeffeti ve 
  1 0.10  5  0.90
 4.4

 On average a manufacturer will get a profit of $4.4

• Definition (Expectation of a continuous random Variable)
 Let X be a continuous random variable with possible values x1,

x2, …, xn.

 Let f(xi) i=1, 2, 3…n.

 Then the expected value of X, denoted by E(X), is defined as:

E X    xf x dx
Rx

 Note: If the series converges absolutely, this number is also called

the mean of X.
Example:
 Let X has a pdf
21  x , 0  x  1

f x   
0, Otherwise

 Find the expected value of X.

Solution: E  X    xf x dx
Rx
1
  2 x1  x dx
0
1
 2 
2
 x 1  x 
 3 0
1
3
 Expectation of a function of a random variable

Definition: Let X be a random variable, and g(x) be a function of the

random variable X whose domain and range is the real line.
• The expected value of g(x) is defined by

 g xi  p X  xi  if x is discrete

E X   
 g xi  f x dx

if x is continuous

8
 Exercise: Let X has a pdf given as follows:
21  x , 0  x  1

f x   
0, Otherwise
 What is E(X 2)
 Expectation of two-dimensional random
variable
• Definition: Let (X, Y) be a two-dimensional random variable
and Z = H(X, Y) be real-valued function of (X, Y) .The expected
value of the random variable Z is defined as follows:
a. If Z is discrete random variable with possible values z1, z2,…
and P(zi) = P(Z = zi) then

b. If Z is a continuous random variable with P.d.f f we have

Theorem: Let (X, Y) be a two-dimensional random variable and let
Z = H(x,y)

• Example 1: Assume that the current I and the resistance R are

independent continuous random variables with the following
P.d.f’s

The voltage Z = IR. Find E(Z)

• Solution:
• Since I and R are independent the joint pdf of I and R is

3 1 3 1
 2ir 2 
E ( Z )    irf (i, r )ir    ir  ir
0 0 0 0  9 
2 r 3 3

3
2 3 i 1 3 4
  r ( )r   
90 30 27  4 0  2
 Properties of Expectation
1. If C is any constant value, then E(C) = C
2. E(CX) = C ∗ E(X)
3. If X and Y are any two random variables, then E(X + Y) =
E(X) + E(Y)
4. If X and Y be two independent random variables, then E(XY)
= E(X) ∗ E(Y)
5. If a and b are any constant numbers, then E(a + bX)= a + bE(X)
6. E(X) > 0 if X > 0
7. |E(X)| ≤ E(|X|)

8. E(g1(x)) ≤ E(g2(x)) if g1(X) ≤ g2(X) for all x

 Variance of a random variable and its properties

 Definition: Let X be a random variable. The variance of X,

denoted by V(X) is defined as:

 That is, the mean value of the square of the deviations of X from
its mean is called the variance of X or the variance of the
distribution.
 The positive square root of V(X) is called the standard deviation
of X and it is denoted by s.d(X).
 It is worthwhile to observe that

 The standard deviation of X may be interpreted as a

measure of the dispersion of the points of the space
relative to the mean value E(X).
 Example 1: Let X is a discrete random variable with probability distribution

• What is the V(X)?

• Solution :

    
E X    xpx   0  1  1 3  2  3  3  1  1.5
8 8 8 8
E X    x px   0  1  1  3  2  3  3  1   3
2 2 2 2 2 2
8 8 8 8
V  X   E X 2  E  X 
2

 0.75
 Example 2: Let x has a pdf

• What is the V(X)?

 Solution 1
 x 1
E  X    x dx
1 
2 
1
1x x 3 2
   
2  3 2  1

 1  13 12   1   13  12 
         
 2  3 2   2  3 2 
1

3
 1
 x 1
 
E X 2   x2  dx
1  2 
1
1x 4
x 
3
   
2 4 3  1
 1  14 13   1   14  13 
         
 2  4 3   2  4 3 
1

3

 
V  X   E X 2  E  X 
2

2
1 1  2
   
3 3 9
 Properties of Variance
 Let X be a random variable

 For any real number k, Var(kX ) = k 2Var(X )

 The expression E[(X −k) 2] assumes its minimum value when k=E(X)

 If Y is another variable that is independent from X, then Var (X + Y)

= Var (X − Y) = Var(X) + Var(Y)
 If X and Y are not independent, i.e., if they are dependent then Var
(X + Y) = Var(X) + Var(Y) + 2(E(XY) – E(X)E(Y))
 The expression E(XY) – E(X)E(Y) is called the Covariance of X and
Y. Cov(X, Y) = E(XY) – E(X)E(Y)
 Note that: If X and Y are independent then Cov(X, Y) = 0
 Chebyshev’s Inequality
• Let X be a random variable with E(X) = µ and let C be any real
number. The, if E(X − C)2 is finite and e is any positive number,
we have

• The following forms are equivalent to the above equation are

immediate
a) By considering the complementary event we obtain

b) Choosing C = µ we obtain
c) Choosing C = µ and ε = kσ, where σ2= Var (X) > 0 we obtain

Covariance and Correlation Coefficient

• Definition (Correlation Coefficient): Let X and Y be two
random variables. The correlation coefficient of X and Y is
denoted by ρ and is defined as:

• Remarks:
• −1 ≤ ρ ≤ 1
• If ρ = −1 there is perfect inverse (negative) relationship or
dependence between the two variables X and Y.
• If -1 < ρ < 0, then there is inverse relationship (correlation)
between X and Y. The strength of the relationship will be
determined based on the magnitude of ρ. If - 0.5 < ρ < 0, then the
relationship is inverse-weak. On the other hand, if - 1 < ρ < - 0.5,
then the relationship is inverse-strong.
• If ρ = 0, then the two variable X and Y are uncorrelated, no linear
correlation or relationship. It has to be stressed that absence of
correlation is not a guarantee for independence. But, the converse
is true.
• If 0 < ρ < 1, then there is direct (positive) relationship between X
and Y. The strength of the relationship will be determined from
the magnitude of ρ. If 0 < ρ < 0.5, the relationship is direct-weak.
If, on the other hand, 0.5 < ρ < 1, the relationship is direct-strong.
• Finally if ρ =1 there is perfect direct relationship between the two
random variables X and Y.
• Example: Let the random variables X and Y have the joint P.d.f,

• Find the correlation coefficient of X and Y.

Solution:
• The result seems to suggest that, we have a weak inverse
relationship or correlation between X and Y.
 Conditional Expectation
• Definition (Conditional Expectation):
a) If (X, Y) is a two-dimensional discrete random variable, we
define the conditional expectation of X for given Y = yi as

b) If (X, Y) is a two-dimensional continuous random variable, we

define the conditional expectation of X for given Y = yi as

• Conditional expectation of Y for given X is defined analogously.

• Theorem:

• Proof:

• Hence
• Theorem: suppose that X and Y are independent random variable.
Then

• Example 1: Let

• Find the conditional expectation of X given Y = 1/2

• Solution:

• Hence
• Example 2: Consider the following joint probability distribution.

• Use the table to find the conditional probability functions and

conditional expectations of (i) Y given X=0; (ii) Y given X=1; and (iii)
Y given X=2.
Solution
• The table has the marginal probability functions of X and Y added to
it. You can see that once the joint probability mass function has been
given in a table it is easy to write down the marginal (we just sum the
rows and columns of the table).
• The conditional probability functions are obtained by dividing each
column by the column total.
• We can now use the conditional probability functions to find
the conditional expectations:
 Moments and moment generating function
• Definition (The rth Moment about the origin): If X is a random
variable, the rth moment of the random variable X, usually
denoted by , is defined as:

• Verbally, a moment is the expectations of the powers of the

random variable that has the given distribution. Moments are
defined for a random variable or a distribution.
• Definition (Central-Moments): The rth moment about the mean
of the random variable X, denoted by μr, is the expected value of
( X - μx)r; symbolically,

Remarks:
• The 1st central moment about the mean is zero.
• The second central moment about the mean is the variance of the
random variable.
• All odd moments of X about μx are 0 if the density function of X is
symmetrical about μx, provided that such moment exist.
• Example: Let X have the P.d.f

• Find a) the first moment about the origin

b) the second central moment
Solution:
 Moment Generating Functions
• Definition (Moment Generating Function): Let X be a random
variable with P.d.f f . The expected value of etx is defined to be
the moment generating function of X if in some interval –
ℎ<t<ℎ; ℎ > 0. The moment generating function, denoted by
Mx(t), is

• To explain why we refer to this function as “moment-generating”

function, let us substitute for etx its Taylor series expansion,
namely,
• For the discrete case, we thus get

• and it can be seen that in the Taylor series of moment generating

function of X the coefficient of is , the rth moment about the
origin of the random variable X. In the continuous random
variable the argument is the same.
• Notes:
1. From the above expression one can show that;

i.e, is the rth derivative of Mx(t) with respect to t evaluated at t = 0.

2. If a and b are constants, then

3. If X and Y are independent r.vs having moments generating

functions .... (..) and .... ( ..) respectively, then
4. In general, If X1, X2, X3,…, Xn are independent random variables
with moment generating function’s MXi(t); i= 1, 2, 3, …, n then
the moment generating function of their sum, MZ(t), where Z= X1
+ X2 + X3 + … +Xn, is given by

5. Uniqueness theorem: Suppose that X and Y are random variables

having moment generating function MX(t) and MY(t) respectively.
Then X and Y have the same probability distribution iff MX(t)=
MY(t) identically.
• Example: Find the moment generating function of the random
variable X having P.d.f

• Solution: