Chapter 1

Probability

1.1 Introduction
The subject of probability had its origin in connection with games of chance,
such as flipping coins, rolling dice or playing cards. It began in the 17th Cen-
tury through efforts of such mathematicians as Fermat, Laplace and Pascal
to answer questions concerning games of chance. In the 20th Century a rig-
orous mathematical theory of probability based on axioms, definitions and
theorem was developed. Through times, probability theory found its way
into many applications- from engineering and science to business, medicine,
psychology and politics.

Statistics originated much earlier than probability and dealt mainly with
collection, organization, presentation and interpretation of chance outcomes
that occur in a planned study or scientific investigation. With the advent
of probability it was realized that statistics could be used in drawing valid
conclusions and making reasonable decisions on the basis of analysis of data,
such as in sampling theory, prediction or forecasting.

1.2 Random Experiments, Outcomes and Events

Games of chance such as rolling a die or flipping a coin are examples of “ran-
dom experiments.”

A random experiment or random observation- briefly, an experiment or ob- Random

Experi-
servation is a process that can be repeated a number of times, essentially ment
under the same conditions, and whose result cannot be predicted beforehand.

1
 Class Notes on
1.3. VENN DIAGRAM Applied Probability and Statistics ECEG-342

A single performance of an experiment is called a trial and each possible Trial,

Outcome
result an outcome of the trial.

We call the set of all possible outcomes of an experiment the Sample Space Sample
Space
S and each outcome an element or a point of S.

Example 1.1: In flipping a coin, the possible outcomes are H (head) or T (tail),
so that
S = {H, T }
In rolling a die,
S = {1, 2, 3, 4, 5, 6}
In measuring the humidity (in percent) in an air-conditioned room, any outcome
between 1 and 100 is theoretically possible, so that S is the interval from 0 to 100
and is thus an infinite set.

In most practical problems we are not interested in the individual outcomes

but in whether an outcome belongs (or does not belong) to a certain set of
outcomes. Such a set A ia a subset of the sample space S and it is called an Event
event.
An outcome is an event- rather a special event, called an elementary event.
Similarly, the entire space S is another special event.

Example 1.2: In rolling a die, some possible events are:

E1 : The die turns up an even number
E2 : The die turns up a number not smaller than 4

1.3 Venn Diagram

A sample space S and the events of an experiment can be represented graph-
ically by a Venn diagram. We let the sample space be a rectangle and Venn Di-
agram
represent events by circles drawn inside the rectangle.

The complement of an event A with respect to S is the set of all elements Complement
(outcomes) of S that are not in A. It is denoted by A0 .

An event containing no element is called the impossible event or empty

event and is denoted by ∅.

Murad Ridwan, 2 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.3. VENN DIAGRAM Applied Probability and Statistics ECEG-342

The intersection of two events A and B, denoted by A ∩ B, is the event Intersection

containing all elements that are common to A and B.

Two events A and B are mutually exclusive or disjoint if A ∩ B = ∅. Mutually

Exclusive

The union of the two events A and B, denoted by A ∪ B, is the event con- Union
taining all the elements that belong to A or B or both.

The difference of A and B, denoted by A − B, is the set consisting of all

elements of A which do not belong to B. Difference

Example 1.3: A die is rolled and consider the events:

A = Number greater than 3

B = Number less than 6
C = Even number

Use a Venn diagram to represent and determine the events A ∩ B, B ∩ C, C ∩ A, A ∩

B ∩ C, A ∪ B, A ∪ B ∪ C, A0 ∩ C, C − B.

Exercise 1.1: Using a Venn diagram, verify the following identities.

1. A ∩ ∅ = ∅

2. A ∪ ∅ = A

3. A ∩ A0 = ∅

4. A ∪ A0 = S

5. S 0 = ∅

6. ∅0 = S

7. (A0 )0 = A

8. (A ∩ B)0 = A0 ∪ B 0 - De Morgan’s first law

9. (A ∪ B)0 = A0 ∩ B 0 - De Morgan’s second law

10. A = (A ∩ B) ∪ (A ∩ B 0 )

11. A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C)

12. A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C)

13. A − B = A ∩ B 0

Murad Ridwan, 3 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.4. COUNTING TECHNIQUES Applied Probability and Statistics ECEG-342

Note: Principle of Duality: Any true result involving sets is also true if we
replace ∅ by S, S by ∅, ∪ by ∩, ∩ by ∪, sets by their complements whenever they
occur. Thus A ∪ ∅ = A is the dual of A ∩ S = A and A ∪ S = S is the dual of
A ∩ ∅ = ∅.

1.4 Counting Techniques

In many cases we shall be able to solve a probability problem by counting the
number of points in the sample space without actually listing each elements.
The basic rules of counting can help us in solving a tremendous variety of
problems.

The Sum Rule

If a first task can be done in n1 ways and a second task in n2 ways, and if
these tasks cannot be done at the same time, then there are n1 + n2 ways to
do either task.

Example 1.4: A student can choose a computer project from one of three lists.
The three lists contain 10, 15 and 29 possible projects, respectively. How many
possible projects are there to choose from?

The Product Rule

If a task can be done in n1 ways, and if for each these a second task can be
done in n2 ways, then the two tasks can be done together in n1 n2 ways.

Example 1.5: How many sample points are in a sample space when a pair of dice
is thrown once?

Example 1.6: How many different bit strings are there of length seven?

More complicated counting problems can be solved by using both the sum
and product rules.

Example 1.7: Each user on a computer system has a password, which is six to
eight characters long, where each character is an uppercase letter (total 26) or a
digit (total 10). Each password must contain at least one digit. Each password
must contain at least one digit. How many possible passwords are there?

Murad Ridwan, 4 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.4. COUNTING TECHNIQUES Applied Probability and Statistics ECEG-342

Counting problems can be solved using tree diagrams. A tree consists of a Tree Dia-
gram
root, a number of branches leaving the root, and possible additional branches
leaving the endpoints of other branches. We use a branch to represent each
possible choice, and represent the possible outcomes by the leaves, which are
the endpoints of branches not having other branches starting at them.

Example 1.8: An experiment consists of flipping a coin and then flipping it a

second time if a head occurs. If a tail occurs on the first flip, then a die is tossed
once. Construct a tree diagram to list the sample space.

Example 1.9: Suppose that three items are selected at random from a manufac-
turing process. Each item is inspected and classified defective, D, or non defective,
N. Construct a tree diagram and find S.

Exercise 1.2:
1. A multiple-choice test contains 10 questions. There are four possible answers
for each question.

(a) In how many ways can a student answer the questions on the test if
every question is answered?
(b) In how many ways can a student answer the questions on the test if
the student can leave answers blank?

Answer: a. 410 b. 510

2. How many different three-letter initials are there that begin with an A?
Answer: 276

3. How many strings of three decimal digits

(a) do not contain the same digits three times?

(b) begin with odd digits?
(c) have exactly two digits that are 4?

Answer: a. 990 b. 500 c. 27

4. In how many ways can a photographer at a wedding arrange six peoples in

a row, including the bridge and groom, if

(a) the bridge must be next to the groom?

(b) the bridge is not next to the groom?
(c) the bridge is positioned somewhere to the left of the groom?

Answer: a. 240 b. 480 c. 360.

Murad Ridwan, 5 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.4. COUNTING TECHNIQUES Applied Probability and Statistics ECEG-342

5. How many bit strings of length eight contain either three consecutive 0s or
four consecutive 1s?
Answer: 147.

Permutations
Frequently, we are interested in a sample space that contains as elements
possible orders or arrangements of a group of objects. The different arrange-
ments are called permutations.

Theorem 1. The number of permutations of n distinct objects is n!

Exercise 1.3: Prove the above theorem.

Exercise 1.4: List the possible permutations of the letters a, b and c.

Theorem 2. The number of permutations of n distinct objects taken r at a

time is
n!
n Pr =
(n − r)!

Exercise 1.5: Prove the above theorem.

Example 1.10: Two lottery tickets are drawn from 20 for first and second prizes.
Find the number of sample points in the space S.

Theorem 3. The number of permutations of n distinct objects arranged in

a circle is (n − 1)!

Exercise 1.6: Prove the above theorem.

Theorem 4. The number of distinct permutations of n objects of which n1

are of one kind, n2 of a second kind,· · · , nk of a k th kind is
n!
n1 !n2 ! · · · nk !

Exercise 1.7: Prove the above theorem

Example 1.11: How many different ways can 3 red, 4 yellow, and 2 blue balls be
arranged in a row?

Murad Ridwan, 6 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.4. COUNTING TECHNIQUES Applied Probability and Statistics ECEG-342

Combination
In many problems we are interested in the number of ways of selecting r
objects from n without regard to order. These selections are called combi-
nations. Combinations can occur without repetitions or with repetitions.

Theorem 5. The number of combinations of n distinct objects taken r at a

time, without repetition, is
 
n n Pr n!
n Cr ≡ = =
r r! r!(n − r)!
and the number of those combinations with repetitions is
 
n+r−1
r

Exercise 1.8: Prove the above theorem.

Note that for large n, the factorial can be approximated using Stirling for-
mula √  n n
n! ∼ 2πn
e

Exercise 1.9: The Binomial Coefficients

The binomial coefficients are defined by the formula
 
n n(n − 1)(n − 2) · · · (n − k + 1)
=
k k!
and appear as a coefficient in the expansion of (x + y)n , where n is an integer.
n  
n
X n n−k k
(x + y) = x y
k
k=0

The binomial coefficients have many interesting properties. Prove the following
   
n n
1. =
k n−k
     
n n n+1
2. + = - recursive relation
k k+1 k+1
     
n+1 n n
3. = + - Pascal’s identity
k k−1 k

Murad Ridwan, 7 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.4. COUNTING TECHNIQUES Applied Probability and Statistics ECEG-342

 
Pn n k
4. k=0 k = 2

    
m+n Pr m n
5. = k=0
r r−k k
 
Pn k n
6. k=0 (−1) =0
k
   
Pr n+k n+r+1
7. k=0 =
k r
   
2n n
8. =2 + n2
2 2
101 99 200
9. What is the coefficient
 of  x y in the expansion of (2x − 3y) ?
200
Answer: −2101 399
99
Exercise 1.10:

1. How many distinct permutations can be done from the letters of the word
infinity?
Answer: 3360.

2. A group contains n men and n women. How many ways are there to arrange
these people in a row if men and women alternate?
Answer: 2(n!)2

3. In how many ways can a set of five letters be selected from the English
alphabet?
Answer: 65780.

4. The English alphabet contains 21 consonants and five vowels. How many
strings of six lowercase letters of the English alphabet contain

(a) exactly 1 vowel?

(b) exactly 2 vowels?
(c) at least 1 vowel?
(d) at least 2 vowels?

Answer: a. 122 523 030 b. 72 930 375 c. 223 149 655 d. 100 626 625.

Murad Ridwan, 8 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.5. PROBABILITY OF AN EVENT Applied Probability and Statistics ECEG-342

1.5 Probability of an Event

Probability of an event is a measure of the chance with which we can expect
the event to occur in a given random experiment. It is assigned a number
between 0 (event will not occur) and 1 (event sure to occur.)
Probabilities can be assigned through classical or axiomatic approaches.

Classical Approach
1. Priori Definition: If an event can occur in m different ways out of
a total number of n possible ways, all of which are equally likely, then
the probability of the event is m/n.

2. Frequency or Posteriori Definition: If after n repetitions of an

experiment, where n is very large, an event is observed to occur in m
of these, then the probability of the event is m/n.

Axiomatic Approach
Axiom 1. If A is any event in a sample space S, then

0 ≤ P (A) ≤ 1

where P (A) is the probability of the event A.

Axiom 2. The entire sample space S has the probability

P (S) = 1

Axiom 3. If A1 , A2 , · · · are mutually exclusive events, then

P (A1 ∪ A2 ∪ · · · ) = P (A1 ) + P (A2 ) + · · ·

From the above axioms, we can obtain a number of useful theorems.

Theorem 6. If A1 ⊂ A2 then

P (A1 ) ≤ P (A2 )

and
P (A2 − A1 ) = P (A2 ) − P (A1 )
Theorem 7. P (∅) = 0
Theorem 8. P (A0 ) = 1 − P (A)

Murad Ridwan, 9 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.6. ASSIGNMENT OF PROBABILITIES Applied Probability and Statistics ECEG-342

Theorem 9. If A and B are any two events then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

Theorem 10. If A = A1 ∪ A2 ∪ · · · An , where A1 , A2 , · · · , An are mutually

exclusive events, then

P (A) = P (A1 ) + P (A2 ) + · · · + P (An )

In particular if A = S, then

P (A1 ) + P (A2 ) + · · · + P (An ) = 1

Theorem 11. For any events A and B,

P (A) = P (A ∩ B) + P (A ∩ B 0 )

Theorem 12. If an event A must result in one of the mutually exclusive

events A1 , A2 , · · · , An , then

P (A) = P (A ∩ A1 ) + P (A ∩ A2 ) + · · · + P (A ∩ An )

1.6 Assignment of Probabilities

If a sample space S consists only of elementary events A1 , · · · , An then by
theorem 10 above,
P (A1 ) + · · · + P (An ) = 1
It follows that we can arbitrarily choose any nonnegative numbers for the
probability of these simple events so long as the equality is satisfied. In
particular, if we assume equal probabilities for all simple events, then
1
P (Ak ) = , k = 1, 2, · · · , n
n
and if A is any event made up of m such simple events we have
m
P (A) =
n
Example 1.12: A biased coin is flipped such that head appears twice as that of
tail. Find P (H) and P (T ).

Example 1.13: A single dice is tossed. Find the probability of an even turning
up.

Murad Ridwan, 10 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.7. CONDITIONAL PROBABILITY Applied Probability and Statistics ECEG-342

1.7 Conditional Probability

Often it is required to find the probability of an event B under the condition
that event A occurs. This probability is called conditional probability of
B given A and is denoted by P (B/A). A serves as a new reduced sample
space.
P (A ∩ B)
P (B/A) =
P (A)
Similarly,
P (B ∩ A)
P (A/B) =
P (B)
Theorem 13. If A and B are events in a sample space S, then

P (A ∩ B) = P (A)P (B/A) = P (B)P (A/B)

Example 1.14: The probability that a regular scheduled flight departs on time
is P (D) = 0.83; the probability that it arrives on time is P (A) = 0.82; and the
probability that it departs and arrives on time is P (D ∩ A) = 0.78. Find the
probability that a plane
1. arrives on time given that it departed on time.

2. departed on time given that it has arrived on time.

If events A and B are such that P (A ∩ B) = P (A)P (B), they are called
independent events. Assuming P (A) 6= 0, P (B) 6= 0, we see that Independent
Events
P (A/B) = P (A), P (B/A) = P (B)

Bayes Theorem
Theorem 14. Let {A1 , · · · , An } be a disjoint partition of the sample space
S and let P (Ai ) 6= 0. Then for any event A
n
X
P (A) = P (A1 )P (A/A1 ) + · · · + P (An )P (A/An ) = P (Ai )P (A/Ai )
i=1

Exercise 1.11: Prove Bays theorem.

Theorem 15. Let {A1 , · · · , An } be a partition of S and let P (Ai ) 6= 0, then

P (Ak )P (A/Ak )
P (Ak /A) = Pn
k=1 P (Ai )P (A/Ak )

Murad Ridwan, 11 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.
 Class Notes on
1.7. CONDITIONAL PROBABILITY Applied Probability and Statistics ECEG-342

Example 1.15: Three boxes contain lamp bulbs some of which are defective. The
proportion of defective in box A1 , A2 and A3 are 12 , 18 and 43 respectively. A box is
selected at random and a bulb is drawn from it. If the selected bulb is found to
be defective, what is the probability that box A1 was selected?

Example 1.16: Box I contains 3 green and 5 red balls. Box II contains 2 green,
1 red and 2 yellow balls. We select a box at random and then draw one ball at
random from that box. What is the probability that we obtain a green ball?

Exercise 1.12:
1. A drawer contains 50 bolts and 150 nuts. Half of the bolts and half of the
nuts are rusted. If one item is chosen at random, w hat is the probability
that it is rusted or is a bolt?
Answer: 58
2. An integer is chosen at random from the first 200 positive integers. What is
the the probability that the integer chosen is divisible by 6 or 8?
Answer : 14
3. In a factory, machine A produces 30% of the output, machine B produces
25%, and machine C produces the remaining 45%. One percent of the
output of machine A is defective as is 1.2% of B’s output and 2% of C’s. In
a day’s run, the three machines produce 10000 items. an item is drawn at
random from the day’s output is defective. What is the probability that it
was produced by A? by B? by C?
Answer : 0.20, 0.20, 0.60
4. An ordinary die is thrown once. If a 1 or 6 appears, a ball i s then drawn
from urn I, otherwise a ball is drawn from urn II. urn I contains 3 red, 2
white and 1 blue balls. Urn II contains 4 white, 2 blue and no red balls.
Find the probability that

(a) a white ball is drawn

(b) that urn I was used, given that a white ball was drawn.
5 1
Answer : a. 9 b. 5

5. A bag contains 3 black and 4 white balls. Two balls are drawn at random
one at time without replacement.

(a) what is the probability that the second ball selected is white?
(b) what is the conditional probability that the first ball selected is white
if the second ball is known to be white?
Answer : a. 47 b. 12 .

Murad Ridwan, 12 of 12
Dep. of Electrical & Computer Engineering
AiOT, Addis Ababa University.
July 2010.