CHAPTER 4

PROBABILITY
THEORY
 4.1. IMPORTANCE CONCEPTS OF
PROBABILITY

Definition of Probability
 Probability is a branch of mathematics that deals with the
likelihood of a specific event or outcome occurring.
 It quantifies the uncertainty of an event by assigning a
numerical value between 0 and 1,
 where 0 indicates impossibility and 1 indicates certainty
 CONT..IMPORTANCE CONCEPTS OF
PROBABILITY

 Basic Concepts in Probability

 Sample Space:-The sample space is the set of all possible
outcomes of an experiment or random process.
eg, when rolling a fair six-sided die, the sample space consists of
the numbers S={ 1 , 2, 3, 4, 5, 6}
 Event: An event is a subset of the sample space. e.g., rolling an
even number
 Probability Distribution: A probability distribution describes
how the probabilities are distributed over the values in the
sample space.
 Probability Rules: There are several rules that govern
probability calculations, such as the addition rule,
multiplication rule, complement rule, and conditional
probability.
 CONT..IMPORTANCE CONCEPTS OF
PROBABILITY

Importance of Probability
Here are some key reasons why probability is important:
 Decision Making:
 Risk Analysis
 Predictive Modeling In fields such as weather forecasting
 Statistical Inference involves drawing conclusions about a
population based on sample data.
 Machine Learning
 Gaming and Gambling
 Financial Markets
 4.2. TYPES OF EVENT

In probability theory, events are subsets of a sample space.

 Simple Event: A simple event is an event that consists of only
one outcome.
Eg, when rolling a fair six-sided die, the event of getting a 3 is a
simple event.
 Compound Event A compound event is an event that consists
of two or more simple events. These events involve the
probability of more than one event occurring together.
Eg: the probability of rolling an even number on a die, then
tossing a head on a coin.
Here P (even number) = 3/6
P (head) = 1/2
Hence required probability = (3/6) (½) = 3/12
 CONT.…TYPES OF EVENT

 Mutually exclusive events :are two or more events that cannot occur
at the same time.
Example, getting heads and tails in a coin toss or rolling a 2 and
a 3 on a die.
 If two events are mutually exclusive then
P (A or B) +P (A) + P (B)
 Independent Events: Independent events are events where the
occurrence of one event does not af fect the probability of the other
event occurring. If two events are independent then
P (A and B) = P (A) × P (B).
E.g. If we are rolling a dice twice, we can find the probability of
getting two sixes.
 The probability of getting a 6 is 1/6
 So the probability of getting a 6 and a 6 is
 1/6×1/6=1/36
 CONT.…TYPES OF EVENT

 Dependent Events: Dependent events are events where the

outcome of one event affects the probability of the other
event occurring
Example: A juggler has seven red, five green, and four blue
balls. During his stunt, he accidentally drops a ball and doesn't
pick it up. As he continues, another ball falls down. What is the
probability that the first ball that was dropped is blue, and the
second ball is green?
P (blue than green) = P (blue) ×P (green) P (blue than green) =
P (blue) ×P (green)
=4/16×5/15=1/12
 CONT.…TYPES OF EVENT

 Complementary Event: The complementary event of an event

A is denoted as A’ and represents all outcomes that are not in
A. The probability of an event and its complementary event
sum up to 1.
P (A) + P (A') = 1
 Impossible Event: An impossible event has no possible
outcomes and thus has a probability of 0.
 Sure Event (Certain Event): A sure or certain event is an event
that will always occur and has a probability of 1
 Equally Likely Events: Equally likely events are events where
each outcome is as likely as any other outcome to occur.
When rolling a fair six-sided die, each face has an equal
chance of appearing.
 4.3. TYPES OF PROBABILITY

 There are several types of probability that are commonly used

in various fields such as statistics, mathematics, and science.
These types of probability include:
 Classical Probability: Classical probability, also known as a
priori probability, is based on the assumption that all
outcomes in a sample space are equally likely
Example: Roll a fair die on a hard, flat floor and observe the
number appearing on the upper face. A = {Observe an odd
number}, find the probability of A
P (A) = P (1) + P (3) + P (5) = 1/6 + 1/6 + 1/6 = ½
 Empirical probability, also known as experimental probability,
is based on observations or experiments. It is calculated by
dividing the number of times an event occurs by the total
number of trials.
 CONT…TYPES OF PROBABILITY

Example that you want to look at a small dataset such as the

possibility of rolling a six when you roll a single die. If on the
first roll you roll a 2, on the second a 5, and on the third a 4,
the empirical probably is 0/3=0%. The empirical probability in
this case is 0%.
 Subjective Probability: Subjective probability is based on an
individual’s personal judgment or belief about the likelihood
of an event occurring.
 Conditional Probability: is the probability of an event
occurring given that another event has already occurred. It is
calculated by dividing the joint probability of both events by
the probability of the given event. The formula for conditional
probability is
 CONT…T YPES OF PROBABILIT Y

Exa mple of Conditional Probability

Let’s consider an exa mple to illustrate co nditional probability. Suppose we
have a standard dec k of 52 playing cards. What is the probability of drawing a
red card (eve nt A) give n that we have already drawn a hear t (eve nt B)?

Calc ulate the Probability of Event B (Drawing a Hear t):

There are 1 3 hear ts in a deck of 52 cards, so P (B) = 1 3 /52=1/4.
Calc ulate the Probability of Event A and B (Drawing a Red Card and a
Hear t):
There are 26 red cards in a deck , including 1 3 hear ts.
Therefore, P(A∩B)=1 3 /52=1/4.
Apply the Formula for Conditional Probability: Using the formula:
We get:
 P (A|B) =1/4/1/4=1 .
 CONT…T YPES OF PROBABILIT Y

 Bayesian Probability:
 Bayesian probability is a type of statistical inference that
uses Bayes’ theorem to update the probability of a hypothesis
as new evidence becomes available.
 Frequency Probability:
 Frequency probability, also known as relative frequency
probability, is based on the long-term relative frequency of an
event occurring in repeated trials. It is calculated by dividing
the number of times an event occurs by the total number of
trials in the long run.

 CONT…T YPES OF PROBABILIT Y

Marginal Probability:
 Marginal probability is the probability of a single event
occurring without considering any other events. It is obtained
by summing or integrating joint probabilities over all other
variables.
 Joint Probability:
 Joint probability is the probability of two or more events
occurring simultaneously. It is calculated by multiplying the
probabilities of each individual event.
 CONT…T YPES OF PROBABILIT Y

E xa m p l e S u p p o s e w e have a s t a nd a rd d e c k o f 5 2 p l ay i ng c ar d s . We wa n t to f i n d th e
p ro b a b i l i t y o f d raw i ng a red c a rd a nd a f a c e c a rd ( j a c k , qu ee n, o r k i n g ) f r o m t h e d e c k
i n t w o co n s ec u t i ve d raw s w i t ho ut re p l a ce m e nt .
C a l c u l a t i n g t he Pro b a b i l i t y o f D raw i ng a Re d C ar d :
T he r e ar e 2 6 red c a rd s i n a s t a nd a r d d e c k ( hea r t s a nd d i a m o nd s ) o u t o f a to t a l o f 5 2
c a rd s .
T he p r o b a b i l i t y o f d raw i ng a red c a rd o n t he fi r s t d raw i s P( Re d ) = 2 6 / 5 2 = 1 / 2 .
C a l c u l a t i n g t he Pro b a b i l i t y o f D raw i ng a Fa c e C a rd :
T he r e ar e 1 2 f a c e c a rd s i n a s t and a rd d e c k ( j a c k , qu ee n, a nd k i ng fo r e a c h s u i t ) o u t
o f a to ta l o f 5 2 c a rd s .
T he p r o b a b i l i t y o f d raw i ng a f a ce c a rd o n t he s e co nd d raw, g i ve n t h a t a r ed c a r d wa s
d raw n o n t he fi r s t d raw, i s P( Fa c e | Re d ) = 1 2 / 51 .
C a l c u l a t i n g t he J o i nt P ro b a b i l i ty :
 T he j o i n t p ro b a b i l i t y o f d raw i ng a red c a rd a nd t hen a f a c e c a r d c a n b e c a l cu l a te d
by m u l t i p l y i ng t he i nd i v i d u a l p ro b a b i l i t i es : P ( Re d a nd Fa c e ) = P ( Re d ) P ( Fa ce | Re d )
P ( Re d a n d Fa c e) = ( 1 / 2 ) ( 1 2 / 51 ) = 6 / 51
 4.4. RULES OF PROBABILIT Y

Rule 1 (for not the event): (Complement rule)

To find the probability of Ac, the complement of A , use P (Ac) = 1 P (A)
Example Event A: The roll of a die is 2; P (A) = 1/6
 Event AC: The roll of a die is NOT 2 (1 or 3-6). P (Ac) = 1-1/6=5/6
 Rule 2 (addition rule for either/or/and):
 To find the probability that either A or B or both happen:
 ( Rule 2a (general): P(A or B) = P(A) + P(B) - P(A and B)
 ( Rule 2b (for mutually exclusive events): If A and B are mutually
exclusive events:
 P (A or B) = P (A) + P (B)
 Example Event A: The roll of a die is odd (1 , 3, 5); P (A) = 3/6 = 1/2
 Event B: The roll of a die is even (2, 4, 6). P (B) = 3/6 = 1/2
 Because Event A and Event B are mutually exclusive, P (A or
B) = 1/2 +1/2= 1
 4.4. RULES OF PROBABILIT Y

Rule 3 (multiplication rule for and):

To find the probability those two events, A and B, both occur
simultaneously or in a sequence:
( Rule 3a (general): P (A and B) = P (A) P (B|A) = P (B) P (A|B)
( Rule 3b (for independent events): If A and B are independent events,
 P (A and B) = P (A) P (B)
 E.g.
 Event A: Roll a die, and the outcome is 1; P (A) = 1/6
 Event B: Roll another die simultaneously, and the outcome is
also 1 . P (B) = 1/6
 Because Event A and Event B are independent, P (A and B)
(meaning that roll two dice simultaneously, and both outcomes are
1) = 1/6 * 1/6= 1/36
 4.4. RULES OF PROBABILIT Y

Rule 4 (conditional probability):

To find the probability that B occur s given that A has occurred or will occur:
P (B|A) = P (A and B)/P (A)
The assignment of letter s to events A and B is arbitrar y, so it is also true
that
P (A|B) =P (A and B)/P (B)
E.g.
Event A: The roll of a die is odd (1 , 3, 5); P (A) = 1/2
Event B: The roll of a die is not 1 . P (B) = 5/6
P (A and B), or the probability that the roll of a die is odd and not 1 = the
probability that the roll of a die is 3 or 5 = 2/6 = 1/3
 P (A|B) = P (A and B) / P (B) = (1/3) / (5/6) = 2/5
4.5. BAYES THEOREM

4.5. Bayes theorem

 Bayes theorem is a fundamental principle in the field of
probability and statistics. It describes the probability of an
event based on prior knowledge or information.
4.5. BAYES THEOREM

Example: for Bayes theorem can be applied.

S u p p o s e t h e r e a r e t w o b a g s , b a g X a n d b a g Y, e a c h c o n t a i n i n g r e d a n d b l u e b a l l s . B a g X c o n t a i n s 3 r e d
balls and 2 blue balls, while bag Y contains 2 red balls and 3 blue balls. A bag is chosen at random,
and a ball is drawn from it. The ball is found to be red. What is the probability that the ball was drawn
from bag X?
Lets denote:
A as the event that the ball was drawn from bag X.
B as the event that the ball drawn is red.
We want to find P (A|B), the probabi lity that the ball was drawn from bag X gi ven that it is red.
We are given:
P(B|A) = 3/5 (the probability of drawing a red ball from bag X)
P(A) = 1/2 (the probability of choosing bag X)
P ( B | ¬ A ) = 2 / 5 ( t h e p r o b a b i l i t y o f d r a w i n g a r e d b a l l f r o m b a g Y, w h e r e ¬ A d e n o t e s t h e c o m p l e m e n t a r y
event of (A)
P ( ¬ A ) = 1 / 2 ( t h e p r o b a b i l i t y o f c h o o s i n g b a g Y, w h i c h i s t h e c o m p l e m e n t a r y e v e n t o f c h o o s i n g b a g ( X )
Using Bayes theorem, we can calculate P (A|B) as follows:
P (A|B) = [P (B|A) * P (A)] / [P (B|A) * P (A) + P (B|¬A) * P (¬A)] = (3/5 * 1/2) / [(3/5 * 1/2) + (2/5 *
1/2)] = 3/5 / (3/5 + 2/5) = 3/5 / 1 = 0.6
 Therefore, the probability that the ball was drawn from bag X given that it is red is 0.6 or 60%.
 4.6. TREE DIAGRAM

4.6. Tree diagram

What are p robab ility tree diagrams?
Prob ab ility tree d iagrams are a way of organizing the information of two or
more prob ab ility events. Prob ab ility tree diagrams show all the possible
outcome s of the events and can be used to
Example
A coin is tossed twice. List all the possib le outcomes.
Solution you can use a tree diagram to re present this solution.

 S = {TH, TH, TH, TT, TH, HT, HH,}

 4.7. JOINT PROBABILIT Y TABLE

A joint probability table is a method used in probability theor y and

statistics to represent the probabilities of multiple events occurring
simultaneously. It provides a systematic way to display the joint
probabilities of two or more random variables. In a joint probability
table, each cell represents the probability of the intersection of specific
values of the variables involved.The joint probability formula is ver y
simple and straight for ward:

Where:
P(A∩B) is the joint probability of two events (A ,B)
P (A) is the probability of the first event on its own, and
P (B) is the probability of the second event on its own.
 4.7. JOINT PROBABILIT Y TABLE

Example: Find the probability that the number three will occur twice
when two dice are rolled at the same time.
Solution:
Number of possible outcomes when a die is rolled = 6
i.e. {1 , 2, 3, 4, 5, 6}
Let A be the event of occurring 3 on first die and B be the event of
occurring 3 on the second die.
Both the dice have six possible outcomes, the probability of a three
occurring on each die is 1/6.
P (A) =1/6
P (B) =1/6
 P (A , B) = 1/6 x 1/6 = 1/36
 4.7. JOINT PROBABILIT Y TABLE

J o i n t P r o b a b i l i t y Ta b l e
A j o i n t p r o b a b i l i t y d i s t r i b u t i o n r e p r e s e n t s a p r o b a b i l i t y d i s t r i b u t i o n fo r t w o o r m o r e
r a n d o m va r i a b l e s . I n s te a d o f ev e n t s b e i n g l a b e l l e d A a n d B , t h e c o n d i t i o n i s to u s e X a n d
Y a s g i ve n b e l ow.
F(x,y) = P(X = x, Y = y)
T h e m a i n p u r p o s e o f t h i s i s to l o o k fo r a r e l a t i o n s h i p b e t w e e n t w o va r i a b l e s . Fo r ex a m p l e ,
t h e b e l ow t a b l e s h ow s s o m e p r o b a b i l i t i e s fo r ev e n t s X a n d Y h a p p e n i n g a t t h e s a m e t i m e :

T h i s t a b l e c a n b e u s e d to f i n d t h e p r o b a b i l i t i e s o f ev e n t s .
Find the probability of X = 3 and Y = 3.
S o l u t i o n : Fr o m t h e a b ov e t a b l e , i d e n t i f y t h e p r o b a b i l i t y u n d e r X = 3 a n d Y = 3 .
 That is ⅙.
C o n s t r u c t i n g a J o i n t P r o b a b i l i t y Ta b l e

To c o n s t r u c t a j o i n t p r o b a b i l i t y t a b l e , f o l l o w t h e s e s te p s :

Identify the sample space: The sample space is the set of all possible outcomes of the
e v e n t s b e i n g c o n s i d e r e d . Fo r e x a m p l e , i f y o u a r e r o l l i n g t w o d i c e , t h e s a m p l e s p a c e w o u l d
be the set of all possible combinations of rolls (e.g., (1 ,1), (1 ,2), (1 ,3), …, (6,6)).
I d e n t i f y t h e ev e n t s : T h e ev e n t s a r e t h e s p e c i f i c o u t c o m e s o r c a te g o r i e s b e i n g c o n s i d e r e d .
Fo r ex a m p l e , i f y o u a r e r o l l i n g t w o d i c e , t h e e v e n t s m i g h t b e “ r o l l i n g a 3 o n t h e f i r s t d i e ”
and “rolling an even number on the second die.”
D e te r m i n e t h e p r o b a b i l i t i e s : C a l c u l a te t h e p r o b a b i l i t i e s o f e a c h e v e n t o c c u r r i n g
s e p a r a te l y. Fo r e x a m p l e , i f y o u a r e r o l l i n g t w o d i c e , t h e p r o b a b i l i t y o f r o l l i n g a 3 o n t h e
f i r s t d i e i s 1 / 6 , a n d t h e p r o b a b i l i t y o f r o l l i n g a n ev e n n u m b e r o n t h e s e c o n d d i e i s 1 / 2 .
C r e a te t h e t a b l e : S e t u p a t a b l e w i t h r ow s a n d c o l u m n s r e p r e s e n t i n g e a c h ev e n t . L a b e l
t h e r ow s a n d c o l u m n s w i t h t h e p o s s i b l e o u t c o m e s o r c a te g o r i e s f o r e a c h ev e n t . Fo r
e x a m p l e , f o r r o l l i n g t w o d i c e , y o u w o u l d h a v e s i x r ow s a n d s i x c o l u m n s f o r e a c h p o s s i b l e
outcome on each die.

 4.7. JOINT PROBABILIT Y TABLE

 Fill in the table: Calculate and enter the probabilities for each
combination of events. To find these probabilities, multiply
the individual probabilities for each event occurring together.
For example, if you are considering rolling a 3 on the first die
and an even number on the second die, you would calculate
the probability as (1/6) x (1/2) = 1/12.
 4.8. RULES OF COUNTING

When the number of possible outcome is very large it will be difficult to find possible outcomes by listing. So
you have to investigate dif ferent Counting techniques.

Multplication Principle

If an event can occur in m different ways, and for every such choice another event can occur in n dif ferent
ways, then both the event can occur in the given order in m*n different ways.

Example Suppose you have 3 coat, 8 shirt, and 6 dif ferent trousers in how mony dif ferent ways can you dress?

Solution 3*8*6=144 ways

Addition principle

If an event E1 can occur in m ways and if another event E2 can occur in n ways and either of the event can occur
in m+n ways. This is true if E1 and E2 is mutually exclusive events
Example a question paper has two parts where 1 part contains four questions and the other 3 questions. If a
student has to choose one question from either part in how mony way can the student do it
Solution the student can choose 4+3 =7 ways
 4.8. RULES OF COUNTING

Permut at ion
The permutations is the number of arrangement s of object s wit h at tenti on give n
to the order of arrangement s.
Exa mple Suppose there are five seats arranged in a row. I n how mony dif fe rent
ways can five people people seated on t hem?
Solution 5*4*3*2*1=1 20 ways
The number of permutations of the set of n object s taken all together is denoted
by p(n,n) and is equal to n!
n! = n*(n-1)*(n-2)*(n-3)*...3*2*1
Exa mple Suppose we have 5 peopl e to be seated on i n only 3 seats in how mony
way can t hey seats?
Solution The fir st ch air can be filled by any one of the 5 people, the second by
any one of the four people, and t he the t hird by anyone of t he 3 peopl e by t he
multi pl ication principles this give s 5*4*3 = 60 ways
 5*4*3=60=
 4.8. RULES OF COUNTING


 4.8. RULES OF COUNTING

Example in how many different ways can three men and three
women be selected from six men and eight women ?
