Probability

Life and our environment are uncertain

Common analysis method for uncertain situations
• Use “long-term averages”, i.e., probabilities

Common approach for modeling decision under uncertainty

• Optimizing the “average value” of the result of the decision
• perhaps subject to some risk constraint

Probability theory deals with phenomena

• whose outcomes are not fully predictable
• but exhibit some regularity when observed many times
History of Probability
Games of chances
Cardano (16th century):
• First systematic treatment for his gambling addiction

Pascal and Fermet (Question of division of stakes)

Christian Huygens (first book on probability)
(18-19 Century)
• Jacob Bernoulli
• Laplace: Applied to many practical problems, classic interpretation
• Poisson, Gauss (18-l9th Century): Mathematical organization

Kolmogorov (20th Century): Axiomatization

• Definition of probability that is precise enough for use in mathematics, yet comprehensive
enough to be applicable to a wide range of phenomena
Applications
Engineering:
• Communications, information theory
• Signal processing and control
• Queuing theory and modeling computer systems
• Decision/resource allocation under uncertainty
• Reliability

Statistics:
• collection and organization of data so that useful inferences can be drawn from them

Machine Learning, Optimization

Physics: statistical mechanics, thermodynamics

Computer Science:
• Randomized algorithms, random search, bioinformatics

Economics and Finance:

• Investment/insurance risk assessment
What is Probability?
A Dialogue for Uncertainty and Probability

Patient is administered (potentially) life-saving drug.

N = Nurse, R = concerned relative.
R: Nurse, what is the probability that the drug will work?
N: I hope that it works, we'll know tomorrow
R: Yes, but what is the probability that it will work?
N: Each case is different, we have to wait.
R: But lets see, out of a hundred patients that are treated under
similar conditions, how many times would you expect it to work?
N (somewhat annoyed): I told you, every person is different, for some
it works, for some it doesn't.
A Dialogue for Uncertainty and Probability

Patient is administered (potentially) life-saving drug.

N = Nurse, R = concerned relative.

R (insisting): Then tell me, if you had to bet whether it will work or not,
which side of the bet would you take?
N (cheering up for a moment): I’d bet it would work.
R (somewhat relieved): OK, now, would you be willing to lose two
dollars if it doesn’t work, and gain one dollar if it does?
N (exasperated): What a sick thought! You are wasting my time.
Probability Interpretations
Classical Interpretation
• For any random experiment, find the collections of atomic and equi-probable outcomes

• Probability of any event is equal to number of outcomes in favour of this/total outcomes

• Dice example

• Limitations!
Probability Interpretations
Frequentism Interpretation
• Frequency an event occurs in a large number of repetitions:
• Reality may or may not involve repetition!

Subjective belief:
• probability =expert's opinion
• How much you can bet!
Role of Maths
Probability is common sense reduced to calculation.
(Laplace)
The book of the universe is written in the language of mathematics.
(Galileo)

• Probabilistic analysis is mathematical, but intuition dominates and guides the

math.
(One of our goals: build intuition )
• Problem formulation in terms of probabilities is typically more challenging than
the calculations.
(Another goal: build modeling skill. Need lots of practice !)
Why intuition is not that
intuitive?
Mr Smith’s Children
Mr. Smith has two children. The older child is a girl. What is the probability that
both children are girls?
Mr Smith’s Children
Mr. Smith has two children. The older child is a girl. What is the probability that
both children are girls?

𝟏
𝟐
Mr Smith’s Children
Mr. Smith has two children. The older child is a girl. What is the probability that
both children are girls?
Mr. Smith has two children. At least one of them is a girl. What is the probability
that both children are girls?
Mr Smith’s Children
Mr. Smith has two children. The older child is a girl. What is the probability that
both children are girls?
Mr. Smith has two children. At least one of them is a girl. What is the probability
that both children are girls?

𝟏
𝟑
Mr Smith’s Children
Mr. Smith has two children. The older child is a girl. What is the probability that
both children are girls?
Mr. Smith has two children. At least one of them is a girl. What is the probability
that both children are girls?

BB ¼
BG ¼ 

GB ¼ 

GG ¼ 
Monty Hall’s Problem
Suppose you're on a game show, and you're given the choice of three
doors: Behind one door is a car; behind the others, goats.
You pick a door, say No. 1, and the host, who knows what's behind the
doors, opens another door, say No. 3, which has a goat.
He then says to you, "Do you want to pick door No. 2?" Is it to your
advantage to switch your choice?
Winning strategy is to switch
Result if
Result if
Behind Behind switching to
Behind door 1 staying at
door 2 door 3 the door
door #1
offered
Goat Goat Car Wins goat Wins car
Goat Car Goat Wins goat Wins car
Car Goat Goat Wins car Wins goat
Betrand’s Paradox
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
What is the probability that the chord is longer than a side of the triangle?
Betrand’s Paradox
Consider an equilateral triangle inscribed in a circle.
Suppose a chord of the circle is chosen at random.
What is the probability that the chord is longer than a side of the triangle?

1/3 1/2 1/4

Models
Random Events

Choose a
number

Finite number of Countable number of Uncountable number

possibilities possibilities of possibilities
Random Variables
Assign number to each events
Why?
• For tractability and mathematical notation.
• Order to events
• Measure and quantify different distributions

Can be physical interpretation, intuitive or any

arbitrary interpretation

𝟏 𝒊𝒇 𝑯𝒆𝒂𝒅𝒔
𝑿=
𝟎 𝒊𝒇 𝑻𝒂𝒊𝒍𝒔
0 1
Repeated Trials
Random Process
 Continuous process
 Can be used to model noise,
random behavior of systems,
packet arrival process,
unknown parameters process,
perturbed process, time series
models, spatial process (to
model BS locations), speech,
signal

Discrete time process

• Transition probability

Correlation, time evolution

 “Sample Space’’
What is interesting!!
Sample Space
A sample space is a collection of all possible outcomes
Elements must be
• Mutually exclusive
• Collectively exhaustive
• Finest grain

How to chose granularity?

• Example: coin toss

Two important things:

• Our interest in outcome
• Can we measure it? Tell with full confidence if the event has occurred.