Scribd
Upload
14 views

Lect - 01 Introduction Probability (Part B)

The document discusses machine learning for signal processing. It covers topics like conditional probability, random variables, and different types of random variables. Random variables can be continuous, taking any value in a range, or discrete, having specific values. The document provides definitions and examples to explain these concepts.

Uploaded by

trasakroft
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views

Lect - 01 Introduction Probability (Part B)

The document discusses machine learning for signal processing. It covers topics like conditional probability, random variables, and different types of random variables. Random variables can be continuous, taking any value in a range, or discrete, having specific values. The document provides definitions and examples to explain these concepts.

Uploaded by

trasakroft
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Machine Learning for Signal Processing

EHB 328E

Ibraheem Shayea
Electronics and Communication Engineering Department,
Faculty of Electrical and Electronics Engineering
İstanbul Teknik Üniversitesi - İTÜ
Syllabus 1

Fundamentals of Probability and


Statistics
Part B
Outline

❑ Conditional Probability

❑ RandomVariables
Conditional Probability
Conditional Probability
Definition
❑ The conditional probability of an event A given event B, denoted by;
𝑃(𝐴 \ 𝐵),is defined as :

𝑃(𝐵 \ 𝐴), is the conditional probability of an event B given event A.


From the above two Eqs.We have :

This Equation is often quite useful in computing the joint probability of events.
Conditional Probability
Bayes' Rule:
From

We can obtain the following Bayes' rule:


Random Variables
Random Variables
Definition

❑ A Random Variable (R.V.) is a variable that its value is unknown or


a function that assigns values to each of an experiment's outcomes.

❑ A Random Variable is a numerical variable whose measured value can


change from one replicate of the experiment to another.

❑ Traditionally, numerical measurements or observations that have


uncertain variability each time they are repeated are called Random
Variables.
Random Variables
Definition

❑ Range of the R.V. X: The collection of all numbers [values of 𝑿(𝜻) ] is termed
the range of the R.V. X.

o The range of X is a certain subset of the set of all real numbers (Fig.).

Note That two or more different sample points


might give the same value of 𝑿(𝜻) , but two
different numbers in the range cannot be
assigned to the same sample point.

Fig. Random variable X as a function.


Random Variables
Definition
Relationship between sample space, random variable, and probability

Fig. Illustration of the relationship among sample space, random variable, and probability
Random Variables
Definition
❑ Not Like an Algebra Variable
In Algebra a variable, like 𝒙, is an unknown value:
Example: 𝑥 + 2 = 6
In this case we can find that 𝑥 = 4

❑ But a Random Variable is different


A Random Variable has a whole set of values and it could take on
any of those values, randomly.
Example: X = {0, 1, 2, 3}
X could be 0, 1, 2, or 3 randomly.

And they might each have a different probability.


Random Variables
How Events can be Defined by Random Variables?
These events have probabilities that are denoted by:

P 𝑋 = 𝑥 = 𝑃 𝜁: 𝑋 𝜁 = 𝑥

P 𝑋 ≤ 𝑥 = 𝑃 𝜁: 𝑋 𝜁 ≤ 𝑥

P 𝑋 > 𝑥 = 𝑃 𝜁: 𝑋 𝜁 > 𝑥

P 𝑥1 < 𝑋 ≤ 𝑥2 = 𝑃 𝜁: 𝑥1 < 𝑋 𝜁 ≤ 𝑥2
Random Variables
Example:
❑ Find the sample space for the experiment of tossing a coin

(a) once and


(b) twice.
Random Variables
Example :
❑ Let us construct a model for counting the number of heads in a
sequence of three coin tosses. For the underlying sample space, we take

𝑆 = {𝑇𝑇𝑇, 𝑇𝑇𝐻, 𝑇𝐻𝑇, 𝐻𝑇𝑇, 𝑇𝐻𝐻, 𝑇𝐻𝑇𝐻, 𝐻𝐻𝑇, 𝐻𝐻𝐻}

which contains the eight possible sequences of tosses. However, since we are only
interested in the number of heads in each sequence, we define the Random Variable (function)
𝑋 by:

0, 𝑥 = 𝑇𝑇𝑇, 𝑆
1, 𝑥 ∈ 𝑇𝑇𝐻, 𝑇𝐻𝑇, 𝐻𝑇𝑇 ,
𝑋 𝑥 =
2, 𝑥 ∈ 𝑇𝐻𝐻, 𝐻𝑇𝐻, 𝐻𝐻𝑇 ,
3, 𝑥 = 𝐻𝐻𝐻.
Random Variables
What are the types of Random Variables ??

A RandomVariable (R.V.) has two main types:

A. Continuous RandomVariables (C.R.V.)

o Continuous (any value in a continuous range)

B. Discrete RandomVariables (D.R.V.)


o Discrete (having specific values)
Random Variables
What are the Types of Random Variables ??
A. Continuous Random Variables
A Continuous Random Variable is a random variable with infinitely many possible
values (think an interval of real numbers, e.g., [0,1] ).

Examples: income, age, speed, distance, etc.


❑ PDF
fX(x)
Marginal Changes
𝑑𝐹𝑋 (𝑥)
𝑓𝑋 𝑥 = 𝑑𝑥
fX(x)

❑ CDF dx x
Bounded Area
P  x  X  x + dx  = f X ( x ) dx
𝑥
𝐹𝑋 𝑥 = ‫׬‬−∝ 𝑓𝑋 𝑡 𝑑𝑡
Random Variables
What are the types of Random Variables ??
B. Discrete Random Variables
A Discrete Random Variable is a random variable that has only a finite or
countably infinite (think integers or whole numbers) number of possible
values (points).
Examples: race, religion, etc.
No marginal change No bounded area
𝐹𝑋 𝑥 = ෍ 𝑃𝑋 𝑥𝑘 𝑢(𝑥 − 𝑥𝑘 )
𝑃𝑋 𝑥𝑘 = 𝑃[𝑋 = 𝑥𝑘 ]
𝑘

Experiment Random Variable Possible Values


Make 100 calls # of calls 0,1, 2, … , 100
Answer 33 questions # of correct answers 0, 1, 2, …, 33
Count cars at toll plaza # of cars passed 0, 1, 2, …, ∞
Random Variables
Random Variables and Distribution Function

Probability Mass
Function (𝑃𝑀𝐹)

Cumulative
Discrete and Mixed Generalized Probability
Distribution
Random Variable Density Function
Function (𝑪𝑫𝑭)

Probability Density
Function (𝑃𝐷𝐹)

Fig. illustration of the relationship among CDF, PMF, and PDF.


The End

Thank You
References
Chapter 2

You might also like