1

16.584: Lecture 2 : REVIEW Given a experiment with outcomes in sample space: Probability measure applied to subsets of : P [A] 0 P [A B ] = P [A] + P [B ] if AB = P [A B ] = P [A] + P [B ] P [AB ] P [A] = 1 P [Ac] Conditional Probability P (A|B ) = Total Probability P (B ) = P (B |A1)P (A1) + P (B |A2)P (A2) + ....P (B |An)P (An) (2) P (A B ) P (AB ) = P (B ) P (B ) (1)

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

Independence P (AB ) = P (A)P (B ) Bayes Theorem Ai i = 1, ...n : set of disjoint and exhaustive events and P (Ai) = 0 n i=1 Ai = AiAj = . B any event with P (B ) > 0 P (B |Aj )P (Aj ) n P (B |A )P (A ) i i i=1 P [Aj |B ]: Posteriori probability of Aj given B P (Aj |B ) = P [B |Aj ]: Apriori probability of B given Aj P [Aj ]: Causal or apriori probability of Aj (3)

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

Examples A box contains 6 red pens and 4 black pens. Assume each pen is equally likely to be chosen. Take two pens from the box, one after the other. What is the probability both pens are black? There are two kids named X and Y in a room. (a) What is the probability both are boys? (b) What is the probability both are boys given that at least one of them is a boy? Item Z is manufactured by two companies :X and Y . X produces twice as many items produced by Y . 20% of Z supplied from X is defective. 5% of Z supplied from Y is defective. What is the probability that a randomly chosen Z is not defective. (b) If Z is defective what is probability it came from Y ?

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

Permutations Count number of ways a particular event in an experiment can occur. Enumerate all possible ways : Brute Force Approach : Tedious as number of elementary outcomes increase Apply a general mathematical approach under appropriate assumptions Consider the ordered arrangement of n elements of a set A First element can be chosen n dierent ways Second element can be chosen n 1 dierent ways Total number of ways (or arrangements): n (n 1) (n 2)...1 = n! (n factorial) Permutation of n objects taken from n dierent outcomes : nPn

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

r-Permutation rPermutation of set A: Permutation of r objects taken from n dierent elements: nPr where r n.
n Pr

= (n) (n 1) (n 2) (n (r 1)) = n(n 1)(n 2)...(n r + 1) (n)(n 1)...(1) = (n r) (n r 1)...1 n! = (n r)! Assumption: All n objects are dierent and there is no replacement of the object If object replaced: number of ways of permuting r objects : nr

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

Combinations Selection/Arrangement of objects without regard to order Number of ways r objects selected from n with order : nPr Number of ways r objects can be arranged: r! Combinations of n objects taken r at a time: nCr or nCr =
n r
n Pr

n r

r!

n! (nr)! r!

: Binomial Coecients

n Other Notations: Ck

Example: Let set A : (a, b, c) Find number of ordered permutations of A Find 2 permutation of A
c Prof. K. Chandra, May 21, 2006 16.584: Probability and Random Processes; ECE, UMASS Lowell

Permutations of n non-unique objects If one element is repeated r times : It can be selected r! ways Number of permutations nPn reduced to where k is the number of unique objects Permutations of n objects: Stirlings Formula n! n e
n n n! n1 ! n2 ! n3 !...nk ! n! r!

In general if object i repeated ni times : n1 + n2 + nk = n

2n

! limn nnen n 2n = 1

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

Examples: A license plate is denoted by LLN N N , where L is a letter and N is a digit. It is required that the rst digit be greater than zero. Find probability of a license plate with two vowels (a,e,i,o,u) and three identical digits. Consider the set of elements (1, 2, 3, 4, 5). What are the number of ways of forming 4 digit sequences, assuming repetitions are allowed. What is the probability of a sequence with no repeated digits? Consider ve people denoted (a, b, c, d, e) arranged in a line. What is the probability that a and b will be adjacent? Consider four pairs of people. Each pair composed of (1M, 1F ). Form a committee of three people. (a) How many committees possible? (b) If committee must have (2F, 1M ), how many possible combinations exist. (c) If the committee must consist of only one person (either M or F) per pair, how many such combinations can be formed?
c Prof. K. Chandra, May 21, 2006 16.584: Probability and Random Processes; ECE, UMASS Lowell

How may ways can you choose six people from a group of 7 men and 8 women with the constraint there are atleast 3 W and atleast 2 M in the group. (b) What is the probability of a random selection of 3M and 3W ?

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

10

Bernoulli Trials Consider an experiment where outcomes : success or failure : : (S, F ) Probability of a success : p Probability of a failure : q = 1 p Conduct the experiment n times : n Trials The sample space for n trials consists of 2n number of possible outcomes Each outcome is of length n and made up of a random sequence of SSSF F SS..S Assuming that successive outcomes of S or F are independent Then: P r[SSSF SS ] = pppqpp = p5q 1 : Probab of 5 successes and 1 failure for n = 6 But there are other combinations of S/F where 5 successes and 1 failure can occur
c Prof. K. Chandra, May 21, 2006 16.584: Probability and Random Processes; ECE, UMASS Lowell

11

Bernoulli Contd. Number of ways k successes and n k failures can occur: P rob[k successes; n] =
n k n k

pk q (n k ) Union operation

Note that order in which successes or failures occur is not considered here Bernoulli Trials: Binomial Probability Law : b(k ; n; p) = k) Probabilities: b(k ; n; p) =
n k =0 b(k ; n; p) n k n k

pk q ( n

pk q (n k ) k = 0, 1, 2..n

=1

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

12

n Note: Coecent Ck : Binomial Coecient. Binomial Theorem

Probability b(k ; n; p) is (k + 1)th coecient of (q + p)n.

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

13

Random Variables and Probability Distributions Outcomes of a random experiment are not necessarily numerical values To facilitate mathematical analysis outcomes are mapped to a random variable (RV) X RV X is function that maps outcome of onto a number on the real line R. For an outcome , the RV X : R Generate a new sample space on the R Events of the original probability space replaced by events which are sets of numbers. Every event of is a subset of R Not necessary that every subset of R is always an event

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

14

RVs Contd. Subsets of Practical Interest [x = a] [x : a x b] [x : a < x b] [x : a x < b] [x : a < x < b] and their unions and intersections. Representations : [a], [a,b], (a,b], [a,b) and (a,b).

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

15

SUMMARY

Let be outcomes of experiment H Elements of are . To every assign a real number X ( ) Domain of the function X is and its range is R Every generates a specic X ( ) For a particular X ( ) there may be more than one that generates it. RVs may be Continuous or Discrete

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

16

Cumulative Probability Distribution Functions (PDF/CDF) RV X is continuous valued: FX (x) = = P r[X (, x)] = P r[X x] Representations: P [X x] Properties FX () = 1 P [a < X b]. FX () = 0 . (4)

FX (x) is a non-decreasing function of x. FX (x) is continuous from the right i.e. FX (x) = lim 0FX (x + ) >0

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

17

If point (a) or (b) are discontinuities P [a X b] = FX (b) FX (a) + P [X = a] P [a < X < b] = FX (b) P [X = b] FX (a) P [a X < b] = FX (b) P [X = b] FX (a) + P [X = a] At continuous points x P r[X = x] = 0

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

18

Probability Density Function (pdf) If FX (x) is continuous and dierentiable, then the pdf fX (x) = The properties of the pdf: fX (x) 0
fX ( )d

dFX (x) dx

(5)

= 1 = [FX () FX ()]
x fX ( )d

FX (x) = P (X x) =

FX (x2) FX (x1) = =

x2 fX ( )d x2 x1 fX ( )d

1 fX ( )d = P [x1 < X x2]

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

19

Interpret fX (x) Since P [x < X x + x] = FX (x + x) FX (x) If FX (x) is continuous in its rst derivative then, as x 0 FX (x + x) FX (x) =
x+x f ( )d x

= fX (x)x

(6)

fX (x)x = FX (x + x) FX (x)

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

20

Examples of some Distributions (univariate) Normal (Gaussian) pdf N (, 2)

(7)

Mean and variance 2. The mean and variance can be computed given the pdf as : = 2 =
xfX (x)dx

(8) (9)

(x

)2fX (x)dx

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

21

Standard Normal Distribution : N (0, 1) Given X : N (, 2) to determine P (a < X b) , require integral 1 P (a < X b ) = 2 2 Using the transformations =
x dx b 12 [(x)2 ] 2 dx a e

(10)

d =

1 P (a < X b ) = 2 where a = b =
(a) (b)

b 1 x2 2 dx a e

(11)

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

22

1 P (a < X b ) = 2 1 2 The error function

2 b 1 2 x dx e 0 a 1 x2 2 dx 0 e

(12)

1 x 1 t2 2 dt (13) erf (x) = 0 e 2 is tabulated and can be calculated using upper and lower bound approximations.

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell

23

Uniform distribution function 1 fX (x) = ba Rayleigh Distribution a<x<b (14)

x x22 fX (x) = 2 e 2 u(x) Exponential PDF

>0

(15)

1 x/ u(x ) fX (x) = e where > 0 and represents the avg rate of occurrence of events. Laplacian PDF c c|x| fX (x) = e c>0 2

(16)

(17)

c Prof. K. Chandra, May 21, 2006

16.584: Probability and Random Processes; ECE, UMASS Lowell