0% found this document useful (0 votes)
73 views13 pages

Machine Learning - 9: BITS Pilani

The document discusses Naive Bayes classifiers and Bayesian belief networks. It explains that Naive Bayes classifiers assume conditional independence between attributes, allowing it to estimate probabilities efficiently. Bayesian belief networks also represent conditional independence relationships between variables using a directed acyclic graph structure. The networks allow calculating joint probabilities as a product of conditional probabilities values stored in tables at each node. An example network is provided to illustrate how it captures conditional independence between variables like lung cancer, family history, and other related variables.

Uploaded by

Simran sandhu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
73 views13 pages

Machine Learning - 9: BITS Pilani

The document discusses Naive Bayes classifiers and Bayesian belief networks. It explains that Naive Bayes classifiers assume conditional independence between attributes, allowing it to estimate probabilities efficiently. Bayesian belief networks also represent conditional independence relationships between variables using a directed acyclic graph structure. The networks allow calculating joint probabilities as a product of conditional probabilities values stored in tables at each node. An example network is provided to illustrate how it captures conditional independence between variables like lung cancer, family history, and other related variables.

Uploaded by

Simran sandhu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 13

Machine Learning - 9

BITS Pilani
Hyderabad Campus
Naïve Bayes Classifier

BITS Pilani, Hyderabad Campus


Naïve Bayes Classifier

 But it requires a lot of data to estimate (roughly O(|A|n) parameters


for each class):
P(a1,a2,…an| vj)

 Naïve Bayesian Approach: We assume that the attribute values


are conditionally independent so that
P(a1,a2,..,an|vj) =i P(a1|vj)

 Naïve Bayes Classifier:


vNB = argmaxvj V P(vj) i P(ai|vj)

BITS Pilani, Hyderabad Campus


BITS Pilani, Hyderabad Campus

I.e. Estimate P(v )j and P(a |v


i )j –
possibly by counting occurence
of each class an each attribute in
each class among all examples
Naïve Bayes Classifier
Naïve Bayes Classifier

What is the predicted outcome (Yes/No) of the following instance?

BITS Pilani, Hyderabad Campus


Naïve Bayes Classifier

BITS Pilani, Hyderabad Campus


Bayesian Belief Networks

Bayesian belief networks assume


conditional independence only between
subsets of the attributes
– Conditional independence
• Discrete-valued random variables X,Y,Z

• X is conditionally independent of Y given Z if

P(X |Y,Z)= P(X |Z)

BITS Pilani, Hyderabad Campus


Bayesian Belief Networks
Conditional Independence

– Once we know that the patient has cavity we do


not expect the probability of the probe catching to
depend on the presence of toothache
P(catch | cavity  toothache)  P(catch | cavity)
P(toothache | cavity  catch)  P(toothache | cavity)

BITS Pilani, Hyderabad Campus


Bayesian Belief Network: An Example

Family
Smoker
History
(FH, S) (FH, ~S) (~FH, S) (~FH, ~S)

LC 0.8 0.5 0.7 0.1


LungCancer Emphysema ~LC 0.2 0.5 0.3 0.9

The conditional probability table for


the variable LungCancer: Shows the
conditional probability for each
PositiveXRay Dyspnea possible combination of its parents
n
P( z1,..., zn)   P ( z i | Parents ( Z i ))
Bayesian Belief Networks i 1
Bayesian Belief Networks

BITS Pilani, Hyderabad Campus


Bayesian Belief Networks

Representation
• A Bayesian network represents the joint probability
distribution of a set of variables
• Each variable is represented by a node

• Conditional independence assumptions are


indicated by a directed acyclic graph
• Variables are conditionally independent of its
nondescendents in the network given its inmediate
predecessors

BITS Pilani, Hyderabad Campus


Bayesian Belief Networks
P (X1, … ,Xn) = πni =1 P (Xi | X1, … , Xi-1) (chain rule)
= πni =1P (Xi | Parents(Xi)) (by construction)

The joint probabilities are calculated as

P(Y1,Y2,...,Yn) = i=1,n P [Yi|Parents(Yi)]

The values P [Yi|Parents(Yi)] are stored in tables


associated to nodes Yi

Example:

P(Campfire=True|
Storm=True,BusTourGroup=True)=0.4
BITS Pilani, Hyderabad Campus
Thank You!!

BITS Pilani, Hyderabad Campus

You might also like