Scribd
Upload
8 views

06 Uncertainty

Uploaded by

Nandini Ganjewar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views

06 Uncertainty

Uploaded by

Nandini Ganjewar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Uncertainty

Chapter 13

Mausam
(Based on slides by UW-AI faculty)
Knowledge Representation

KR Language Ontological Commitment Epistemological Commitment


Propositional Logic facts true, false, unknown
First Order Logic facts, objects, relations true, false, unknown
Temporal Logic facts, objects, relations, times true, false, unknown
Probability Theory facts degree of belief
Fuzzy Logic facts, degree of truth known interval values

Probabilistic Relational Models


- combine probability and first order logic
Need for Reasoning w/ Uncertainty
• The world is full of uncertainty
– chance nodes/sensor noise/actuator error/partial info..
– Logic is brittle
• can’t encode exceptions to rules
• can‘t encode statistical properties in a domain
– Computers need to be able to handle uncertainty
• Probability: new foundation for AI (& CS!)
• Massive amounts of data around today
– Statistics and CS are both about data
– Statistics lets us summarize and understand it
– Statistics is the basis for most learning
• Statistics lets data do our work for us
•© UW CSE AI Faculty •3
Logic vs. Probability
Symbol: Q, R … Random variable: Q …

Boolean values: T, F Domain: you specify


e.g. {heads, tails} [1, 6]
State of the world: Atomic event: complete
Assignment to Q, R … Z specification of world: Q… Z
• Mutually exclusive
• Exhaustive
Prior probability (aka
Unconditional prob: P(Q)
Joint distribution: Prob.
of every atomic event
•© UW CSE AI Faculty •4
Probability Basics
• Begin with a set S: the sample space
– e.g., 6 possible rolls of a die.
• x ϵ S is a sample point/possible world/atomic event
• A probability space or probability model is a sample
space with an assignment P(x) for every x s.t.
0≤P(x)≤1 and ∑P(x) = 1
• An event A is any subset of S
– e.g. A= ‘die roll < 4’
• A random variable is a function from sample points
to some range, e.g., the reals or Booleans
Types of Probability Spaces

•© UW CSE AI Faculty •6
Axioms of Probability Theory
• All probabilities between 0 and 1
– 0 ≤ P(A) ≤ 1
– P(true) = 1
– P(false) = 0.
• The probability of disjunction is:
P( A  B)  P( A)  P( B)  P( A  B)

A  B
A
True

B
•© UW CSE AI Faculty •7
Prior Probability

Joint distribution can answer any question


•© UW CSE AI Faculty •8
Conditional probability
• Conditional or posterior probabilities
e.g., P(cavity | toothache) = 0.8
i.e., given that toothache is all I know there is 80% chance of cavity

• Notation for conditional distributions:


P(Cavity | Toothache) = 2-element vector of 2-element vectors)

• If we know more, e.g., cavity is also given, then we have


P(cavity | toothache, cavity) = 1

• New evidence may be irrelevant, allowing simplification:


P(cavity | toothache, sunny) = P(cavity | toothache) = 0.8

• This kind of inference, sanctioned by domain knowledge, is crucial


•© UW CSE AI Faculty •9
Conditional Probability
• P(A | B) is the probability of A given B
• Assumes that B is the only info known.
• Defined by: P( A  B)
P( A | B) 
P( B)

A AB
True

•© UW CSE AI Faculty •10


Chain Rule/Product Rule
• P(X1, …, Xn) = P(Xn|X1..Xn-1)P(Xn-1|X1..Xn-2)… P(X1)
= P(Xi|X1,..Xi-1)
Dilemma at the Dentist’s

What is the probability of a cavity given a toothache?


What is the probability of a cavity given the probe catches?

•© CSE AI Faculty •12


Inference by Enumeration

P(toothache)=.108+.012+.016+.064
= .20 or 20%
•© UW CSE AI Faculty •13
Inference by Enumeration

P(toothachecavity) = .20 + ??
.072 + .008
.28
•© UW CSE AI Faculty •14
Inference by Enumeration

•© UW CSE AI Faculty •15


Complexity of Enumeration

• Worst case time: O(dn)


– Where d = max arity
– And n = number of random variables
• Space complexity also O(dn)
– Size of joint distribution

• Prohibitive!

•© UW CSE AI Faculty •16


Independence
• A and B are independent iff:
P( A | B)  P( A) These two constraints are
logically equivalent
P( B | A)  P( B)

• Therefore, if A and B are independent:


P( A  B)
P( A | B)   P( A)
P( B)

P( A  B)  P( A) P( B)
•© UW CSE AI Faculty •17
Independence

Complete independence is powerful but rare


What to do if it doesn’t hold?
•© UW CSE AI Faculty •18
Conditional Independence

Instead of 7 entries, only need 5


•© UW CSE AI Faculty •19
Conditional Independence II
P(catch | toothache, cavity) = P(catch | cavity)
P(catch | toothache,cavity) = P(catch |cavity)

Why only 5 entries in table?

•© UW CSE AI Faculty •20


Power of Cond. Independence
• Often, using conditional independence
reduces the storage complexity of the joint
distribution from exponential to linear!!

• Conditional independence is the most basic &


robust form of knowledge about uncertain
environments.

•© UW CSE AI Faculty •21


Bayes Rule Bayes rules!

posterior

P ( x, y )  P ( x | y ) P ( y )  P ( y | x ) P ( x )
P( y | x) P( x) likelihood  prior
P( x y )  
P( y ) evidence

•© UW CSE AI Faculty •22


Computing Diagnostic Prob. from Causal Prob.

E.g. let M be meningitis, S be stiff neck


P(M) = 0.0001,
P(S) = 0.1,
P(S|M)= 0.8

P(M|S)

•© UW CSE AI Faculty •23


Other forms of Bayes Rule
P( y | x) P( x) likelihood  prior
P( x y )  
P( y ) evidence
P( y | x) P( x)
P( x y ) 
 P( y | x) P ( x)
x

P ( x y )  P ( y | x ) P ( x )
posterior  likelihood  prior
Conditional Bayes Rule

P ( y | x, z ) P ( x | z )
P( x y, z ) 
P( y | z )
P ( y | x , z ) P ( x, z )
P( x y, z ) 
 P ( y | x, z ) P ( x | z )
x

P ( x y , z )  P ( y | x , z ) P ( x | z )
Bayes’ Rule & Cond. Independence

•© UW CSE AI Faculty •26

You might also like