Artificial Intelligence

Module V
Probabilistic reasoning in AI

• Uncertainty
• To represent uncertain knowledge, where we are not sure
about the predicates, we need uncertain reasoning or
probabilistic reasoning.
• Causes of uncertainty:
• Information occurred from unreliable sources.
• Experimental Errors
• Equipment fault
• Temperature variation
• Climate change.

Department of Computer Science 3

Probabilistic reasoning:
• Probabilistic reasoning is a way of knowledge
representation where we apply the concept of
probability to indicate the uncertainty in knowledge.
• In probabilistic reasoning, we combine probability
theory with logic to handle the uncertainty.
• We use probability in probabilistic reasoning because it
provides a way to handle the uncertainty that is the
result of someone's laziness and ignorance.
Need of Probabilistic reasoning
Terms with Probabilistic reasoning
• Probability:
• Probability can be defined as a chance that an uncertain event
will occur.
• It is the numerical measure of the likelihood that an event will
occur.
• The value of probability always remains between 0 and 1 that
represent ideal uncertainties.
Terms with Probabilistic reasoning
Terms with Probabilistic reasoning
Conditional Probability
• Conditional probability is a probability of occurring an
event when another event has already happened.
• If to calculate the event A when event B has already
occurred, "the probability of A under the conditions of
B", it can be written as:

Department of Computer Science 9

Conditional Probability
• If the probability of A is given and we need to find the
probability of B, then it will be given as:

Department of Computer Science 10

Terms with Probabilistic reasoning
Bayes' theorem in AI
• Bayes' rule, Bayes' law, or Bayesian reasoning
• determines the probability of an event with uncertain knowledge.
• it relates the conditional probability and marginal probabilities of
two random events.
• named after the British mathematician Thomas Bayes.
• The Bayesian inference is an application of Bayes' theorem,
which is fundamental to Bayesian statistics.
• It is a way to calculate the value of P(B|A) with the knowledge of
P(A|B).
• allows updating the probability prediction of an event by
observing new information of the real world.
Department of Computer Science 12
Bayes' theorem in AI
• Bayes' theorem can be derived using product rule and
conditional probability of event A with known event B:

The above equation (a) is called as Bayes' rule or Bayes' theorem.

This equation is basic of most modern AI systems for probabilistic inference.

Department of Computer Science 13

Bayes' theorem in AI
• It shows the simple relationship between joint and
conditional probabilities

Department of Computer Science 14

Applying Bayes' rule
• allows us to compute the single term P(B|A) in terms of
P(A|B), P(B), and P(A).
• useful in cases where we have a good probability of
these three terms and want to determine the fourth
one.
• Suppose we want to perceive the effect of some
unknown cause, and want to compute that cause, then
the Bayes' rule becomes:

Department of Computer Science 15

Applying Bayes' rule

Department of Computer Science 16

 Applying Bayes' rule
standard deck of playing cards, a single card is drawn. The probability that the card is king is 4
alculate posterior probability P(King|Face), which means the drawn face card is a king card.

Department of Computer Science 17

Application of Bayes' theorem in AI

Department of Computer Science 18

 Naïve Bayes Classifier Algorithm
•Naïve Bayes algorithm is a supervised learning algorithm, which is
based on Bayes theorem and used for solving classification problems.
•It is mainly used in text classification that includes a high-dimensional
training dataset.
•Naïve Bayes Classifier is one of the simple and most effective.
•It is a probabilistic classifier, which means it predicts on the basis
of the probability of an object.
•Some popular examples of Naïve Bayes Algorithm are spam filtration,
Sentimental analysis, and classifying articles.
Why is it called Naïve Bayes?
The Naïve Bayes algorithm is comprised of two words Naïve and Bayes,
Which can be described as:
•Naïve: It is called Naïve because it assumes that the occurrence of a certain
feature is independent of the occurrence of other features.
•Bayes: It is called Bayes because it depends on the principle of
 Working of Naïve Bayes' Classifier

Working of Naïve Bayes' Classifier can be understood with the

help of the below example:
Suppose we have a dataset of weather conditions and
corresponding target variable "Play". So using this dataset we
need to decide that whether we should play or not on a
particular day according to the weather conditions. So to solve
this problem, we need to follow the below steps:
1.Convert the given dataset into frequency tables.
2.Generate Likelihood table by finding the probabilities of given
features.
3.Now, use Bayes theorem to calculate the posterior probability.
 should play or not?
Solution: To solve this, first consider the below
dataset: Outlook Play
0 Rainy Yes
1 Sunny Yes
2 Overcast Yes
3 Overcast Yes
4 Sunny No
5 Rainy Yes
6 Sunny Yes
7 Overcast Yes
8 Rainy No
9 Sunny No
10 Sunny Yes
11 Rainy No
12 Overcast Yes
13 Overcast Yes
Frequency table for the Weather Conditions:
Weather Yes No
Overcast 5 0
Rainy 2 2
Sunny 3 2
Total 10 4

Likelihood table weather condition:

Weather No Yes
Overcast 0 5 5/14= 0.35
Rainy 2 2 4/14=0.29
Sunny 2 3 5/14=0.35
All 4/14=0.29 10/14=0.71
 Applying Bayes'theorem:
Weather No Yes
Overcast 0 5 5/14= 0.35
Rainy 2 2 4/14=0.29
Sunny 2 3 5/14=0.35
All 4/14=0.29 10/14=0.71

Problem: If the weather is sunny, then the Player should

play or not?

P(Yes|Sunny)= P(Sunny|Yes)*P(Yes)/P(Sunny)
P(Sunny|Yes)= 3/10= 0.3
P(Sunny)= 0.35
P(Yes)=0.71
So P(Yes|Sunny) = 0.3*0.71/0.35= 0.60
 Applying Bayes'theorem:
Weather No Yes
Overcast 0 5 5/14= 0.35
Rainy 2 2 4/14=0.29
Sunny 2 3 5/14=0.35
All 4/14=0.29 10/14=0.71

Problem: If the weather is sunny, then the Player should

play or not?

P(No|Sunny)= P(Sunny|No)*P(No)/P(Sunny)
P(Sunny|NO)= 2/4=0.5
P(No)= 0.29
P(Sunny)= 0.35
So P(No|Sunny)= 0.5*0.29/0.35 = 0.41
 Applying Bayes'theorem:
Weather No Yes
Overcast 0 5 5/14= 0.35
Rainy 2 2 4/14=0.29
Sunny 2 3 5/14=0.35
All 4/14=0.29 10/14=0.71

Problem: If the weather is sunny, then the Player should

play or not?
So P(Yes|Sunny) = 0.3*0.71/0.35= 0.60
So P(No|Sunny)= 0.5*0.29/0.35 = 0.41

P(Yes|Sunny)>P(No|Sunny)

Hence on a Sunny day, Player can play the

game.
Advantages of Naïve Bayes Classifier:
•Naïve Bayes is one of the fast and easy ML algorithms to predict a class of
datasets.
•It can be used for Binary as well as Multi-class Classifications.
•It performs well in Multi-class predictions as compared to the other
Algorithms.
•It is the most popular choice for text classification problems.
Disadvantages of Naïve Bayes Classifier:
•Naive Bayes assumes that all features are independent or unrelated, so it
cannot learn the relationship between features.
•May assign zero probability to unseen events, leading to poor
generalization.
Applications of Naïve Bayes Classifier:
•It is used for Credit Scoring.
•It is used in medical data classification.
•It can be used in real-time predictions because Naïve Bayes Classifier is
Bayesian Belief Network in AI
• key computer technology for dealing with probabilistic events and
to solve a problem which has uncertainty.
• "A Bayesian network is a probabilistic graphical model which
represents a set of variables and their conditional dependencies
using a directed acyclic graph.”
• It is also called a Bayes network, belief network, decision
network, or Bayesian model.
• It is a classifier with no dependency on attributes i.e it is condition
independent.
• It can also be used in various tasks including prediction, anomaly
detection, diagnostics, automated insight, reasoning, time
series prediction, and decision making under uncertainty.
Department of Computer Science 27
Bayesian Belief Network in AI
• Bayesian Network can be used for building models from
data and experts opinions, and it consists of two parts:
• Directed Acyclic Graph
• Table of conditional probabilities.
• The generalized form of Bayesian network that
represents and solve decision problems under uncertain
knowledge is known as an Influence diagram.

Department of Computer Science 28

Bayesian Belief Network in AI

Department of Computer Science 29

Bayesian Belief Network in AI
• Each node in the Bayesian network
has condition probability
distribution P(Xi |Parent(Xi) ),
which determines the effect of the
parent on that node.
• Bayesian network is based on Joint
probability distribution and
conditional probability.

Department of Computer Science 30

Joint Probability Distribution
• If we have variables x1, x2, x3,....., xn, then the
probabilities of a different combination of x1, x2, x3..
xn, are known as Joint probability distribution.

Department of Computer Science 31

Joint Probability Distribution

Department of Computer Science 32

Joint Probability Distribution

Department of Computer Science 33

Joint Probability Distribution

Department of Computer Science 34

Joint Probability Distribution

P( j | m, a, ¬b, ¬e) P( m, a, ¬b, ¬e)

= P( j | a) P( m | a, ¬b, ¬e) P( a, ¬b, ¬e)

= P( j | a) P( m | a) P( a | ¬b, ¬e) P( ¬b, ¬e)

= P( j | a) P( m | a) P( a | ¬b, ¬e) P( ¬b) P( ¬e)

Department of Computer Science 35

Joint Probability Distribution

Department of Computer Science 36

Exact Inference in Bayesian
Networks
• Exact inference in Bayesian Networks is a fundamental
process used to compute the probability distribution of
a subset of variables, given observed evidence on a set
of other variables.
Basics of Inference in Bayesian
Networks
Methods of Exact Inference
1.Variable Elimination

2.Junction Tree Algorithm

3.Belief Propagation
1. Variable elimination
• Popular exact inference technique that systematically
sums out the variables not of interest.
• The process involves manipulating and combining the
network's CPTs to answer queries efficiently.
1. Variable elimination
2. Junction Tree Algorithm
• also known as the Clique Tree Algorithm, is a more
structured approach that converts the Bayesian
Network into a tree structure called a "junction tree" or
"clique tree," where each node (clique) contains a
subset of variables that form a complete (fully
connected) subgraph in the network.
2. Junction Tree Algorithm
3. Belief Propagation
• used particularly in networks that form a tree structure or can be
restructured into a tree-like form using the Junction Tree
Algorithm.
• It involves passing messages between nodes and uses these
messages to compute marginal probabilities at each node.
• Belief Propagation is especially effective in tree-structured
networks where messages can be propagated without loops,
ensuring that each node's final belief is computed exactly once
all messages have been passed.
What is an Uncertain Domain in AI?
• refers to a field or environment where the information
available is incomplete, ambiguous, noisy, or inherently
unpredictable.
• Unlike deterministic domains where outcomes can be
predicted with certainty given the inputs, uncertain
domains require AI systems to handle and reason about
uncertainty in a structured manner.
Characteristics of Uncertain Domains
Importance of Handling Uncertainty
Representing Knowledge in an
Uncertain Domain
1. Probabilistic Reasoning
2. Hidden Markov Models
3. Markov Decision Processes
4. Fuzzy Logic
5. Dempster-Shafer Theory
6. Belief Networks
7. Case-Based Reasoning
1. Probabilistic Reasoning
• involves representing knowledge using probability
theory to manage uncertainty.
• This approach is widely used in AI for tasks such as
diagnosis, prediction, and decision-making under
uncertainty.
• Bayesian Networks
• Bayesian networks (BNs) are graphical models that
represent the probabilistic relationships among a set of
variables. Each node in a BN represents a variable, and
the edges represent conditional dependencies. BNs
allow for efficient computation of posterior probabilities
given observed evidence.
2. Hidden Markov Models
• used to model time series data where the system being
modeled is assumed to be a Markov process with hidden
states.
• HMMs are widely used in speech recognition,
bioinformatics, and other sequential data applications.
• Example: In speech recognition, the observed sound
waves are modeled as emissions from hidden phonetic
states, allowing the system to decode spoken language.
3. Markov Decision Processes
• provide a framework for modeling decision-making in
environments with stochastic dynamics.
• MDPs consist of states, actions, transition probabilities,
and rewards, enabling the computation of optimal
policies for decision-making.
• Example: An autonomous robot navigating a grid world
can use an MDP to determine the optimal path to its
destination while accounting for uncertain movements
and rewards.
4. Fuzzy Logic
• is an approach to reasoning that deals with approximate rather than
fixed and exact values.
• Unlike traditional binary logic, fuzzy logic variables can have a truth
value that ranges between 0 and 1, representing the degree of truth.
• Fuzzy Sets and Membership Functions
• Fuzzy sets allow for the representation of concepts with vague
boundaries. Each element in a fuzzy set has a membership value
indicating its degree of belonging to the set.
• Example: In a temperature control system, the concept of "warm"
can be represented as a fuzzy set with a membership function
assigning values between 0 (not warm) and 1 (completely warm) to
different temperatures.
4. Fuzzy Logic
• Fuzzy Rules and Inference
• Fuzzy rules define the relationships between fuzzy
variables using if-then statements. Fuzzy inference
systems apply these rules to input data to derive
conclusions.

• Example: A fuzzy rule for a temperature control system

might be: "If the temperature is high, then reduce the
heater power."
5. Dempster-Shafer Theory
• also known as evidence theory, is a mathematical
framework for modeling uncertainty without the need
for precise probabilities.
• It allows for the combination of evidence from different
sources to calculate the degree of belief (or plausibility)
for various hypotheses.
• Example: In an expert system for fault diagnosis,
evidence from different sensors can be combined using
Dempster-Shafer theory to assess the likelihood of
different fault conditions.
6. Belief Networks
• extend Bayesian networks by allowing for the
representation of uncertainty in the strength of the
dependencies between variables.
• They provide a way to handle imprecise and incomplete
knowledge.
• Example: A belief network for an intelligent tutoring
system might include nodes for student knowledge,
engagement, and performance, with edges representing
uncertain dependencies between these factors.
7. Case-Based Reasoning
• approach where past cases (experiences) are used to
solve new problems.
• In uncertain domains, CBR can be combined with
probabilistic methods to estimate the likelihood of
various outcomes based on similar past cases.
• Example: A customer support system can use CBR to
suggest solutions based on previous similar customer
queries, adjusting recommendations based on the
uncertainty of the current context.