Bayes’ theorem describes the probability of occurrence of an event related to
any condition which is also called as probability of “causes”.
Conditional probability is known as the possibility of an event happening, based on the
existence of a previous event. It is calculated by multiplying the probability of the
preceding event by the renewed probability of the succeeding, or conditional, event.
Conditional Probability: Bayes Theorem
Bayes Theorem Statement
Let E1, E2,…, En be a set of events associated with a sample space S, where all
the events E1, E2,…, En have nonzero probability of occurrence and they form a
partition of S. Let A be any event associated with S, then according to Bayes
theorem,
for any k = 1, 2, 3, …., n
Terminologies:
Hypotheses: The events E1, E2,… En is called the hypotheses
Priori Probability: The probability P(Ei) is considered as the priori probability of
hypothesis Ei
Posteriori Probability: The probability P(Ei|A) is considered as the posteriori probability
of hypothesis Ei
Bayes’ theorem is also called the formula for the probability of “causes”.
Hence, the above formula gives the probability of a particular Ei (i.e. a “Cause”), given
that the event A has occurred.
Naive Bayes classifiers
Is an algorithms based on Bayes’ Theorem.
Despite “naive” assumption of feature independence, these classifiers are utilized
for their simplicity and efficiency in machine learning.
What is Naive Bayes classifiers?
Naive Bayes classifiers are a collection of classification algorithms based on
Bayes’ Theorem.
Here every pair of features being classified is independent of each other.
Why it is called Naive Bayes?
“Naive” indicates that simplifying assumption made by Naïve Bayes classifier.
The classifier assumes that features used to describe an observation are
conditionally independent, given the class label.
“Bayes” refers to Reverend Thomas Bayes, an 18th-century statistician and
theologian who formulated Bayes’ theorem.
The dataset is divided into 2 parts - feature matrix and the response vector.
Feature matrix contains all the vectors(rows) of dataset in which each
vector(column) consists of the value of dependent features. Example: features
are ‘Outlook’, ‘Temperature’, ‘Humidity’ and ‘Wind’.
Response vector contains the value of class variable(prediction or output) for
each row of feature matrix. Example: ‘Play Tennis’.
Assumption of Naive Bayes
Feature independence: The features of the data are conditionally independent
of each other, given the class label.
Continuous features are normally distributed: If a feature is continuous, then
it is assumed to be normally distributed within each class.
Discrete features have multinomial distributions: If a feature is discrete, then
it is assumed to have a multinomial distribution within each class.
Features are equally important: All features are assumed to contribute equally
to the prediction of the class label.
No missing data: The data should not contain any missing values.
Bayes Theorem Formula
If A and B are two events, then the formula for the Bayes theorem is given by:
Where P(A|B) is the probability of condition when event A is occurring while event B has
already occurred.
From the above bayes statement formula for Naïve Bayes classification
derived as follows :
Consider any two events A and B are independent, then,
P(A,B) = P(A) P(B)
Hence, we reach to the result:
which can be expressed as:
Now, as the denominator remains constant for a given input, we can remove that
term:
Now, to create a classifier model, find the probability of given set of inputs for all
possible values of the class variable y and pick up the output with maximum
probability.
This can be expressed mathematically as:
So, finally, we are left with the task of calculating P(Y) and P(xi|y).
Note that P(Y) is also called class probability and P(xi|y) is called conditional
probability.
Example
As VNB(no) >VNB(yes)
The Given instance is classified as the class(table-tennis) No.
