Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
109 views

What Is Naive Bayes Algorithm?

The Naive Bayes algorithm is a classification technique based on Bayes' theorem that assumes independence between predictors. It calculates the probability of a class given attribute values using the frequency of values in the training data. Even though it assumes independence, Naive Bayes often performs well on real-world problems and is particularly suitable for large datasets due to its simplicity and speed.

Uploaded by

JUAN ESTEBAN LOPEZ BEDOYA
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
109 views

What Is Naive Bayes Algorithm?

The Naive Bayes algorithm is a classification technique based on Bayes' theorem that assumes independence between predictors. It calculates the probability of a class given attribute values using the frequency of values in the training data. Even though it assumes independence, Naive Bayes often performs well on real-world problems and is particularly suitable for large datasets due to its simplicity and speed.

Uploaded by

JUAN ESTEBAN LOPEZ BEDOYA
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

What is Naive Bayes

algorithm?
What is Naive Bayes algorithm?
It is a classification technique based on Bayes’ Theorem with an
assumption of independence among predictors. In simple terms, a
Naive Bayes classifier assumes that the presence of a particular feature
in a class is unrelated to the presence of any other feature.
What is Naive Bayes algorithm?
For example, a fruit may be considered to be an apple if it is red, round,
and about 3 inches in diameter. Even if these features depend on each
other or upon the existence of the other features, all of these
properties independently contribute to the probability that this fruit is
an apple and that is why it is known as ‘Naive’.
What is Naive Bayes algorithm?
Naive Bayes model is easy to build and particularly useful for very large
data sets. Along with simplicity, Naive Bayes is known to outperform
even highly sophisticated classification methods.
What is Naive Bayes algorithm?
Bayes theorem provides a way of calculating posterior probability P(c|x) from P(c),
P(x) and P(x|c). Look at the equation below:

Above,
• P(c|x) is the posterior probability of class (c, target) given predictor (x, attributes).
• P(c) is the prior probability of class.
• P(x|c) is the likelihood which is the probability of predictor given class.
• P(x) is the prior probability of predictor.
How Naive Bayes algorithm works?
Let’s understand it using an example. Below I have a training data set of
weather and corresponding target variable ‘Play’ (suggesting
possibilities of playing). Now, we need to classify whether players will
play or not based on weather condition. Let’s follow the below steps to
perform it.
• Step 1: Convert the data set into a frequency table
• Step 2: Create Likelihood table by finding the probabilities
• Step 3: Now, use Naive Bayesian equation to calculate the posterior
probability for each class. The class with the highest posterior
probability is the outcome of prediction.
How Naive Bayes algorithm works?
How Naive Bayes algorithm works?
Problem: Players will play if weather is sunny. Is this statement is correct?
We can solve it using above discussed method of posterior probability.
P(Yes | Sunny) = P( Sunny | Yes) * P(Yes) / P (Sunny)
Here we have P (Sunny |Yes) = 3/9 = 0.33, P(Sunny) = 5/14 = 0.36, P( Yes)=
9/14 = 0.64
Now, P (Yes | Sunny) = 0.33 * 0.64 / 0.36 = 0.60, which has higher
probability.
Naive Bayes uses a similar method to predict the probability of different
class based on various attributes.
Anything else?

Yes please, another example!


Game Prediction Using Bayes' Theorem
Let's continue our Naive Bayes tutorial and predict the future with
some weather data.
Here we have our data, which comprises the day, outlook, humidity,
and wind conditions. The final column is 'Play,' i.e., can we play outside,
which we have to predict.
Game Prediction Using Bayes' Theorem
Game Prediction Using Bayes' Theorem
First, we will create a frequency table using each attribute of the dataset.
Game Prediction Using Bayes' Theorem
For each frequency table, we will generate a likelihood table.

• Likelihood of ‘Yes’ given ‘Sunny‘ is:


P(c|x) = P(Yes|Sunny) = P(Sunny|Yes)* P(Yes) / P(Sunny) = (0.3 x 0.71) /0.36 = 0.591 
• Similarly, the likelihood of ‘No’ given ‘Sunny‘ is:
P(c|x) = P(No|Sunny) = P(Sunny|No)* P(No) / P(Sunny) = (0.4 x 0.36) /0.36 = 0.40
Game Prediction Using Bayes' Theorem
Now, in the same way, we need to create the Likelihood Table for other
attributes as well.
Game Prediction Using Bayes' Theorem
Suppose we have a Day with the following values :
• Outlook = Rain
• Humidity = High
• Wind = Weak
• Play = ?

• So, with the data, we have to predict wheter "we can play on that day or not."
• Likelihood of 'Yes' on that Day = P(Outlook = Rain|Yes)*P(Humidity= High|Yes)* P(Wind= Weak|
Yes)*P(Yes)
• = 2/9 * 3/9 * 6/9 * 9/14 = 0.0199
• Likelihood of 'No' on that Day = P(Outlook = Rain|No)*P(Humidity= High|No)* P(Wind= Weak|No)*P(No)
• = 2/5 * 4/5 * 2/5 * 5/14 = 0.0166
• Now, when we normalize the value, we get:
• P(Yes) =  0.0199 / (0.0199+ 0.0166) = 0.55
• P(No) = 0.0166 / (0.0199+ 0.0166)  = 0.45
• Our model predicts that there is a 55% chance there will be a game tomorrow.
So?

At the end!
What are the Pros and Cons of Naive
Bayes?
Pros:
• It is easy and fast to predict class of test data set. It also perform well
in multi class prediction
• When assumption of independence holds, a Naive Bayes classifier
performs better compare to other models like logistic regression and
you need less training data.
• It perform well in case of categorical input variables compared to
numerical variable(s). For numerical variable, normal distribution is
assumed (bell curve, which is a strong assumption).
What are the Pros and Cons of Naive
Bayes?
Cons:
• If categorical variable has a category (in test data set), which was not
observed in training data set, then model will assign a 0 (zero) probability
and will be unable to make a prediction. This is often known as “Zero
Frequency”. To solve this, we can use the smoothing technique. One of the
simplest smoothing techniques is called Laplace estimation.
• On the other side naive Bayes is also known as a bad estimator, so the
probability outputs from predict_proba are not to be taken too seriously.
• Another limitation of Naive Bayes is the assumption of independent
predictors. In real life, it is almost impossible that we get a set of predictors
which are completely independent

You might also like