Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 11 views

    Naive Bayes

    Uploaded by

    Thành Dương Nhật
    The document provides an overview of the naive Bayes classifier for classification problems. It describes how the naive Bayes algorithm uses Bayes' theorem to calculate the probability of a sample belonging to each class based on the features. It also discusses how the classifier is trained by calculating the probabilities of each feature value for each class using maximum likelihood estimation. The document notes some relevant issues with the naive Bayes classifier, such as dealing with zero probabilities and continuous valued features.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Naive Bayes

    Uploaded by

    Thành Dương Nhật
    11 views9 pages
    The document provides an overview of the naive Bayes classifier for classification problems. It describes how the naive Bayes algorithm uses Bayes' theorem to calculate the probability of a sample belonging to each class based on the features. It also discusses how the classifier is trained by calculating the probabilities of each feature value for each class using maximum likelihood estimation. The document notes some relevant issues with the naive Bayes classifier, such as dealing with zero probabilities and continuous valued features.

    Original Title

    5. Naive Bayes

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document provides an overview of the naive Bayes classifier for classification problems. It describes how the naive Bayes algorithm uses Bayes' theorem to calculate the probability of a sample belonging to each class based on the features. It also discusses how the classifier is trained by calculating the probabilities of each feature value for each class using maximum likelihood estimation. The document notes some relevant issues with the naive Bayes classifier, such as dealing with zero probabilities and continuous valued features.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    11 views9 pages

    Naive Bayes

    Uploaded by

    Thành Dương Nhật
    The document provides an overview of the naive Bayes classifier for classification problems. It describes how the naive Bayes algorithm uses Bayes' theorem to calculate the probability of a sample belonging to each class based on the features. It also discusses how the classifier is trained by calculating the probabilities of each feature value for each class using maximum likelihood estimation. The document notes some relevant issues with the naive Bayes classifier, such as dealing with zero probabilities and continuous valued features.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 9

    Naive Bayes classifier

    Tuan Nguyen
    April 20, 2023

    AI For Everyone AI4E April 20, 2023 1/9


    Overview

    Classification problem

    Bayes theorem

    Naive Bayes algorithm

    Relevant Issues

    AI For Everyone AI4E April 20, 2023 2/9


    Classification problem

    The goal in classification is to take an input vector x with d features


    x = [x1 , x2 , ..., xd ]T and to assign it to one of K discrete classes Ck where
    k = 1,...,K.

    We need to calculate the probability of a given sample belonging to each


    class

    p(y = Ck |x) = p(Ck |x)


    K
    X
    p(Ck |x) = 1
    k=1

    Then we pick the class with the highest probability

    c = arg max p(y = Ck |x)


    Ci

    AI For Everyone AI4E April 20, 2023 3/9


    Bayes

    Bayes’ theorem

    c = arg max p(Ck |x)


    Ck
    p(x|Ck )p(Ck )
    = arg max
    Ck p(x)
    = arg max p(x|Ck )p(Ck )
    Ck
    d
    Y
    = arg max p(xi |Ck )p(Ck )
    Ck i=1

    Assumption: Features in the sample are independent!!!


    In the training phase, based on the dataset, we will calculate all
    components, p(Ck ), p(xi |Ck )

    AI For Everyone AI4E April 20, 2023 4/9


    Training

    Figure 1: Dataset

    AI For Everyone AI4E April 20, 2023 5/9


    Training (cont.)

    number of samples play tennis 9


    p(C1 ) = p(Play = Yes) = =
    total number 14
    5
    p(C0 ) = p(Play = No) = 1 − p(C1 ) =
    14
    Each component
    Outlook Play=Yes Play=No
    Sunny 2/9 3/5
    Overcast 4/9 0/5
    Rain 3/9 2/5
    Exercise: Calculate the table for each feature Temperature, Humidity,
    Wind.

    AI For Everyone AI4E April 20, 2023 6/9


    Prediction

    Given a new instance, x=(Outlook=Sunny, Temperature=Cool,


    Humidity=High, Wind=Strong). We need to predict whether the player
    plays tennis or not.
    Lookup table
    p(Outlook=Sunny|Yes) = 2/9 p(Outlook=Sunny|No) = 3/5
    p(Temperature=Cool|Yes) = 3/9 p(Temperature=Cool|No) = 1/5
    p(Huminity=High|Yes) = 3/9 p(Huminity=High|No) = 4/5
    p(Wind=Strong|Yes) = 3/9 p(Wind=Strong|No) = 3/5
    p(Play=Yes) = 9/14 p(Play=No) = 5/14

    Then we calculate and compare p(Yes|x) and p(No|x), then make the
    prediction

    AI For Everyone AI4E April 20, 2023 7/9


    Zero conditional probability Problem

    ▶ If no example contains the attribute value, the probability will be


    zero, p(Overcast|No) = 0
    ▶ Then during the testing phase, the posterior of the example
    containing this attribute will be zero, p(No|x[Overcast]) = 0
    ▶ Laplace smoothing

    Nik + α
    p(xi |Ck ) =
    Nk + dα
    α is a positive number, default value is 1.
    ▶ When there are a lot of feature, the multiplication of probability
    could lead to 0, we can take the log then compare.
    d
    X
    log p(Ck |x) = log p(Ck ) + log p(xi |Ck )
    i=1

    AI For Everyone AI4E April 20, 2023 8/9


    Continuous-valued Input Attributes

    ▶ Numberless values for an attribute


    ▶ Conditional probability modeled with the normal distribution

    (xi − µik )2
     
    1
    p(xi |Ck ) = √ exp − 2
    2πσik 2σik

    ▶ In the training phase, we need to find σik , µik for each feature in
    each category based on the dataset.

    AI For Everyone AI4E April 20, 2023 9/9

    You might also like