Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
34 views

Classification Problems: - Outcome Is Categorical - E.G. Customer Responds or Not

This document discusses binary classification problems and logistic regression. It explains that logistic regression estimates the log odds of the target variable being 1 using the regression equation. The logistic function is used to transform the output of the regression into a probability value between 0 and 1. Maximum likelihood estimation finds the coefficient values that maximize the likelihood of the predicted probabilities matching the actual observed probabilities.

Uploaded by

Siddharth Gupta
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
34 views

Classification Problems: - Outcome Is Categorical - E.G. Customer Responds or Not

This document discusses binary classification problems and logistic regression. It explains that logistic regression estimates the log odds of the target variable being 1 using the regression equation. The logistic function is used to transform the output of the regression into a probability value between 0 and 1. Maximum likelihood estimation finds the coefficient values that maximize the likelihood of the predicted probabilities matching the actual observed probabilities.

Uploaded by

Siddharth Gupta
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

Classification Problems

• Outcome is categorical
• e.g. Customer responds or not
• Defaults or not
• Churns or Not churns
• Binary classification tasks

• Multinomial Class=> Sentiment - +ve, -ve or neutral , Risk=>Hi , Low , medium


• Binary
• Several techniques- Logistic, SVM, Discriminant, NN
Binary Logistic Regression

𝑃 𝑌=1
• 𝑙𝑛 = 𝑍 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 +……. +𝛽𝑚 𝑋𝑚
1−𝑃 𝑌=1

• Where P(Y=1) indicates the probability of the positive outcome

• X1 to Xm are the independent variables

• Non-linear transformation of the regression equation


Odds
• Suppose p = 0.8
• Odds of an event = Prob / (1-Prob)
• = 0.8/(0.2) = 4
• Success is 4 times more likely to happen than failure

• Probability of an event = Odds / (1+Odds)


• 4/(1+4) = 0.8

• Logistic regression estimates log of Odds of the Y variable


Purchase
1.2

0.8

0.6

0.4

0.2

0
0 5 10 15 20 25 30 35
INCOME ($000) LINE FIT PLOT
Purchase Predicted Purchase Linear (Predicted Purchase)

1.2

0.8

0.6
PURCHASE

0.4

0.2

0
0 5 10 15 20 25 30 35

-0.2

-0.4

-0.6
INCOME ($000)

1/Negative values prediction


2/ Residuals will not follow a normal distribution
f(z)= exp(Z)/1+exp(Z) or 1/(1+exp(-Z))
1.1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
-30 -20 -10 0 10 20 30 40
A model that fits better-Logit Function

Probability
Of Event

7
Estimation-Maximum Likelihood Function
• Iterative
• Finds the value of coefficients (βo,β1) such that the predicted
probabilities are as close to the observed probabilities as
possible.

8
To Interpret ! - We work backwards
• Model estimates log(Odds)
𝑃 𝑌=1
• 𝑙𝑛 = 𝑍 = 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 +……. +𝛽𝑚 𝑋𝑚
1−𝑃 𝑌=1
• => Get the antilog to derive the Odds
𝑃 𝑌=1
• = exp 𝑍 = exp( 𝛽0 + 𝛽1 𝑋1 + 𝛽2 𝑋2 +……. +𝛽𝑚 𝑋𝑚 )
1−𝑃 𝑌=1

• => Convert Odds to Probability


𝑒𝑧 1
• Probability (Y=1) = Odds / (1+Odds) = 𝑃 𝑌 = 1 = =
1+𝑒 𝑧 1+𝑒 −𝑧
Estimation-Maximum Likelihood Function
• How well did the function perform ? Does it explain the variance
in the output variable ?
• Maximum likelihood fits the likelihood value ( How likely does the
function (combination of the independent variables) match the actual
probability ?)
• likelihood function which will range from 0 up to a maximum of 1
• -2 log of Likelihood value indicates model estimation fit
• “0” indicates perfect fit
-2LL is also called deviance

We multiply by -2 to make it a chi-square


distribution
10

You might also like