Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
5 views

Multinomial Logistic Regression

This document discusses multinomial logistic regression, which is used for multiclass classification problems with more than two categories. It explains that multinomial logistic regression uses multiple logistic regression equations to model the probabilities of each category compared to a baseline or reference category. The document provides examples of multinomial logistic regression equations for problems with three categories. It also discusses how to interpret model parameters, validate the model, select features, and make predictions for new data using the probability prediction equations.

Uploaded by

Rushad Dumasia
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
5 views

Multinomial Logistic Regression

This document discusses multinomial logistic regression, which is used for multiclass classification problems with more than two categories. It explains that multinomial logistic regression uses multiple logistic regression equations to model the probabilities of each category compared to a baseline or reference category. The document provides examples of multinomial logistic regression equations for problems with three categories. It also discusses how to interpret model parameters, validate the model, select features, and make predictions for new data using the probability prediction equations.

Uploaded by

Rushad Dumasia
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

Multinomial Logistic Regression

Dinesh Kumar
Multiclass Classification
• Outcome of a sporting event – Win, Draw or Loss

• Brand Propensity – Nike, Adidas, Puma

• Image Classification – Car, Truck, cycle, Auto

• Text Classification – Positive, Negative, Neutral,


Sarcastic
Multinomial Logistic Regression (MLR)
Model
1. For MLR model with n classes, (n-1) equations are used. One class is used
as reference category. (or Base Category)
2. Logit models are written by (by taking class 0 as reference category):

𝜋𝑖
ln( ) = 𝛽0𝑖 + 𝛽1𝑖 𝑥1 + 𝛽2𝑖 𝑥2 +. . . +𝛽𝑘𝑖 𝑥𝑘
𝜋0
Multinomial LR equations with three
categories

 P(Y = 1) 
 1 ( x) = ln   = Z1 =  0 + 1 X 1
 P(Y = 0) 
 P(Y = 2) 
 2 ( x) = ln   = Z 2 =  0 + 1 X 1
 P(Y = 0) 
Probability Prediction Equations

1
P(Y = 0) = 0 + 1 X1  0 +1 X1
1+ e +e
0 + 1 X1
e
P(Y = 1) = 0 + 1 X1  0 +1 X1
1+ e +e
 0 +1 X1
e
P(Y = 2) = 0 + 1 X1  0 +1 X1
1+ e +e
Interpretation of Beta
• Exp(Beta) is interpreted as the change in odds ratio when the
value of a feature is changed.
Model Validation and Selection

• Similar to Binary Logistic Regression:

• Z test or Wald’s test for statistical significance.

• Reduction in Deviance (-2LL) for feature selection.

• Akaike information criteria (AIC) and Bayesian


Information Criteria (BIC)

• Classification matrix
Akaike Information Criteria (AIC)
• Akaike information criteria (AIC) is used in model selection.
The smaller AIC value is better.

• AIC = -2LL(Model) + 2(k+1)

• Where k is the number of features. 2(k+1) is added as a


penalty since any additional covariates may improve the
likelihood function.
Bayesian Information Criteria (BIC)
• Bayesian information criteria (BIC) is used for model selection.

• BIC = -2LL(Model) + (K+1) ln(N)

• K is the number of features and N is the number of records in


the data.

• Smaller BIC is better.


EPL Example
Parameters

1
 P(Y1 = 0) =
ln( 1 ) = Z1 =  0 + 1  HTGD = 0.20 + 0.799 HTGD 1 + e Z1 + e Z2
0
e Z1
 P(Y1 = 1) =
ln( 2 ) = Z 2 =  0 + 1  HTGD = 0.470 + 2.206 HTGD 1 + e Z1 + e Z2
0
e Z2
P(Y1 = 2) =
1 + e Z1 + e Z2
Classification Table
Model with FGS (First Goal Scored as
Feature)
References
• D W Hosmer and S Lemeshow, “Applied Logistic Regression”,
Wiley Interscinece, 2000.

• D G Kleinbaum and M Klein, “Logistic Regression – A self


learning Text”, Springer, 2012

You might also like