Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
25 views

Lecture 4-Logistic Regression

Logistic regression is a supervised machine learning algorithm used for classification tasks, predicting the probability of an instance belonging to a specific class. It extends linear regression by mapping outputs to a range between 0 and 1 using the sigmoid function, making it suitable for binary and multinomial classification problems. Key assumptions include independent observations, binary dependent variables, and a linear relationship between independent variables and log odds.

Uploaded by

VRP Productions
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
25 views

Lecture 4-Logistic Regression

Logistic regression is a supervised machine learning algorithm used for classification tasks, predicting the probability of an instance belonging to a specific class. It extends linear regression by mapping outputs to a range between 0 and 1 using the sigmoid function, making it suitable for binary and multinomial classification problems. Key assumptions include independent observations, binary dependent variables, and a linear relationship between independent variables and log odds.

Uploaded by

VRP Productions
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Logistic Regression

➢ In Linear regression, we predicts the numerical variables, but it is not suitable for predicting the categorical
variables.
➢ When categorical variables are involved, it is called classification problem.
➢ Logistic regression is a supervised machine learning algorithm used for classification tasks where the goal is
to predict the probability that an instance belongs to a given class or not.
➢ Logistic regression is a statistical algorithm which analyzes the relationship between one or more
independent variables and classifies data into discrete classes.
➢ It is extensively used in predictive modeling, where the model estimates the mathematical probability of
whether an instance belongs to a specific category or not.
➢ For example, 0 – represents a negative class; 1 – represents a positive class.
➢ Logistic regression is commonly used in binary classification problems where we use sigmoid function, that
takes input as independent variables and produces a probability value between 0 and 1, which reveals the
outcome variable falls either of the two categories (0 and 1).
➢ For example, we have two classes Class 0 and Class 1 if the value of the logistic function for an input is
greater than 0.5 (threshold value) then it belongs to Class 1 otherwise it belongs to Class 0.
➢ It’s referred to as regression because it is the extension of linear regression but is mainly used for
classification problems.
➢ Logistic regression can be viewed as an extension of linear regression, but the only difference is that output
of linear regression can be extremely high numbers, this needs to be mapped into the range of 0 to 1 as
probability can have values in the range of 0 to 1.
➢ Logistic regression is a predictive algorithm and is based on the concept of probability (0,1 ) range,
whereas Linear regression range is (-α to +α)
➢ The target variable is categorical.
➢ This problem is solved using logit function/ log odds. (Logistic Regression predicts the probability of
occurrence of a binary event using a logit function.)
➢ Odds and probability( likelihood) are two sides of a coin and represents the uncertainty.
➢ Logistic regression uses a log of odds as the dependent variable.
➢ The Odds are defined as the ratio of the probability of success and the probability of failure. ( Probability of
an event and probability of an event that is not happening)

𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑎𝑛 𝑒𝑣𝑒𝑛𝑡
➢ Odds =
𝑃𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑛𝑜𝑛 𝑒𝑣𝑒𝑛𝑡
𝑃
=
1−𝑃
𝑃
Here Odds is = 𝑒𝑦
1−𝑃
𝑃
Log odds = log ( ) = y
1−𝑃
where y = β0 + β1 𝑥
𝑃(𝑥)
log ( ) = β0 + β1 𝑥
1−𝑃(𝑥)

𝑒𝑥𝑝(β0
+ β1 𝑥 )
P(x) =
1+ 𝑒𝑥𝑝(β0 + β1 𝑥 )

Dividing Numerator and denominator by numerator

1 1
P(x) = (same as sigmoid function p = )
1+ 𝑒𝑥𝑝(− β0 + β1 𝑥 ) 1+𝑒 −𝑦

1
=
1+𝑒
− β0 + β1 𝑥

Thus, the hypothesis function of logistic regression is

1
ℎθ (x) =
1+𝑒
− β0 + β1 𝑥
➢ The sigmoid function also called the logistic function gives an ‘S-shaped curve that can take any real-valued number and
map it into a value between 0 and 1.
➢ If the curve goes to positive infinity, y predicted will become 1 and If the curve goes to negative infinity, y predicted will
become 0.
➢ If the output of the sigmoid function is more than 0.5, we can classify the outcome as 1 or YES and if it is less than 0.5, we
can classify it as 0 or NO.
➢ .
Types of Logistic Regression
On the basis of the categories, Logistic Regression can be classified into three types:
1.Binomial: In binomial Logistic regression, there can be only two possible types of the dependent
variables, such as 0 or 1, Pass or Fail, etc.
2.Multinomial: In multinomial Logistic regression, there can be 3 or more possible unordered types of the
dependent variable, such as “cat”, “dogs”, or “sheep”
3.Ordinal: In ordinal Logistic regression, there can be 3 or more possible ordered types of dependent
variables, such as “low”, “Medium”, or “High”.
Assumptions of Logistic Regression
1.Independent observations: Each observation is independent of the other. meaning there is no
correlation between any input variables.
2.Binary dependent variables: It takes the assumption that the dependent variable must be binary or
dichotomous, meaning it can take only two values. For more than two categories SoftMax functions are
used.
3.Linearity relationship between independent variables and log odds: The relationship between the
independent variables and the log odds of the dependent variable should be linear.
4.No outliers: There should be no outliers in the dataset.
5.Large sample size: The sample size is sufficiently large
Let us Assume a binomial logistic regression problem where the classes are pass and fail. The student
dataset has entrance marks based on the historic data of those who are selected or not selected. Based on the
logistic regression, the values of learnt parameters are β0 =1 and β1 = 8. 𝐴𝑠𝑠𝑢𝑚𝑖𝑛𝑔 𝑚𝑎𝑟𝑘 𝑜𝑓 𝑥 = 60.
Compute the resultant class.
β0 =1 and β1 = 8, 𝑥 = 60.

1
ℎθ (x) =
1+𝑒
− β0 + β1𝑥

1
=
1+𝑒
− 1 + 8 ∗60 )

= 0.44

If we assume the threshold value as 0.5 then it is observed that 0.44< 0.5 , therefore the candidate with
marks 60 is not selected.

You might also like