Logistic Regression

Algorithm

G A YA T H R I P R A S A D S
 Overview

Logistic regression is a supervised , classification

algorithm used to assign observations to a discrete
set of classes.
Examples of classification problems are : Email
spam or not spam, Online transactions Fraud or
not Fraud, Tumour Malignant or Benign
Logistic regression transforms its output using the
logistic sigmoid function to return a probability
value.
It’s an S-shaped curve that can take any real-
valued number and map it into a value between 0
and 1
 Types of logistic regression

Binary (eg. Tumour Malignant or Benign)

Multi-linear (eg. Cats, dogs or Sheep's)
Ordinal (eg. Grade classification)

It takes linearly combined input features and

parameters and make binary predictions with
the help of natural-logarithm (log base e)
In Logistic regression, instead of fitting a
regression line, we fit an "S" shaped logistic
function, which predicts two maximum values (0 or
1).
The model computes the probability value which
lies between the interval 0 and 1 (inclusive of
both).
This probability can be used to classify the data.
For example, if the computed probability comes
out to be greater than 0.5, then the data belonged
to class A and otherwise, for less than 0.5, the data
belonged to class B.
Graphical Comparison
 Logistic Regression Assumptions

In case of binary logistic regression, the target

variables must be binary always and the
desired outcome is represented by the factor
level 1.
There should not be any multi-collinearity in
the model, which means the independent
variables must be independent of each other.
Include meaningful variables in model
building.
Needs a large sample size for logistic
regression
 Linear Regression Logistic Regression
Outcome In linear regression, the In logistic regression,
outcome (dependent the outcome (dependent
variable) is continuous. variable) has only a
It can have any one of limited number of
an infinite number of possible values.
possible values.
The dependent Linear regression is Logistic regression is
variable used when your used when the response
response variable is variable is categorical in
continuous. For nature. For instance,
instance, weight, yes/no, true/false,
height, number of red/green/blue,
hours, etc. 1st/2nd/3rd/4th, etc.
Equation Linear regression gives Logistic regression
an equation which is of gives an equation which
the form Y = mX + C, is of the form Y = e^X
means equation with + e^(-X).
degree 1.
Error minimization Linear regression uses While logistic
technique ordinary least squares regression uses
method to minimise the maximum likelihood
errors and arrive at a method to arrive at the
best possible fit. solution.
 Confusion Matrix

A confusion matrix is a summary of prediction results

on a classification problem. The number of correct
and incorrect predictions are summarized with count
values The confusion matrix shows the ways in which
your classification model is confused when it makes
predictions.
“true positive” for correctly predicted event values.
“false positive” for incorrectly predicted event values.
“true negative” for correctly predicted no-event
values.
“false negative” for incorrectly predicted no-event
values.
 Performance Metrics

Sensitivity / Recall: Fraction of positives that were

correctly identified
A model can cheat and maximize this by always
returning “positive”.
 Sensitivity= true positives/ (true positive + false
negative)
Specificity : Measures the proportion of negatives
which are correctly identified .
A model can cheat and maximize this by always
returning “negative”.
Specificity= true negatives/ (true negative + false
positives)
Precision : Accuracy of predictions. It is
used as a measure to calculate the success of
predicted values to the values which were
supposed to be successful.
A test can cheat and maximize this by only
returning positive on one result it’s most
confident in.
Precision= true positives/ (true positive +
false positive)
 Low Precision/High Recall: In applications where we want to reduce
the number of false negatives without necessarily reducing the number
of false positives, we choose a decision value which has a low value of
Precision or high value of Recall. For example, in a cancer diagnosis
application, we do not want any affected patient to be classified as not
affected without giving much heed to if the patient is being wrongfully
diagnosed with cancer. This is because, the absence of cancer can be
detected by further medical diseases but the presence of the disease
cannot be detected in an already rejected candidate.
 High Precision/Low Recall: In applications where we want to reduce
the number of false positives without necessarily reducing the number
of false negatives, we choose a decision value which has a high value of
Precision or low value of Recall. For example, if we are classifying
customers whether they will react positively or negatively to a
personalised advertisement, we want to be absolutely sure that the
customer will react positively to the advertisement because otherwise, a
negative reaction can cause a loss of potential sales from the customer.
F1 Score: The f1-score tells the accuracy of
the classifier in classifying the data points in
that particular class compared to all other
classes. It is calculated by taking the
harmonic mean of precision and recall.
F1
score=2*recall*precision/(recall+precision)
Support: The support is the number of
samples of the class in the data set.
Accuracy=TP+TN/(TP+TN+FP+FN)
 Advantages

 Efficient Computation
 Due to its simple probabilistic interpretation, the training time of
logistic regression algorithm comes out to be far less than most
complex algorithms
 The predicted parameters (trained weights) give inference about
the importance of each feature. The direction of association i.e.
positive or negative is also given. So we can use logistic
regression to find out the relationship between the features.
 Compared to some other machine learning algorithms, Logistic
Regression will provide probability predictions and not only
classification labels.
 In a low dimensional dataset having a sufficient number of
training examples, logistic regression is less prone to over-fitting.
 Logistic Regression proves to be very efficient when the dataset
has features that are linearly separable.
 Disadvantages

 Non linear problems can't be solved with logistic

regression since it has a linear decision surface.
Linearly separable data is rarely found in real
world scenarios.
 Logistic regression is less prone to overfitting but
it can overfit in high dimensional datasets
 This algorithm is sensitive to outliers.
 Logistic Regression requires moderate or no
multicollinearity between independent variables.
 Logistic Regression requires a large dataset and
also sufficient training examples for all the
categories it needs to identify.
 End of the Session

Dataset used:
https://
drive.google.com/file/d/15pc24lVzokKXhPvjqj
vgmMNqSc611EoL/view?usp=sharing

Thanks for listening..