1

Logistic Regression Model Study

A Statistical Analysis Method

University Canada West

BUSI 650: BUSINESS ANALYTICS

CAMPUS-SUMMER22-04: Jul. 11, 2022 - Sep. 25, 2022

Professor Aliosman Pektas

Due: Monday, Sep 19, 2022, 11:59 pm (PDT)

Abstract:
2

This paper entails the theoretical and basic technical study of logistic regression, a statistical analysis
approach that uses historical observations from a data set to predict a binary result, such as yes or no, true, or
false, etc., (Vadapalli, 2020). In addition, it is about the study of examining the correlation between one or more
independent variables, where a logistic regression model forecasts a dependent data variable.

This paper follows a basic structure of 5 critical questions and answers which reflects the basic
understanding to logistic regression, its purpose, its application in business or other fields, and the technicalities.
The five questions that will be addressed in this paper are as follows:

Question 1: What is the definition and overview/purpose of Logistic Regression?

Question 2: What does logistic regression serves as a tool in business?

Question 3: What are the logistic regression's key assumptions?

Question 4: What are the distinctive features between logistic regression and linear regression?

Question 5: What are the limitations of logistic regression?

Addressing Questions

Question 1: What is the definition and overview/purpose of Logistic Regression?

To define logistic regression, it is an illustration in the form of supervised learning which is used to
determine or forecast the likelihood that a binary (yes/no) event will occur.
It examines the association between one or more independent variables and a qualitative dependent variable
(one that can only take certain discrete binary values, such as presence or absence, yes or no, true, or false. The
dependent variable has a probability from a range of 0 to 1. In logistic regression, the odds, which are the
success probability divided by the failure probability, are transformed using a logit formula. (Vadapalli, 2020).

For instance, of a common scenario, one use of logistic regression to identify whether a person is likely
to have the COVID-19 virus or not, where a binary categorization of this event can occur which, there are only
two answers to this question: either they are infected or they are not.

The following equation represents the logistic regression function,

Formula Chart 1: Logistic Regression Function Equation

Note. Logistic Regression for Machine Learning (CapitalOne, 2021)

Illustration 1: Logistic Regression

3

Note. Logistic Regression for Machine Learning (CapitalOne, 2021)

The use of logistic regression simplifies the mathematics involved in determining how various factors,
like as age, gender, and the location of advertisements, affect a result (e.g., click-through or ignore). The
generated models may be used to dissect the relative efficacy of various interventions for distinct population
subgroups, such as young vs elderly or male versus female.

Question 2: What does logistic regression serves as a tool in business?

In business fields, logistic regression can be utilized and served as an effective forecasting tool.
Businesses may make smart decisions and utilization of resources by using this paradigm. The logit tool can
comprehend associations, calculate probabilities, and forecast events. For instance, in marketing field, business
can use the model to predict whether a specific target market would purchase a new product or not, which is a
very effective game changer.

Businesses can get insights from the results of logistics regression, which they then employ to grow
their operations and meet organisational objectives. It can be used to minimise costs and maximize revenue
which can impact the firm’s rising rate of return or raise the degree of corporate productivity.

Another example would be for cosmetics firms, where they want to know how consumers will react to
their promotional offer of "Buy 2 Get Sample of New Skincare Range" on their beauty products. The regression
can generate the likelihood of outcome by determining whether clients would "Respond" or "Not Respond" to
the offer. The results will aid them in creating better marketing and advertising.

Question 3: What are the logistic regression's key assumptions?

According to Statology (2022), it states that the logistic regression model is predicated on the following
basis of six assumptions:

Assumption 1: The response variable in logistic regression only has two possible outcomes, according to
assumption one (the variable is binary).

Assumption 2: The dataset's observations are unrelated to one another, according to the logic regression model.
Or, to put it another way, the observations shouldn't be connected to one another or come from repeated
measurements of the same thing. The simplest method to evaluate this assumption is to plot the residuals versus
time and search for a pattern that suggests randomness.

Assumption 3: The independent variables do not exhibit multicollinearity. Severe multicollinearity between the
independent variables is believed to be absent in logistic regression. Multicollinearity, which results from
significant correlation between two or more explanatory variables, is the lack of separate or independent
information in the regression model. The most often used technique for detecting multicollinearity is the
variance inflation factor (VIF), which evaluates the degree of correlation between predictor variables in a
regression model.

Assumption 4: Logistic regression assumes that the dataset has lack of severe outliers or important observations.
A technique for scanning a dataset for extreme outliers and important observations is to compute Cook's
distance for each observation.

Assumption 5: Determining whether there is a linear relationship between the explanatory factors and the logit
of the response variable. Logistic regression presumes a linear relationship between each explanatory variable
and the response variable's logit. The most straightforward technique to check if this premise is true is to use a
Box-Tidwell test.
4

Assumption 6: The fitted logistic regression model requires a suitable sample size of the dataset in order to
produce reliable results. asserts that in order for logistic regression to offer sufficient numbers in both categories
(binary) of the response variable, large sample sizes are required. With more explanatory factors, a larger
sample size is required. (Bewick, 2005)

Question 4: What are the distinctive features between logistic regression and linear regression?

The distinctive features between Logistic and Linear Regression

Regression issues are handled using linear regression, whereas classification difficulties are handled using
logistic regression. While the output of a linear regression is continuous, that of a logistic regression is discrete.
In contrast to logistic regression, which goes one step farther and fits the line values to the sigmoid curve, linear
regression's goal is to determine the best-fit line. While maximum likelihood estimation is used for logistic
regression, mean squared error is used to calculate the loss function in linear regression. (Kanade, 2022)
Even though the relationship between a dependent variable and one or more independent variables is
estimated by both logistic regression and linear regression, linear regression predicts a continuous variable
whereas logistic regression predicts a categorical one. In logistic regression, categorical variables include those
that can only be true or false, yes or no, 1 or 0, and so on. In contrast to linear regression, the logit function turns
the S-curve into a straight line and generates a probability. (Mondal, 2020)

Question 5: What are the limitations of logistic regression?

 In logistic regression, if there are less data than features, it might result in overfitting.
 It constructs linear boundaries; thus it is limited to linear relationships.
 The assumption of linearity between the dependent variable and the independent variables is the main
drawback of logistic regression.
 It only works for forecasting discrete functions. As a result, the discrete number set is confined to the
dependent variable of the logistic regression.
 A moderate level of multicollinearity between the independent variables is required for logistic
regression. Complex associations are difficult to establish using logistic regression. The performance of
this method can be readily surpassed by more potent and condensed algorithms like neural networks.
(Geeksforgeeks, 2022)

References
Bewick, V., Cheek, L., & Ball, J. (2005). Statistics review 14: Logistic regression. Critical care, 9(1), 1-7.

Capitalone. (2021, November 8). Logistic Regression for Machine Learning.

https://www.capitalone.com/tech/machine-learning/what-is-logistic-regression/
5

Geekforgeeks. (2022, Aug 23). Advantages and Disadvantages of Logistic Regression.

https://www.geeksforgeeks.org/advantages-and-disadvantages-of-logistic-regression/#:~:text=The
%20major%20limitation%20of%20Logistic,association%20(positive%20or%20negative).

Kanade, V. (2022, June 10). Linear Regression vs. Logistic Regression: Understanding 13 Key Differences.
https://www.spiceworks.com/tech/artificial-intelligence/articles/linear-regression-vs-logistic-
regression/

Mondal, S. (2020, December 1). How Logistic Regression is related to Linear Regression.
https://www.analyticsvidhya.com/blog/2020/12/beginners-take-how-logistic-regression-is-related-to-
linear-regression/#:~:text=The%20Differences%20between%20Linear%20Regression,Logistic
%20regression%20provides%20discreet%20output.

Statology. (2022). The 6 Assumptions of Logistic Regression (With Examples).

https://www.statology.org/assumptions-of-logistic-regression/

Vadapalli, P. (2020, July 27). 6 Types of Regression Models in Machine Learning You Should Know About.
Upgrad. https://www.upgrad.com/blog/types-of-regression-models-in-machine-learning/