Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
94 views

Linear Regression

The document provides an overview of linear regression. It explains that linear regression fits a linear model to the data by finding coefficient values that minimize the deviations between predicted and actual values. These coefficient values can be estimated using calculus by taking the partial derivatives of the error function and setting them equal to zero. The document also discusses evaluating linear regression models using various error metrics like mean absolute error, mean squared error, and root mean squared error.

Uploaded by

Parthraj Solanki
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
94 views

Linear Regression

The document provides an overview of linear regression. It explains that linear regression fits a linear model to the data by finding coefficient values that minimize the deviations between predicted and actual values. These coefficient values can be estimated using calculus by taking the partial derivatives of the error function and setting them equal to zero. The document also discusses evaluating linear regression models using various error metrics like mean absolute error, mean squared error, and root mean squared error.

Uploaded by

Parthraj Solanki
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Linear Regression

2
Linear Regression: Pictorially
▪ Linear regression is like fitting a line or (hyper)plane to a set of points 𝑧1
𝑧2 = 𝜙(𝑥)

What if a line/plane doesn’t


model the input-output
relationship very well, e.g., if Original (single) feature Two features
Nonlinear curve needed Can fit a plane (linear)
their relationship is better

(Output 𝑦)
modeled by a nonlinear (Output 𝑦)
curve or curved surface?

Do linear models No. We can even fit a


become useless curve using a linear
in such cases? model after suitably
transforming the inputs
Input 𝑥 (single feature) (Feature 2) (Feature 1)
𝑦 ≈ 𝒘⊤ 𝜙(𝒙)
The transformation 𝜙 . can be predefined or learned
(e.g., using kernel methods or a deep neural network
based feature extractor). More on this later

▪ The line/plane must also predict outputs the unseen (test) inputs well
Simplest Possible Linear Regression Model
 This is the base model for all statistical
machine learning
 x is a one feature data variable
 y is the value we are trying to predict
 The regression model is
y = w0 + w1 x + 
Two parameters to estimate – the slope of
the line w1 and the y-intercept w0
 ε is the unexplained, random, or error
component.
Solving the regression problem
 We basically want to find {w0, w1} that
minimize deviations from the predictor
line

 How do we do it?
 Iterate over all possible w values
along the two dimensions?
 Same, but smarter? [next class]
 No, we can do this in closed form with
just plain calculus
 Very few optimization problems in ML
have closed form solutions
 The ones that do are interesting for
that reason
Parameter estimation via calculus
 We just need to set the partial derivatives
to zero (full derivation)

 Simplifying

(𝑥𝑦) − (𝑥)(
ҧ 𝑦)
ത
=
(𝑥𝑖2 ) − (𝑥)ҧ 2
Problem
Xi(week) Yi (Sales in Thousands)
1 1.2
2 1.8
3 2.6
4 3.2
5 3.8
Evaluation
x y x2 x*y
1 1.2 1 1.2
2 1.8 4 3.6
3 2.6 9 7.8
4 3.2 16 12.8
5 3.8 25 19
Average=15/5=3 Average=12.6/5=2.52 Average=55/5=11 Average=44.4/5=8.88
Slope=(8.88-3(2.52))/(11- Intercept=2.52-0.66*3= Line: y=0.54-0.66*x
32)=0.66 0.54
Validation
• Standard Error: difference between actual and predicted value
1 𝑛−1
• Mean Absolute Error(MAE) : σ𝑖=0 𝑦𝑖 − 𝑦ෝ𝑖
𝑛
1 𝑛−1
• Mean squared Error (MSE): σ𝑖=0 𝑦𝑖 − 𝑦ෝ𝑖 2
𝑛
• Root Mean Square Error(RMSE): Root of MSE
• Relative MSE: ratio of the prediction ability of y cap to the average of
the trivial population
σ𝑛−1 𝑦𝑖 2
𝑖=0 𝑦𝑖 −ෞ
• σ𝑛−1 2
𝑖=0 𝑦𝑖 −𝑦𝑖

• Coefficient of variation: RMSE/ 𝑦ത

You might also like