Least Squares Fitting of Data To A Curve

Least Squares Fitting
of Data to a Curve
Gerald Recktenwald
Portland State University
Department of Mechanical Engineering
gerry@me.pdx.edu
These slides are a supplement to the book Numerical Methods with

Matlab: Implementations and Applications, by Gerald W. Recktenwald,
c 2001, Prentice-Hall, Upper Saddle River, NJ. These slides are c
2001 Gerald W. Recktenwald. The PDF version of these slides may
be downloaded or stored or printed only for noncommercial, educational
use. The repackaging or sale of these slides in any form, without written
consent of the author, is prohibited.
The latest version of this PDF file, along with other supplemental material
for the book, can be found at www.prenhall.com/recktenwald.
Version 0.69 November 26, 2001

Overview
• Fitting a line to data

Geometric interpretation
Residuals of the overdetermined system
The normal equations
• Nonlinear fits via coordinate transformation
• Fitting arbitrary linear combinations of basis functions
Mathematical formulation
Solution via normal equations
Solution via QR factorization
• Polynomial curve fits with the built-in polyfit function
• Multivariate fitting
NMM: Least Squares Curve-Fitting page 1

Fitting a Line to Data
Given m pairs of data:
(xi, yi), i = 1, . . . , m
Find the coefficients α and β such that
F (x) = αx + β
is a good fit to the data
Questions:
• How do we define good fit?
• How do we compute α and β after a definition of “good fit”
is obtained?

Plausible Fits
We can obtain plausible fits by adjusting the slope (α) and

intercept (β ). Here is a graphical representation of some
candidates for best fit to a particular set of data:
4
y
3
1
1 2 3 4 5 6
x
Which of these lines provides the best fit?

The Residual
At each point the difference between the y data and the fit

function is
ri = yi − F (xi) = yi − (αxi + β)
ri is called the residual for the data pair (xi, yi).
ri is the vertical distance between the known data and the fit

function.
4
y
3
1
1 2 3 4 5 6
x

Minimizing the Residual
Two criteria for choosing the “best” fit

2
minimize |ri| or minimize ri
For statistical and computational reasons choose

2
minimization of ρ = ri

m
2
ρ= [yi − (αxi + β)]
i=1

Orthogonal Distance Fit
An alternative to minimizing the residual is to minimize the

orthogonal distance to the line.
d4
d3
d2
d1
x1 x2 x3 x4
2
Minimizing di is known as the Orthogonal Distance
Regression problem. This problem readily solved with Matlab,
but it is beyond the scope of our present discussion1 .
1 See, e.g., Åke Björk, Numerical Methods for Least Squares Problems, 1996, SIAM,
Philadelphia

Least Squares Fit (1)
The least squares fit is obtained by choosing the α and β so that

m
2
ri
i=1
is a minimum. Let ρ = r22 to simplify the notation.
Find α and β by minimizing ρ = ρ(α, β). The minimum

requires
∂ρ
=0
∂α β=constant
and
∂ρ
=0
∂β α=constant

Carrying out the differentiation leads to
Sxxα + Sxβ = Sxy (1)

Sxα + mβ = Sy (2)
where

m
m
Sxx = xixi Sx = xi
i=1 i=1
m m
Sxy = xiyi Sy = yi
i=1 i=1
Note: Sxx, Sx, Sxy , and Syy can be directly computed from
the given (xi, yi) data. Thus, Equation (1) and (2)
are two equations for the two unknowns, α and β .

Solving equations (1) and (2) for α and β yields
1
α = (SxSy − mSxy ) (3)
d
1
β = (SxSxy − SxxSy ) (4)
d
with
2
d = Sx − mSxx (5)

Overdetermined System for a Line Fit (1)
Now, let’s rederive the equations for the fit. This will give us
insight into the process or fitting arbitrary linear combinations of
functions.
For any two points we can write
αx1 + β = y1
αx2 + β = y2
or
x1 1 α y1
=
x2 1 β y2
But why just pick two points?

Overdetermined System for a Line Fit (2)
Writing out the αx + β = y equation for all of the known

points (xi, yi), i = 1, . . . , m gives the overdetermined system.
   
x1 1 y1
 x2 1 α  y2 
 . ...  β = 
 
 ..  ... 
xm 1 ym
or
Ac = y
where
   
x1 1 y1
 x2 1 α  y2 
A=
 ... ... 
 c= y= 
 ... 
β
xm 1 ym
Note: We cannot solve Ac = y with Gaussian elimination. Unless the

system is consistent (i.e., unless y lies in the column space of A)
it is impossible to find the c = (α, β)T that exactly satisfies all
m equations. The system is consistent only if all the data points
lie along a single line.

Normal Equations for a Line Fit
Compute ρ = r22, where r = y − Ac

2 T T
ρ = r2 = r r = (y − Ac) (y − Ac)
T T T T T
= y y − (Ac) y − y (Ac) + c A Ac
T T T T
= y y − 2y Ac + c A Ac.
Minimizing ρ requires
∂ρ T T
= −2A y + 2A Ac = 0
∂c
or
T T
(A A)c = A b
This is the matrix formulation of equations (1) and (2).

linefit.m
The linefit function fits a line to a set of data by solving the

normal equations.
function [c,R2] = linefit(x,y)

% linefit Least-squares fit of data to y = c(1)*x + c(2)
%
% Synopsis: c = linefit(x,y)
% [c,R2] = linefit(x,y)
%
% Input: x,y = vectors of independent and dependent variables
%
% Output: c = vector of slope, c(1), and intercept, c(2) of least sq. line fit
% R2 = (optional) coefficient of determination; 0 <= R2 <= 1
% R2 close to 1 indicates a strong relationship between y and x
if length(y)~= length(x), error(’x and y are not compatible’); end
x = x(:); y = y(:); % Make sure that x and y are column vectors

A = [x ones(size(x))]; % m-by-n matrix of overdetermined system
c = (A’*A)\(A’*y); % Solve normal equations
if nargout>1
r = y - A*c;
R2 = 1 - (norm(r)/norm(y-mean(y)))^2;
end

Line Fitting Example
Store data and perform the fit
>> x = [1 2 4 5];
>> y = [1 2 2 3];
>> c = linefit(x,y)
Evaluate and plot the fit with
>> xfit = linspace(min(x),max(x));

>> yfit = c(1)*xfit + c(2)
>> plot(x,y,’o’,xfit,yfit,’-’);
4
y data and fit function
0
0 1 2 3 4 5 6
x

R2 Statistic (1)
R2 is a measure of how well the fit function follows the trend in

the data.
Define:
ŷ is the value of the fit function at the
known data points
ŷi = c1xi + c2 for a line fit
ȳ is the average of the y values
1
ȳ = yi
m
Then:
2
2
(ŷi − ȳ)
r22
R = =1−
(yi − ȳ)
2 (yi − ȳ)2
0 ≤ R 2 ≤ 1.
When R2 ≈ 1 the fit function follows the trend of the data.

When R2 ≈ 0 the fit is not significantly better than
approximating the data by its mean.

R2 Statistic (2)
Graphical interpretation of
2 r22
R =1−
(yi − ȳ)2
Consider a line fit to a data set with R2 = 0.934

5
Vertical distances between
4
given y data and the least y
squares line fit. Vertical lines 3
show contributions to r2 . 2
1
1 2 3 4 5 6
x
5
Vertical distances between
4
given y data and the average y
of the y . Vertical lines show 3

contributions to (yi −ȳ)2 2
1
1 2 3 4 5 6
x

R2 Statistic: Example Calculation
Consider the variation of the bulk modulus of Silicon Carbide as

a function of temperature (Cf. Example 9.4)
T (◦ C) 20 500 1000 1200 1400 1500

G (GP a) 203 197 191 188 186 184
>> [t,D,labels] = loadColData(’SiC.dat’,6,5);

>> g = D(:,1);
>> [c,R2] = linefit(t,g);
c =
-0.0126
203.3319
R2 =
0.9985
205
GPa
200
Bulk Modulus
195
190
185
180
0 500 1000 1500
°
T C

Fitting Transformed Non-linear Functions (1)
• Some nonlinear fit functions y = F (x) can be transformed

to an equation of the form v = αu + β
• Linear least squares fit to a line is performed on the
transformed variables.
• Parameters of the nonlinear fit function are obtained by
transforming back to the original variables.
• The linear least squares fit to the transformed equations does
not yield the same fit coefficients as a direct solution to the
nonlinear least squares problem involving the original fit
function.
Examples:
y = c1ec2x −→ ln y = αx + β
y = c1xc2 −→ ln y = α ln x + β
y = c1xec2x −→ ln(y/x) = αx + β

Consider
c x
y = c1e 2 (6)
Taking the logarithm of both sides yields
ln y = ln c1 + c2x
Introducing the variables
v = ln y b = ln c1 a = c2
transforms equation (6) to
v = ax + b

The preceding steps are equivalent to graphically obtaining c1

and c2 by plotting the data on semi-log paper.
y = c1ec2x ln y = c2x + ln c1
1
5 10
4.5
3.5
0
10
2.5
y
2
-1
10
1.5
0.5
-2
0 10
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
x x

Consider
c
y = c1x 2 (7)
Taking the logarithm of both sides yields
ln y = ln c1 + c2 ln x
Introduce the transformed variables
v = ln y u = ln x b = ln c1 a = c2
and equation (7) can be written
v = au + b

The preceding steps are equivalent to graphically obtaining c1

and c2 by plotting the data on log-log paper.
y = c1xc2 ln y = c2 ln x + ln c1
1
5 10
4.5
3.5
0
2.5 y 10
y
1.5
0.5
-1
0 10
-2 -1 0 1
0 1 2 3 4 5 6 10 10 10 10
x x

Example: Fitting Data to y = c1xec2x
Consider
c2 x
y = c1xe (8)
The transformation

y
v = ln a = c2 b = ln c1
x
results in the linear equation
v = ax + b
y = c1xec2x ln(y/x) = c2x + ln c1

0.5 1
0.45 0
0.4 -1
0.35 -2
0.3 -3
ln(y/x)
0.25 -4
y
0.2 -5
0.15 -6
0.1 -7
0.05 -8
0 -9
0 1 2 3 4 5 6 0 1 2 3 4 5 6
x x

xexpfit.m
function c = xexpfit(x,y)
% xexpfit Least squares fit of data to y = c(1)*x*exp(c(2)*x)
%
% Synopsis: c = xexpfit(x,y)
%
% Input: x,y = vectors of independent and dependent variable values
%
% Output: c = vector of coefficients of y = c(1)*x*exp(c(2)*x)
z = log(y./x); % Natural log of element-by-element division

c = linefit(x,z); % Fit is performed by linefit
c = [exp(c(2)); c(1); ]; % Extract parameters from transformation

Example: Fit Synthetic Data
Generate synthetic data with y = 5xe−3x. See demoXexp
>> x0 = 0.01; % Starting point ~= 0 avoids log(0)

>> noise = 0.05; % Magnitude of noise
>> x = linspace(x0,2,n);
>> y = 5*x.*exp(-3*x); % Original data
>> yn = y + noise*(rand(size(x))-0.5); % Add noise
>> yn = abs(yn); % Need positive y for log(y)
>> c = xexpfit(x,yn);
0.7
original
noisy
0.6 fit
0.5
c1 = 5.770 c2 = -3.233
0.4
200 points in synthetic data set
y
0.3
0.2
0.1
0
0 0.5 1 1.5 2
x

Summary of Transformations
• Transform (x, y) data as needed

• Use linefit
• Transform results of linefit back
>> x = ... % original data

>> y = ...
>> u = ... % transform the data
>> v = ...
>> a = linefit(u,v)
>> c = ... % transform the coefficients

Summary of Line Fitting
1. m data pairs are given: (xi, yi), i = 1, . . . , m.

2. The fit function y = F (x) = c1x + c2 has n = 2 basis
functions f1(x) = x and f2(x) = 1.
3. Evaluating the fit function for each of the m data points
gives an overdetermined system of equations Ac = y where
c = [c1, c2]T , y = [y1, y2, . . . , ym]T , and
   
f1(x1) f2(x1) x1 1
 f1(x2) f2(x2)   x2 1
A=
 ... 
...  =  .
 .. ... 
.
f1(xm) f2(xm) xm 1
4. The least-squares principle defines the best fit as the values

of c1 and c2 that minimize
2 2
ρ(c1, c2) = y − F (x)2 = y − Ac2.
5. Minimizing of ρ(c1, c2) leads to the normal equations

T T
(A A)c = A y,
6. Solving the normal equations gives the slope c1 and intercept

c2 of the best fit line.

Fitting Linear Combinations of Functions
• Definition of fit function and basis functions

• Formulation of the overdetermined system
• Solution via normal equations: fitnorm
• Solution via QR factorization: fitqr and \

Fitting Linear Combinations of Functions (1)
Consider the fitting function
F (x) = cf1(x) + c2f2(x) + . . . + cnfk (x)
or

n
F (x) = cj fj (x)
j=1
The basis functions
f1(x), f2(x), . . . , fn(x)
are chosen by you — the person making the fit.
The coefficients
c1, c2, . . . , cn
are determined by the least squares method

F (x) function can be any combination of functions that are

linear in the cj . Thus
2 2/3 x 4x
1, x, x , x , sin x, e , xe , cos(ln 25x)
are all valid basis functions. On the other hand,

c3 x c2
sin(c1x), e , x
are not valid basis functions as long as the cj are the parameters
of the fit.
The fit function for a cubic polynomial is

3 2
F (x) = c1x + c2x + c3x + c4,
which has the basis functions

3 2
x , x , x, 1.

The objective is to find the cj such that F (xi) ≈ yi.
Since F (xi)
= yi, the residual for each data point is

n
ri = yi − F (xi) = yi − cj fj (xi)
j=1
The least-squares solution gives the cj that minimize r2.

Consider the fit function with three basis functions
y = F (x) = c1f1(x) + c2f2(x) + c3f3(x).
Assume that F (x) acts like an interpolant. Then
c1f1(x1) + c2f2(x1) + c3f3(x1) = y1,

c1f1(x2) + c2f2(x2) + c3f3(x2) = y2,
...
c1f1(xm) + c2f2(xm) + c3f3(xm) = ym.
are all satisfied.

The preceding equations are equivalent to the overdetermined

system
Ac = y,
where  
f1(x1) f2(x1) f3(x1)
 f1(x2) f2(x2) f3(x2) 
A=
 ... ... ... 
,
f1(xm) f2(xm) f3(xm)
 
  y1
c1  y2 
c =  c2 , y= 
 ...  .
c3
ym

If F (x) cannot interpolate the data, then the preceding matrix

equation cannot be solved exactly: b does not lie in the column
space of A.
The least-squares method provides the compromise solution that

minimizes r2 = y − Ac2.
The c that minimizes r2 satisfies the normal equations

T T
(A A)c = A y.

In general, for n basis functions

 
f1(x1) f2(x1) ... fn(x1)
 f1(x2) f2(x2) ... fn(x2) 
A=
 ... ... ... ,

f1(xm) f2(xm) ... fn(xm)
  
c1 y1
 c2   y2 
c= 
 ...  , y= 
 ...  .
cn ym

Example: Fit a Parabola to Five Points
Consider fitting a curve to the following data.
x 1 2 3 4 5 6
y 10 5.49 0.89 −0.14 −1.07 0.84
12
10
6
y
−2
0 1 2 3 4 5 6 7
x
Not knowing anything more about the data we can start by

fitting a polynomial to the data.

Example: Fit a Parabola to Six Points (1)
The equation of a second order polynomial can be written

2
y = c1x + c2x + c3
where the ci are the coefficients to be determined by the fit and

the basis functions are
2
f1(x) = x , f2(x) = x, f3(x) = 1
The A matrix is
 
x21 x1 1
 x2 x2 1
A=
 ..
.
2
... ... 

x2m xm 1
where, for this data set, m = 6.

Define the data
>> x = (1:6)’;
>> y = [10 5.49 0.89 -0.14 -1.07 0.84]’;
Notice the transposes, x and y must be column vectors.
The coefficient matrix of the overdetermined system is
>> A = [ x.^2 x ones(size(x)) ];
The matrix and right-hand-side vector for the normal equations

are
>> disp(A’*A)
2275 441 91
441 91 21
91 21 6
>> disp(A’*y)
A’*y =
41.2200
22.7800
16.0100

Solve the normal equations
>> c = (A’*A)\(A’*y)
c =
0.8354
-7.7478
17.1160
Evaluate and plot the fit

>> yfit = c(1)*xfit.^2 + c(2)*xfit + c(3);
>> plot(x,y,’o’,xfit,yfit,’--’);
F(x) = c1 x2 + c2 x + c3
12
10
6
y
−2
0 1 2 3 4 5 6 7
x

Example: Alternate Fit to Same Six Points (1)
Fit the same points to

c1
F (x) = + c2x
x
The basis functions are
1
, x
x
In Matlab:
>> x = (1:6)’;
>> y = [10 5.49 0.89 -0.14 -1.07 0.84]’;
>> A = [1./x x];
>> c = (A’*A)\(A’*y)
F(x) = c1/x + c2 x
12
10
6
y
−2
0 1 2 3 4 5 6 7
x

Evaluating the Fit Function
as a Matrix–Vector Product (1)
We have been writing the fit function as
y = F (x) = c1f1(x) + c2f2(x) + · · · + cnfn(x)
The overdetermined coefficient matrix contains the basis

functions evaluated at the known data
 
f1(x1) f2(x1) . . . fn(x1)
 f1(x2) f2(x2) . . . fn(x2) 
A=  ... ... ... 

f1(xm) f2(xm) . . . fn(xm)
Thus, if A is available
F (x) = Ac
evaluates F (x) at all values of x, i.e., F (x) is a vector-valued

function.

Evaluating the fit function as a matrix–vector product can be

performed for any x. Suppose then that we have created an
m-file function that evaluates A for any x, for example
function A = xinvxfun(x)
A = [ 1./x(:) x(:) ];
We evaluate the fit coefficients with
>> x = ..., y = ...

>> c = fitnorm(x,y,’xinvxfun’);
Then, to plot the fit function after the coefficients of the fit

>> Afit = xinvxfun(xfit);
>> yfit = Afit*c;
>> plot(x,y,’o’,xfit,yfit,’--’)

Advantages:
• The basis functions are defined in only one place: in the

routine for evaluating the overdetermined matrix.
• Automation of fitting and plotting is easier because all that is
needed is one routine for evaluating the basis functions.
• End-user of the fit (not the person performing the fit) can
still evaluate the fit function as
y = c1f1(x) + c2f2(x) + · · · + cnfn(x).
Disadvantages:
• Storage of matrix A for large x vectors consumes memory.

This should not be a problem for small n.
• Evaluation of the fit function may not be obvious to a reader
unfamiliar with linear algebra.

Matlab Implementation in fitnorm
Let A be the m × n matrix defined by


... ... ... 
A =  f1(x) f2(x) ... fn(x)
... ... ...
The columns of A are the basis functions evaluated at each of

the x data points.
As before, the normal equations are

T T
A Ac = A y
The user supplies a (usually small) m-file that returns A.

fitnorm.m
function [c,R2,rout] = fitnorm(x,y,basefun)

% fitnorm Least-squares fit via solution to the normal equations
% Given ordered pairs of data, (x_i,y_i), i=1,...,m, fitnorm
% returns the vector of coefficients, c_1,...,c_n, such that
% F(x) = c_1*f_1(x) + c_2*f_2(x) + ... + c_n*f_n(x)
% minimizes the L2 norm of y_i - F(x_i).
%
% Synopsis: c = fitnorm(x,y,basefun)
% [c,R2] = fitnorm(x,y,basefun)
% [c,R2,r] = fitnorm(x,y,basefun)
%
% Input: x,y = vectors of data to be fit
% basefun = (string) name of user-supplied m-file that computes
% matrix A. The columns of A are the values of the
% basis functions evaluated at the x data points.
%
% Output: c = vector of coefficients obtained from the fit
% R2 = (optional) adjusted coefficient of determination; 0 <= R2 <= 1
% R2 close to 1 indicates a strong relationship between y and x
% r = (optional) residuals of the fit
if length(y)~= length(x); error(’x and y are not compatible’); end
A = feval(basefun,x(:)); % Coefficient matrix of overdetermined system

c = (A’*A)\(A’*y(:)); % Solve normal equations, y(:) is always a column
if nargout>1
r = y - A*c; % Residuals at data points used to obtain the fit
[m,n] = size(A);
R2 = 1 - (m-1)/(m-n-1)*(norm(r)/norm(y-mean(y)))^2;
if nargout>2, rout = r; end
end

Example of User-Supplied m-files
The basis functions for fitting a parabola are

2
f1(x) = x , f2(x) = x, f3(x) = 1
Create the m-file poly2Basis.m:
function A = poly2Basis(x)
A = [ x(:).^2 x(:) ones(size(x(:)) ];
then at the command prompt
>> x = ...; y = ...;

>> c = fitnorm(x,y,’poly2Basis’)
or use an in-line function object:
>> x = ...; y = ...;

>> Afun = inline(’[ x(:).^2 x(:) ones(size(x(:)) ]’);
>> c = fitnorm(x,y,Afun);

Example of User-Supplied m-files
To the basis functions for fitting F (x) = c1/x + c2x are
1
, x
x
Create the m-file xinvxfun.m
function A = xinvxfun(x)
A = [ 1./x(:) x(:) ];
then at the command prompt
>> x = ...; y = ...;

>> c = fitnorm(x,y,’xinvxfun’)
or use an in-line function object:
>> x = ...; y = ...;

>> Afun = inline(’[ 1./x(:) x(:) ]’);
>> c = fitnorm(x,y,Afun);

R2 Statistic
R2 can be applied to linear combinations of basis functions.
Recall that for a line fit (Cf. § 9.1.4.)

2
2
(ŷ i − ȳ) r22
R = =1−
(yi − ȳ)
2 (yi − ȳ)2
where ŷi is the value of the fit function evaluated at xi, and ȳ is

the average of the (known) y values.
For a linear combination of basis functions

n
ŷi = cj fj (xi)
j=1
To account for the reduction in degrees of freedom in the data

when the fit is performed, it is technically appropriate to consider
the adjusted coefficient of determination

2 m − 1 (yi − ŷ)2
Radjusted = 1 − ,
m−n−1 (yi − ȳ)2
2
fitnorm provides the option of computing Radjusted

Polynomial Curve Fits with polyfit (1)
Built-in commands for polynomial curve fits:
polyfit Obtain coefficients of a least squares curve fit

of a polynomial to a given data set
polyval Evaluate a polynomial at a given set of x
values.
Syntax:
c = polyfit(x,y,n)
[c,S] = polyfit(x,y,n)
x and y define the data

n is the desired degree of the polynomial.
c is a vector of polynomial coefficients stored in order of

descending powers of x
n n−1
p(x) = c1x + c2x + · · · + cnx + cn+1
S is an optional return argument for polyfit. S is used as input

to polyval

Polynomial Curve Fits with polyfit (2)
Evaluate the polynomial with polyval
Syntax:
yf = polyval(c,xf)
[yf,dy] = polyval(c,xf,S)
c contains the coefficients of the polynomial (returned by

polyfit)
xf is a scalar or vector of x values at which the polynomial is to

be evaluated
yf is a scalar or vector of values of the polynomials: yf= p(xf).
If S is given as an optional input to polyval, then dy is a vector

of estimates of the uncertainty in yf

Example: Polynomial Curve Fit (1)
Fit a polynomial to Consider fitting a curve to the following data.
x 1 2 3 4 5 6
y 10 5.49 0.89 −0.14 −1.07 0.84
In Matlab:
>> x = (1:6)’;
>> y = [10 5.49 0.89 -0.14 -1.07 0.84]’;
>> c = polyfit(x,y,3);
>> yfit = polyval(c,xfit);
>> plot(x,y,’o’,xfit,yfit,’--’)
12
10
−2
1 2 3 4 5 6

Example: Conductivity of Copper Near 0 K
35
30
25
Conductivity (W/m/C)
20
15
10
0
0 10 20 30 40 50 60
Temperature (K)

Example: Conductivity of Copper Near 0 K (2)
Theoretical model of conductivity is
1
k(T ) = c1 2
+ c2T
T
To fit using linear least squares we need to write this as
1 c1 2
γ(T ) = = + c2T
k(T ) T
which has the basis functions

1 2
, T
T
The m-file implementing these basis functions is
function y = cuconBasis1(x)
% cuconBasis1 Basis fcns for conductivity model: 1/k = c1/T + c2*T^2
y = [1./x x.^2];
An m-file that uses fitnorm to fit the conductivity data with the

cuconBasis1 function is listed on the next page.

Least Squares Fitting of Data To A Curve

Uploaded by

Copyright:

Available Formats

Least Squares Fitting of Data To A Curve

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Least Squares Fitting of Data To A Curve

Uploaded by

Copyright:

Available Formats

Least Squares Fitting

These slides are a supplement to the book Numerical Methods with

Version 0.69 November 26, 2001

• Fitting a line to data

NMM: Least Squares Curve-Fitting page 1

Given m pairs of data:

Find the coeﬃcients α and β such that

is a good ﬁt to the data

NMM: Least Squares Curve-Fitting page 2

We can obtain plausible ﬁts by adjusting the slope (α) and

Which of these lines provides the best ﬁt?

NMM: Least Squares Curve-Fitting page 3

At each point the diﬀerence between the y data and the ﬁt

ri is called the residual for the data pair (xi, yi).

ri is the vertical distance between the known data and the ﬁt

NMM: Least Squares Curve-Fitting page 4

Two criteria for choosing the “best” ﬁt

For statistical and computational reasons choose

NMM: Least Squares Curve-Fitting page 5

An alternative to minimizing the residual is to minimize the

NMM: Least Squares Curve-Fitting page 6

The least squares ﬁt is obtained by choosing the α and β so that

is a minimum. Let ρ = r22 to simplify the notation.

Find α and β by minimizing ρ = ρ(α, β). The minimum

NMM: Least Squares Curve-Fitting page 7

Carrying out the diﬀerentiation leads to

Sxxα + Sxβ = Sxy (1)

NMM: Least Squares Curve-Fitting page 8

Solving equations (1) and (2) for α and β yields

NMM: Least Squares Curve-Fitting page 9

For any two points we can write

But why just pick two points?

NMM: Least Squares Curve-Fitting page 10

Writing out the αx + β = y equation for all of the known

Note: We cannot solve Ac = y with Gaussian elimination. Unless the

NMM: Least Squares Curve-Fitting page 11

Compute ρ = r22, where r = y − Ac

This is the matrix formulation of equations (1) and (2).

NMM: Least Squares Curve-Fitting page 12

The lineﬁt function ﬁts a line to a set of data by solving the

function [c,R2] = linefit(x,y)

x = x(:); y = y(:); % Make sure that x and y are column vectors

NMM: Least Squares Curve-Fitting page 13

Store data and perform the ﬁt

Evaluate and plot the ﬁt with

>> xfit = linspace(min(x),max(x));

NMM: Least Squares Curve-Fitting page 14

R2 is a measure of how well the ﬁt function follows the trend in

When R2 ≈ 1 the ﬁt function follows the trend of the data.

NMM: Least Squares Curve-Fitting page 15

Consider a line ﬁt to a data set with R2 = 0.934

show contributions to r2 . 2

NMM: Least Squares Curve-Fitting page 16

Consider the variation of the bulk modulus of Silicon Carbide as

T (◦ C) 20 500 1000 1200 1400 1500

>> [t,D,labels] = loadColData(’SiC.dat’,6,5);

NMM: Least Squares Curve-Fitting page 17

• Some nonlinear ﬁt functions y = F (x) can be transformed

is a minimum. Let ρ = r22 to simplify the notation.

Compute ρ = r22, where r = y − Ac

show contributions to r2 . 2

The c that minimizes r2 satisﬁes the normal equations