Frequently, you would like to estimate the best values of the coefficients in an equation from experimental data. The equation might be a theoretical law or just a polynomial, but the procedure is the same. Let y be the dependent variable in the equation, bi be the coefficients in the equation, and xi be the independent variables in the equation so that the model is of the form
y = f(b0, b1, . . . ; x0, x1, . . .) (N.1)
Let represent the error between the observation of y, Y, and the predicted value of y using the values of xi and the estimated values of the coefficients bi:
Y=y+ (N.2)
The classical way to get the best estimates of the coefficients is by least squares, that is, by minimizing the sum of the squares of the errors (of the deviations) between Y and y for all the j sets of experimental data, j = 1 to p: p p Minimize F = a (Pj)2 = a (Yj - fj)2 (N.3) j=1 j=1
Let us use a model linear in the coefficients with one independent variable x
y = b0 + b1x (N.4)
(in which x0 associated with b0 always equals 1 in order to have an intercept)
to illustrate the principal features of the least-squares method to estimate the model coefficients. The objective function is p p F = a (Yj - yj)2 = a (Yj - b0 - b1xj)2 (N.5) j=1 j=1
924 Appendix N Fitting Functions to Data 925
There are two unknown coefficients, b0 and b1, and p known pairs of experi- mental values of Yj and xj. We want to minimize F with respect to b0 and b1. Recall from calculus that you take the first partial derivatives of F and equate them to zero to get the necessary conditions for a minimum. p 0F = 0 = 2 a (Yj - b0 - b1xj)(-1) (N.6a) 0 b0 j=1 p 0F = 0 = 2 a (Yj - b0 - b1xj)(-xj) (N.6b) 0 b1 j=1
Rearrangement yields a set of linear equations in two unknowns, b0 and b1:
p p p a b0 + a b1xj = a Yj j=1 j=1 j=1 p p p 2 a b0xj + a b1xj = a xjYj j=1 j=1 j=1
The summation Σ j=1
p b0 is (p)(b0) and in the other summations the constants b0 and b1 can be removed from within the summation signs so that p p b0(p) + b1 a xj = a Yj (N.7a) j=1 j=1
p p p b0 a xj + b1 a xj2 = a xjYj (N.7b) j=1 j=1 j=1
The two linear equations above in two unknowns, b0 and b1, can be solved quite easily for b0 the intercept and b1 the slope.
Example N.1 Application of Least Squares
Fit the model y = β0 + β1 x to the following data (Y is the measured response and x the independent variable).
x Y
0 0 1 2 2 4 3 6 4 8 5 10
(Continues) AppN.qxd 3/10/12 9:26 PM Page 926
926 Appendix N Enthalpy-Concentration Data
Example N.1 Application of Least Squares (Continued)
Solution The computations needed to solve Equation (N.7) are
a xj = 15 a xjYj = 110
2 a Yj = 30 a xj = 55
Then
6b0 + 15b1 = 30 15b0 + 55b1 = 110
Solution of these two equations yields
b0 = 0 b1 = 2
and the model becomes Ŷ = 2x, where Ŷ is the predicted value for a given x.
The least-squares procedure outlined above can be extended to any
number of variables as long as the model is linear in the coefficients. For example, a polynomial y = a + bx + cx2 is linear in the coefficients (but not in x), and would be represented as
y = b0 + b1x1 + b2x2
where a = b0, b = b1, c = b2, x1 = x, and x2 = x2. Linear equations equivalent
to Equation (N.7) with several independent variables can be solved via a computer. If the equation you want to fit is nonlinear in the coefficients, such as
y = b0eb1x + b2x2
you can minimize F in Equation (N.3) directly by the computer program on
the disk in the back of this book, or by using a nonlinear least-squares com- puter code taken from a library of computer codes. Appendix N Fitting Functions to Data 927
Additional useful information can be extracted from a least-squares analy-
sis if four basic assumptions are made in addition to the presumed linearity of y in x: 1. The x’s are deterministic variables (not random). 2. The variance of j is constant or varies only with the x’s. 3. The observations Yj are mutually statistically independent. 4. The distribution of Yj about yj given xj is a normal distribution. For further details, see Box, Hunter, and Hunter1 or Box and Draper.2
1G. E. P. Box, W. G. Hunter, and J. S. Hunter. Statistics for Experimenters. New York: Wiley- Interscience (1978). 2G. E. P. Box and N. R. Draper. Empirical Model Building and Response Surfaces. New York:
