Chapter 7
Chapter 7
Approximation Theory
The primary aim of a general approximation is to represent non-arithmetic
quantities by arithmetic quantities so that the accuracy can be ascertained to
a desired degree. Secondly, we are also concerned with the amount of computation required to achieve this accuracy. These general notions are applicable
to functions f (x) as well as to functionals F (f ) (A functional is a mapping
from the set of functions to the set of real or complex numbers). Typical examples of quantities to be approximated are transcendental functions, integrals
and derivatives of functions, and solutions of dierential or algebraic equations.
Depending upon the nature to be approximated, dierent techniques are used
for dierent problems.
A complicated function f (x) usually is approximated by an easier function
of the form (x; a0 , . . . , an ) where a0 , . . . , an are parameters to be determined
so as to characterize the best approximation of f . Depending on the sense in
which the approximation is realized, there are three types of approaches:
(7.1)
Sometimes, we even further require that, for each i, the rst ri derivatives
of agree with those of f at xi .
1
(7.2)
(7.3)
(7.4)
Theorem 7.0.1 Let f (x) be a piecewise continuous function over the interval
[a, b]. Then for any > 0, there exist an integer n and numbers a0 , . . . , an such
n
b
ai xi }2 dx < .
that a {f (x)
i=0
Theorem 7.0.2 (Weierstrass Approximation Theorem) Let f (x) be a continuous function on [a, b]. For any > 0, there exist an integer n and a polynomial
pn (x) of degree n such that max |f (x) pn (x)| < . In fact, if [a, b] = [0, 1],
x[a,b]
n
n
k=0
k
xk (1 x)nk f ( )
n
(7.5)
converges to f (x) as n .
a0 0 (x) + . . . + an n (x)
b0 0 (x) + . . . + bm m (x)
(7.6)
7.1
where
i (x) :=
n
x xj
.
x
i xj
j=1
(7.8)
j=1
n
(x xj )
j=1
n!
where min{x1 , . . . , xn , x} < < max{x1 , . . . , xn , x}.
n1
(7.9)
k=0
n1
k=0
1, x1 , . . . ,
1,
f (n) ()
xn ,
xn1
1
. . . , xn1
n
a0
an1
f (x1 )
f (xn )
(7.10)
(xi
i>j
xj ). Since all xi s are distinct, we can uniquely solve (7.10) for the unknowns
(x xi )
p(x0 )) i=1
n
(x0 xi )
i=1
n!
n
= 0.
(x0 xi )
i=1
n
(x0 xi )
i=1
n!
Denition 7.1.1 The polynomial p(x) dened by (7.7) is called the Lagrange
interpolation polynomial.
(7.11)
1
1 + 25x2
(7.12)
in the interval [1, 1] at equally spaced points. Runge (1901) discovered that as
the degree n of the interpolating polynomial pn (x) tends toward innity, pn (x)
diverges in the intervals .726 . . . |x| < 1 while pn (x) works pretty well in the
central portion of the interval.
7.2
(7.13)
for j = 0, . . . , k.
(7.14)
holds.
(pf): Denote the right-hand side of (7.14) by R(x). Observe that R(x) is a
polynomial of degree k. By denition, it is easy to see that R(xij ) = f (xij )
for all j = 0, . . . , k. That is, R(x) interpolates the same set of data as does the
polynomial Pi0 i1 ...ik (x). By Theorem 7.1.1 the assertion is proved.
The dierence Pi0 i1 ...ik (x) Pi0 i1 ...ik1 (x) is a k-th degree polynomial which
vanishes at xij for j = 0, . . . , k 1. Thus we may write
Pi0 i1 ...ik (x) = Pi0 i1 ...ik1 (x) + fi0 ...ik (x xi0 )(x xi1 ) . . . (x xik1 ). (7.15)
The leading coecients fi0 ...ik can be determined recursively from the formula
(7.14), i.e.,
fi0 ...ik =
(7.16)
where fi1 ...ik and fi0 ...ik1 are the leading coecients of the polynomials Pi1 ...ik (x)
and Pi0 ...ik1 (x), respectively.
Remark. Note that the formula (7.16) starts from fi0 = f (xi0 ).
Remark. The polynomial Pi0 ...ik (x) is uniquely determined by the set of support data {(xij , fij )}. The polynomial is invariant to any permutation of the
indices i0 , . . . , ik . Therefore, the divided dierences (7.16) are invariant to permutation of the indices.
(7.17)
(7.18)
(7.19)
It follows that the k-th degree polynomial that interpolates the set of support
data {(xi , fi )|i = 0, . . . , k} is given by
Px0 ...xk (x)
7.3
Osculatory Interpolation
(0)
(r )
Given {xi }, i = 1, . . . k and values ai , . . . , ai i where ri are nonnegative integers. We want to construct a polynomial P (x) such that
(j)
P (j) (xi ) = ai
(7.21)
k
(ri + 1) 1.
i=1
(j)
(0)
(1)
(r )
= ci + ci (x xi ) + . . . + ci i (x xi )ri
= q1 (x) + (x x1 )r1 +1 q2 (x) + . . .
+ (x x1 )r1 +1 (x x2 )r2 +1 . . . (x xk1 )rk1 +1 qk (x).
k
(j)
(ri + 1) 1. Now P (j) (x1 ) = a1 for j = 0, . . . , r1
i=1
(0)
(j)
c1 , . . . , a1 =
implies
=
Now we rewrite (7.23) as
R(x) :=
(7.22)
(7.23)
(j)
(j)
(j)
a1
j!
P (x) q1 (x)
= q2 (x) + (x x2 )r2 +1 q3 (x) + . . .
(x x1 )r1 +1
Note that R(j) (x2 ) are known for j = 0, . . . , r2 since P (j) (x2 ) are known. Thus
(j)
all c2 , hence q2 (x), may be determined. This procedure can be continued to
determine all qi (x). Suppose Q(x) = P1 (x) P2 (x) where P1 (x) and P2 (x) are
k
two polynomials of the theorem. Then Q(x) is of degree
(ri + 1) 1,
i=1
(7.25)
(7.26)
(7.27)
Dene
hi (x)
gi (x)
=
=
k
(7.29)
i=1
7.4
Spline Interpolation
Thus far for a given function f of an interval [a, b], the interpolation has been
to construct a polynomial over the entire interval [a, b]. There are at least two
disadvantages for the global approximation:
1. For better accuracy, we need to supply more support data. But then
the degree the resultant polynomial gets higher and such a polynomial is
dicult to work with.
2. Suppose f is not smooth enough. Then the error estimate of an high
degree polynomial is dicult to establish. In fact, it is not clear whether
or not that the accuracy will increase with increasing number of support
data.
Denition 7.4.1 Let the interval [a, b] be partitioned into a = x1 < x2 < . . . <
xn = b. A function p(x) is said to be a cubic plite of f on the partition if
10
Denote Mi := p (xi ), i = 1, . . . , n. Since p(x) is piecewise cubic and continuously dierentiable, p
(x) is piecewise linear and continuous on [a, b]. In
particular, over the interval [xi , xi+1 ], we have
p (x) = Mi
x xi+1
x xi
+ Mi+1
.
xi xi+1
xi+1 xi
(7.30)
(7.31)
di
f (xi ) i Mi
i
6
f (xi+1 ) i Mi+1
.
i
6
(7.32)
(7.33)
(7.34)
xxi
(7.35)
xxi
Thus
2
3i1
Mi
6i1
3i2 Mi
=
6i
+
+
i1
6
f (xi+1 ) f (xi ) i (Mi+1 Mi )
,
i
6
(7.36)
or equivalently,
i1
i + i1
i
Mi1 +
Mi + Mi+1
6
3
6
f (xi+1 ) f (xi ) f (xi ) f (xi1 )
.
i
i1
(7.37)
11
(7.38)
(7.39)
Then we have
1
1
M1 + M2
3
6
n1
n1
Mn1 +
Mn
6
3
f (x2 ) f (x1 )
f (x1 )
1
f (xn ) f (xn1 )
= f (xn )
.
n1
=
..
..
..
.
.
.
n2 +n1
n1
n2
3
n1
6
f (x1 )
vdots
f (xi+1 )f (xi )
(xi1 )
f (xi )f
i
i1
..
.
f (xn )f (xn1 )
f (xn )
n1
f (x2 )f (x1 )
1
6
n1
3
(7.40)
(7.41)
M1
M2
..
.
Mn
7.5
Trigonometric Interpolation
(7.42)
M1
a0
aM
+
(ah cos hx + bh sin hx) +
cos M x,
2
2
(7.43)
(x)
h=1
(x)
h=1
12
2k
, k = 0, . . . , N 1.
N
Observe that
ei(hxk ) = ei(2hk/N ) = ei2(N h)k/N = ei(N h)xk .
Thus, we may write
cos hxk
sin hxk
(7.44)
(xk ) =
h=1
h=1
= 0 + 1 eixk + . . . + 2M ei2Mxk ,
(xk ) =
a0
+
2
M1
h=1
(7.45)
(7.46)
(7.47)
(7.48)
(7.49)
13
N 1
N 1
1
1
fk (k )j =
fk (eixk )j .
N
N
k=0
(7.50)
k=0
2k
N
1
k=0
N
1
k=0
summarize that
kjh =
k
jh
=
k=0
N
1
N
1
kj kh = 0. We may therefore
k=0
kj kh =
k=0
h
T
(1, 1h , . . . , N
1 )
Introducing (h) :=
of the complex inner product
(j) , (h) =
0,
N,
if
if
j=
0
j = h.
(7.51)
0,
if
if
N,
j = h
.
j=h
(7.52)
1 1
1
0
1 1 . . . 1N 1
..
.
N 1
N 1
1
N 1
N 1
be written as
f0
fN 1
or simply
0 (0) + 1 (1) + . . . + N 1 (N 1) = f.
By the orthogonality of (h) , (7.50) follows.
(7.53)
N 1
2
fk cos hxk
N
k=0
(7.54)
14
bh
N 1
2
fk sin hxk .
N
(7.55)
k=0
Denition 7.5.1 Given the phase polynomial (7.47) and 0 s N , the ssegment ps (x) is dened to be
ps (x) := 0 + 1 eix + . . . + s eisx .
(7.56)
Theorem 7.5.2 Let p(x) be the phase polynomial that interpolates a given set
of support data (xk , fk ), k = 0, . . . , N 1 with xk = 2k/N . Then the s-segment
ps (x) of p(x) minimizes the sum
S(q) =
N
1
(7.57)
k=0
s
s
j (j) and
j=0
j=0
1
(j)
.
N f, w
1
f ps , w(j) = j j = 0,
N
and hence
f ps , ps q =
s
f ps , (j j ) (j) = 0.
j=1
7.6
15
Suppose that we have a series of sines and cosines which represent a given
function of [L, L], say,
a0
nx
nx
+
+ bn sin
.
f (x) =
an cos
2
L
L
n=1
(7.58)
nx
dx
L
nx
mx
cos
cos
dx
L
L
L
cos
L
sin
L
sin
L
and
0,
nx
dx = 0
L
if n = m
(7.59)
(7.60)
L, if n = m
nx
mx
cos
dx = 0,
L
L
(7.61)
bn
1
L
1
L
f (x) cos
nx
dx, n = 0, 1, . . .
L
(7.62)
f (x) sin
nx
dx, n = 1, 2, . . .
L
(7.63)
L
L
L
Denition 7.6.1 The series (7.58), with coecients dened by (7.62) and
(7.63), is called the fourier series of f (x) on [L, L].
Given a function f (x), its Fourier series does not necessarily converge to
f (x) at every x. In fact,
Theorem 7.6.1 Suppose f (x) is piecewise continuous on [L, L]. Then (1) If
f (x0 + h) f (x0 )
x0 (L, L) and f (x+
),
0 ) and f (x0 ) both exist (where f (x ) := lim
h
h0
+
f (L+ )+f (L )
.
2
16
Remark. Suppose that f (x) is dened on [0, L]. We may extend f (x) to become
an even function on [L, L] (simply by dening f (x) := f (x) for x [L, 0]).
In this way, the Fourier coecients for the extended function become
2
L
0.
an
bn
f (x) cos
0
nx
dx
L
(7.64)
f (x) =
a0
nx
+
an cos
.
2
L
n=1
(7.65)
Similarly, we may extend f (x) to become an odd function on [L, L], in which
case we obtain a pure sine series
f (x) =
bn sin
n=1
with
bn =
2
L
f (x) sin
0
nx
L
(7.66)
nx
dx.
L
(7.67)
Example.
4
cos 3x cos 5x
+
+ . . .).
(cos x +
2
32
52
2. If g(x) = x, x , then
f (x) = 2(sin x
+ . . .).
2
3
4
3. If
h(x)
x( x),
x( + x),
for
for
0x
x 0
then
h(x) =
8
sin 3x sin 5x
(sin x +
+
+ . . .).
33
53
17
N 1
2
g(yk ) cos(nyk )
N
k=0
= (1)n
N 1
2 2k
2kn
f(
) cos
:= a
n .
N
N
N
(7.69)
k=0
f (x) = g(y) =
a0
an cos ny + bn sin ny
+
2
n=1
a0
an cos ny + bn sin ny
+
2
n=1
a
0
+
a
n cos n(x ) + bn sin n(x ).
2
n=1
18
jx
kx
f (x) =
fn ei
nx
L
(7.71)
n=
f (x)e
i kx
L
dx =
kx
L
n=
ei
nx
L
ei
kx
L
dx.
(7.72)
(7.73)
fn ei
nx
L
n=
= f0 +
fn (cos
n=
(fn + fn ) cos
n=1
nx
nx
+ i sin
)
L
L
nx
nx
+ i(fn fn ) sin
.
L
L
(7.74)
f (x) =
n=
fn einx
(7.75)
19
where
2
1
f (x)einx dx.
2 0
Consider the function g(x) of the form
fn =
g(x) =
dj eijx
(7.76)
(7.77)
j=
k=0 j=
N
1
dj (
j=
eijxk einxk ).
(7.79)
k=0
for n = 0, 1, . . . . Once, we see that (7.80) is a Trapezoidal approximation of the integral (7.76). The match between (7.49) and (7.80) is conspicuous except that the ranges of validity do not coincide. Consider the
case where N = 2 + 1. Then obviously j = dj for j = 0, 1, . . . , . But
N
1
N 1
1
N +j = N1
f (xk )ei(N +j)xk =
f (xk )eijxk = dj for j = 1, . . . , .
N
k=0
k=0
The central idea behind the fast Fourier transform (FFT) is that when N is
the product of integers, the numbers dj (or j ) prove to be closely interdependent. This interdependence can be exploited to substantially reduce the amount
of computation required to generated these numbers. We demonstrate the idea
as follows:
Suppose N = t1 t2 where both t1 and t2 are integers. Let
j : = j1 + t1 j2
n : = n2 + t2 n1
for j1 , n1 = 0, 1, . . . , t1 1 and j2 , n2 = 0, 1, . . . , t2 1. Note that both j and n
2
run their required ranges 0 to N 1. Let := ei N . Then N = 1. Thus
j = j1 +t1 j2
N 1
N 1
1
1
i 2jn
N
=
f (xn )e
=
f (xn ) nj
N n=0
N n=0
20
N 1
1
f (xn ) j1 n2 +j1 t2 n1 +t1 j2 n2
N n=0
t2 1 t
1 1
1
(
f (xn2 +t2 n1 ) j1 t2 n1 ) j1 n2 +t1 j2 n2 .
N n =0 n =0
2
(7.81)
t
1 1
f (xn2 +t2 n1 ) j1 t2 n1 ;
(7.82)
t2 1
1
F1 (j1 , n2 ) j1 n2 +t1 j2 n2 .
N n =0
(7.83)
n1 =0
j = F2 (j1 , j2 ) :=
n2
0
0
0
1
0
2
0
j1
0
0
2 3i
j1
j2
1
0
0
2
0
2 3i
21
t
1 1
f (xn ) j1 t3 t2 n1
(7.84)
(7.85)
n1 =0
F2 (j1 , j2 , n3 ) :=
t
2 1
n2 =0
j = F3 (j1 , j2 , j3 ) :=
t3 1
1
F2 (j1 , j2 , n3 ) (j1 +t1 j2 +t1 t2 j3 )n3 . (7.86)
N n =0
3
7.7
Uniform Approximation
(7.87)
22
(7.88)
(pf): Let
M := {x [a, b]||f (x) g(x)| = }.
(7.89)
(7.90)
where e(x) := f (x) g(x). The inequality in (7.90) is possible if and only if the
sign of p(x) is the same as that of e(x). That is, we must have (f (x)g(x))p(x) >
0 for all x M . By (7.88), it follows that the polynomial p must change signs
at least n times in [a, b]. That is, p must have at least n zeros. This contradict
with the assumption that p is not identically zero.
Remark. The above theorem asserts only that g is a best approximation whenever there are at least n + 1 points satisfying (7.87) and (7.88). In general, there
can be more points where the maximal deviation is achieved.
Example. Suppose we want to approximate f (x) = sin 3x over the interval
[0, 2]. It follows from the theorem that if n 1 4, then the polynomial g = 0
is a best approximation of f . Indeed, in this case the dierence f g alternates
between its maximal absolute value at six points, whereas the theorem only
requires n + 1 points. On the other hand, for n 1 = 5 we have n + 1 = 7, and
g = 0 no longer satises conditions (7.87) and (7.88). In fact, in this case g = 0
is not a best approximation from P5 .
Remark. The only property of Pn1 we have used to establish Theorem 7.7.1
is a weaker form of the Fundamental Theorem of Algebra, i.e., any polynomial
of degree n 1 has at most n 1 distinct zeros in [a, b]. This property is in fact
shared by a larger class of functions.
23
We now state without proof the famous result that Theorem 7.7.1 is not
only sucient but is also necessary for a polynomial g to a best approximation.
The following theorem is also known as the Alternation Theorem:
Theorem 7.7.2 The polynomial g Pn1 is a best approximation of the function f [a, b] if and only if there exist points a x1 < . . . < xn+1 b such
that conditions (7.87) and (7.88) are satised.
Denition 7.7.2 The set of points {x1 , . . . , xn+1 } in Theorem 7.7.2 is referred
to as an alternant for f and g.
Corollary 7.7.1 For any f C[a, b], there is a unique best approximation.
Theorem 7.7.1 also provides a basis for designing a method for the computation of best approximations of continuous functions. The idea, known as the
exchange method of Remez, is as follows:
24
The objective here is to have the set {xk } converge to a true alternant and
hence the polynomial converge to a best approximation. It can be proved that
the process does converge for any choice of starting values in step (1) for which
the value of computed in step (2) in not zero. With additional assumptions on
the dierentibility of f , it can also be shown that convergence is quadratic. Also
the assumption that e(x) possesses exactly n + 1 extrem points in step (3) is
not essential. For more details, refer to G. Meinarduss book Approximation of
Functions: Theory and Numerical Methods, Springer-Verlag, New York, 1967.