Section 1.

Floating Point Arithmetic 39

1.4 Floating Point Arithmetic

Todays modern computers, even home personal computers, can usually store enough digits so that roundo error is not too much of a problem, at least at the storage level. However, computers can literally make millions or billions of calculations in a numerical process, so the original roundo error can propogate throughout the calculation, rendering the nal answer meaningless. In this section we look at oating point arithmetic and discuss some numerical eects that can have disastrous results on caclulations. Lets begin with an example that demonstrates catastrophic cancellation of digits and loss of precision. Example 1. Compute 9876 9875 using 5-digit oating point arithmetic. First, lets examine the relative error in storing 9876 in 5-digit oating point form. Were going to use Matlabs Symbolic Toolbox, an interface to the computer algebra system (CAS) Maple. Computations in Maple are done symbolically and are exact. Thus, instead of turning a numeric result, an exact expression is returned. >> sym(sqrt(9876)) ans = sqrt(9876) The Symbolic Toolbox has a command that will return a numerical approximation of a symbolic object, correct to as many places of accuracy that the user needs. The command vpa does the work. >> help vpa VPA Variable precision arithmetic. R = VPA(S) numerically evaluates each element of the double matrix S using variable precision floating point arithmetic with D decimal digit accuracy, where D is the current setting of DIGITS. The resulting R is a SYM. VPA(S,D) uses D digits, instead of the current setting of DIGITS. D is an integer or the SYM representation of a number.

Copyrighted material. See: http://msenux.redwoods.edu/Math4Textbook/

40 Chapter 1

Numeric Types in Matlab

We will use the last paragraph of the help message to approximate 9876, correct to 20 digits. Note that the symbolic object must be entered as a string (delimited by single apostrophes (ticks)). >> vpa(sqrt(9876),20) ans = 99.378065990438755368 Use this result to store 9876 in 5-digit oating point format. 9876 = 9.9378 101 .

We calculate the relative error with

|x x| |9.9378 101 9876| = . |x| | 9876|

(1.1)

To nd an approximate value of the relative error, we will again turn to the Symbolic Toolbox. Well rst store the needed computation as a string in the variable rel (use any variable name you like). >> rel=abs(9.9378e1-sqrt(9876))/abs(sqrt(9876)) rel = abs(9.9378e1-sqrt(9876))/abs(sqrt(9876)) Now well use the vpa command to approximate to 20 digits of accuracy. >> vpa(rel,20) ans = .66403424234193683981e-6 Thus, the relative error in storing x = 9876 in 5-digit oating point form x = 9.9378 101 is approximately 6.6 107 . Recall the denition of signicant digits. Significant Digits. The number x is said to approximate x to n signicant digits if n is the largest nonnegative integer for which |x x| < 5 10n . |x|

Section 1.4

Floating Point Arithmetic 41

Because the relative error is less than 5 106 , we know that x = 9.9378 101 approximates x 9876 to 6 signicant digits. In similar fashion, we can use the Symbolic Toolbox to approximate 9875 and nd the relative error in storing this number in 5-digit oating point form. First, an approximation of 9875. >> vpa(sqrt(9875),20) ans = 99.373034571758952600 Thus, the 5-digit oating point storage of x = relative error in storing 9875 is next. 9875 is x = 9.9373 101 . The

>> rel=abs(9.9373e1-sqrt(9875))/abs(sqrt(9875)) rel = abs(9.9373e1-sqrt(9875))/abs(sqrt(9875)) >> vpa(rel,20) ans = .34789879469399867106e-6 Thus, the relative error is approximately 3.4 107 , which is about the same as the relative error in storing 9876 in 5-digit oating point form. Because the 7 1 relative error is less than 5 10 , we know that x = 9.9373 10 approximates x 9875 to 7 signicant digits. Now, lets subtract the stored 5-digit oating point form numbers and see what happens. 9.9378 101 9.9373 101 = 0.0005 101 Note that all of the digits of the mantissa are gone save one. Further, note that there are no more digits to the right of the 5 in 0.00005 101 , so when we adjust the exponent to place this result in 5-digit oating point form, as in 5.0000 103 , the digits to the right of the decimal point in 5.0000 103 are meaningless. Indeed, the computer could quite possibly shove in some digits from memory that are not zeros, just to pad the number. Now, lets look at the relative error in approximating x = 9876 9875 with this result.

42 Chapter 1

Numeric Types in Matlab

>> rel= abs(5.0e-3-(sqrt(9876)-sqrt(9875)))/abs(sqrt(9876)-sqrt(9875)) rel = abs(5.0e-3-(sqrt(9876)-sqrt(9875)))/abs(sqrt(9876)-sqrt(9875)) >> vpa(rel,20) ans = .62444971890114329880e-2 The relative error 6.2 103 . Recall that the relative error in is approximately approximating 9876 and 9875 with 5-digit oating point numbers was of the order of 107 , so roughly speaking, the relative error made by subtracting, which is of the order 103 , is roughly 103 /107 = 104 , or 10 000 times larger! Further, because the relative error of the subtraction is less than 5 102 , only 2 signicant digits remain! This phenomenon is called catastrophic cancelllation of digits and occurs whenever you attempt to subtract two numbers that are very close to one another. The programmer needs to be aware of this phenomenon and avoid subtracting two nearly equal numbers. In this case, we can change the subtraction into addition by rationalizing the numerator with this calculation. 1 9876 9875 = 9876 9875 = 9876 + 9875 9876 + 9875 If we add the oating point representations of 9876 and 9875, we get 9.9378 101 + 9.9373 101 = 19.8751 102 , which when stored in 5-digit oating point form is 1.9875 102 . Next, we perform the division, 1.0000 100 = 0.50314465408805 102 , 1.9875 102
3 The relative error which when stored in 5-digit oating form is x =5.0314 10 . when approximating x = 9876 9875 with x = 5.0314 10 3 is 5.0314 103 ( 9876 9875) |x x| = . |x| 9876 9875

We can use the Symbolic Toolbox to help with this calculation.

Section 1.4

Floating Point Arithmetic 43

>> rel= abs(5.0313e-3-(sqrt(9876)-sqrt(9875)))/abs(sqrt(9876)-sqrt(9875)) rel = abs(5.0313e-3-(sqrt(9876)-sqrt(9875)))/abs(sqrt(9876)-sqrt(9875)) >> vpa(rel,20) ans = .23587741414644558543e-4 Thus, the relative error when approximating x = 9876 9875 with x = 5.0314 103 is approximately 2.3 105 , which is less than 5 105 , so this time weve kept 5 signicant digits, which is much better than the 2 signicant digits of the previous computation. Lets look at another example. Example 2. Solve the quadratic equation x2 1634x + 2 = 0 using 10-digit oating point arithmetic. Using the quadratic formula to solve x2 1634x + 2 = 0, we get 16342 4(1)(2) = 817 667 487. 2(1) We can use the Symbolic Toolbox to approximate 667 487. x= 1634 >> vpa(sqrt(667487),20) ans = 816.99877600887505858 Thus, in 10-digit oating point form, solution of the quadratic is 667 487 8.169987760 102 . Thus, one

2 2 2 x 1 = 8.170000000 10 + 8.169987760 10 = 16.339987760 10 , 3 or x the Symbolic 1 = 1.633998776 10 in 10-digit oating point form. We can use Toolbox to calculate the relative error in approximating x1 = 817+ 667 487 with this 10-digit oating point number.

44 Chapter 1

Numeric Types in Matlab

>> rel= abs(1.633998776e3-(817+\sqrt(667487)))/abs(817+sqrt(667487)) rel = abs(1.633998776e3-(817+\sqrt(667487)))/abs(817+sqrt(667487)) >> vpa(rel,20) ans = .54314964676275835537e-11 Thus, the relative error is approximately 5.4 1012 , which is less than 5 1011 , so we are approximating 817 + 667 487 to 11 signicant digits. The second root is
2 2 2 x 2 = 8.170000000 10 8.169987760 10 = 0.000012240 10 ,

or x2 = 1.224000000 103 , in 10-digit oating point form. Note the catastrophic cancellation of digits. The last 5 zeros of 1.224000000 are completely meaningless and in some cases the computer will pad these places with nonzero digits that happen to be lying around in memory. We can use the Symbolic Toolbox to calculate the relative error made when approximating x2 = 817 667 487 with the 10-digit oating point number 3 x 2 = 1.224000000 10 . >> rel= abs(1.224000000e-3-(817-\sqrt(667487)))/abs(817-sqrt(667487)) rel = abs(1.224000000e-3-(817-\sqrt(667487)))/abs(817-sqrt(667487)) >> vpa(rel,20) ans = .72509174283635093702e-5 Thus, the relative error is approximately 7.3 106 , which is less than 5 105 , so 3 x2 = 1.224000000 10 is approximating x2 = 817 667 487 to only 5 signicant digits! Programmers have to be aware of catastrophic cancellation of digits any time to numbers are subtracted that are very nearly equal in value. The programmer has to look for an algorithm that avoids subtraction. In this case, consider the idea that if x1 and x2 are roots of the quadratic equation x2 1634x + 2 = 0, then the quadratic quadratic x2 1634x + 2 can be

Section 1.4

Floating Point Arithmetic 45

factored as (x x1 )(x x2 ). If we expand and compare to the original quadratic, then x2 1634x + 2 = (x x1 )(x x2 ) = x2 (x1 + x2 )x + x1 x2 . Comparing the constant terms, x1 x2 = 2, or equivalently, x2 = 1 . x1

This last result will allow avoid subtraction in calculation the second root x2 . x 2 = 2 x 1 2.000000000 100 = 1.633998776 103 = 1.2239911369332985296 103 = 1.223991137 103 We 817 can use Matlab to calculate the relative error in approximating x2 = 3 667 487 with the 10-digit oating point number x2 = 1.223991137 10 . >> rel= abs(1.223991137e-3-(817-\sqrt(667487)))/abs(817-sqrt(667487)) rel = abs(1.223991137e-3-(817-\sqrt(667487)))/abs(817-sqrt(667487)) >> vpa(rel,20) ans = .98518524802025190487e-8 Thus, the relative error is approximately 9.9 109 , which is less than 5 108 , so this time weve managed to hang on to 8 signicant digits.

46 Chapter 1

Numeric Types in Matlab

1.4 Exercises
In Exercises 1-4, perform each of the following tasks for the given number. i. Use Matlabs vpa command to approximate the given number to 20 digits. ii. Place the result of the vpa command into n-digit oating point format for the given value of n. iii. Use the vpa command to determine the relative error. iv. Use the denition of signicant digits to determine the number of signicant digits in your n-digit oating point approximation. 1. 14 385, n = 5 2. 23 888, n = 5 3. ln888 475, 4. ln 1 234 555, n = 10 n = 10 iv. the relative error in approximating the given expression with the ndigit oating point number of part (ii). Use the denition of signicant digits to determine the number of signicant digits in the approximation of part (ii). 8355 8354, n = 5 7565 7564, n = 5 45619 45617, n = 10 387159 387156, n = 10

5. 6. 7. 8.

In Exercises 9-12, perform each of the following tasks for the given quadratic equation. i. Solve the given quadratic by hand. Place your solution in simple radical form. ii. Use Matlabs vpa command to approximate the radical in your solution to 20 digits, then place the result in 10-digit oating point format. iii. Use hand calculations to determine the solution that doesnt cause catastrophic cancellation of digits in 10-digit oating point form. Use the vpa command to determine the relative error and the number of signicant digits of this result. iv. Follow the lead of Example 2 in the narrative to obtain a 10-digit oating point approximation of the second solution. Then use the vpa command to determine the rela-

In Exercises 5-8, perform each of the following tasks for the given expression. i. Use the vpa command of the Symbolic Toolbox to assist in nding n-digit oating point approximations of each root for the given value of n. Calculate the relative error for each and state the number of signicant digits for each. ii. Use the results of the previous part to hand calculate an n-digit oating point approximation for the given expression. iii. Use Matlabs vpa command to nd

Section 1.4 tive error and the number of signicant digits. 9. x2 4744x + 2 = 0 10. x2 5666x + 4 = 0 11. x2 388x + 2 = 0 12. x2 644x + 1 = 0

Floating Point Arithmetic 47