## On Properties of Floating Point Arithmetics: Numerical Stability and the Cost of Accurate Computations (1992)

Citations: | 26 - 0 self |

### BibTeX

@TECHREPORT{Priest92onproperties,

author = {Douglas M. Priest},

title = {On Properties of Floating Point Arithmetics: Numerical Stability and the Cost of Accurate Computations},

institution = {},

year = {1992}

}

### Years of Citing Articles

### OpenURL

### Abstract

Floating point arithmetics generally possess many regularity properties in addition to those that are typically used in roundoff error analyses; these properties can be exploited to produce computations that are more accurate and cost effective than many programmers might think possible. Furthermore, many of these properties are quite simple to state and to comprehend, but few programmers seem to be aware of them (or at least willing to rely on them). This dissertation presents some of these properties and explores their consequences for computability, accuracy, cost, and portability. For example, we consider several algorithms for summing a sequence of numbers and show that under very general hypotheses, we can compute a sum to full working precision at only somewhat greater cost than a simple accumulation, which can often produce a sum with no significant figures at all. This example, as well as others we present, can be generalized further by substituting still more complex algorith...

### Citations

2202 |
The Art of Computer Programming
- Knuth
- 1973
(Show Context)
Citation Context ...bsequently, we show that with a few additional hypotheses, we can reduce the cost of the algorithm by nearly half, thereby obtaining the forms proved correct for the various arithmetics considered in =-=[16, 33, 40, 43, 51, 54]-=-. Proposition: Let a and b be floating point numbers. If the floating point arithmetic is faithful, then the numbers c and d computed by the following algorithm satisfy c + d = a + b and either c = d ... |

1764 |
Computational Geometry: An Introduction
- Preparata, Shamos
- 1985
(Show Context)
Citation Context ...f floating point computation parallels the Blum-Shub-Smale (BSS) model of computation over a ring; see [6]. One could instead define a model which resembles the real RAM model of Preparata and Shamos =-=[55]-=- or the algebraic computation tree of Ben-Or [5], but for our purposes these models are roughly equivalent to the BSS model. One could similarly define floating point algorithms literally in terms of ... |

377 |
On a theory of computation and complexity over the real numbers
- Blum, Shub, et al.
- 1989
(Show Context)
Citation Context ...ust specify a formal model of computation in which we can describe algorithms. Our formal model of floating point computation parallels the Blum-Shub-Smale (BSS) model of computation over a ring; see =-=[6]-=-. One could instead define a model which resembles the real RAM model of Preparata and Shamos [55] or the algebraic computation tree of Ben-Or [5], but for our purposes these models are roughly equiva... |

215 |
bounds for algebraic computation trees
- Ben-Or, Lower
- 1983
(Show Context)
Citation Context ...hub-Smale (BSS) model of computation over a ring; see [6]. One could instead define a model which resembles the real RAM model of Preparata and Shamos [55] or the algebraic computation tree of Ben-Or =-=[5]-=-, but for our purposes these models are roughly equivalent to the BSS model. One could similarly define floating point algorithms literally in terms of Fortran programs, albeit with the understanding ... |

106 |
A floating-point technique for extending the available precision
- Dekker
(Show Context)
Citation Context ...thfulness as compared with the (1 + ffl) property: it states that we can compute exactly the roundoff error incurred in the floating point addition of two numbers. As observed by M��ller [51], Dek=-=ker [16]-=-, Linnainmaa [44], and others, this technique can be used to effectively extend the precision of a floating point computation, in some cases to the point of computing an exact representation of the re... |

94 |
Rounding Errors
- Wilkinson
- 1963
(Show Context)
Citation Context ... a simple property we refer to as the "(1 + ffl)" property, which first appeared in the earliest floating point roundoff error analyses of Givens, Wilkinson, and their colleagues (see the co=-=mments in [64]-=-). Considering the weakness of the (1 + ffl) property, these error analyses achieve remarkably strong results by relying on the elegant technique of backward error analysis, and the error estimates th... |

74 |
L.: Computer Arithmetic in Theory and Practice
- Kulisch, Miranker
- 1981
(Show Context)
Citation Context ...roblems we can solve accurately, we are naturally led to consider the costs of accurate computations. In section 3, we compare our model with several others which exhibit similar computational power (=-=[41, 2]-=-). As a result, we show that the differences between these models can lead to significant variations in the cost we can predict for certain types of computations given the accuracy we wish to achieve.... |

70 | Essential of Numerical Analysis - Henrici - 1982 |

69 |
Verifiable implementations of geometric algorithms using finite precision arithmetic
- Milenkovic
- 1988
(Show Context)
Citation Context ...l perturbations into the geometry and manipulate certain derived objects symbolically without actually computing them, as suggested by the data normalization and hidden variable methods of Milenkovic =-=[49]-=-. Needless to say, in order to maintain consistency, this approach requires great care and expense. Again, we propose a simpler solution: by computing the vertices of the clipped line segments correct... |

65 | Algorithms for arbitrary precision floating point arithmetic - Priest - 1991 |

60 |
Sylvester’s identity and multistep integer-preserving Gaussian elimination
- Bareiss
- 1968
(Show Context)
Citation Context ...mediate results and hence the cost of subsequent computations either by performing exact divisions at an earlier stage, as is done in integer-preserving Gaussian elimination, for example (see Bareiss =-=[3]-=-), or by rounding off intermediate results; we simply suggest that these steps be done only when they can be proved not to contaminate the result. Consider how these different paradigms impact the rel... |

58 |
Pracniques: further remarks on reducing truncation errors
- Kahan
- 1965
(Show Context)
Citation Context ...bsequently, we show that with a few additional hypotheses, we can reduce the cost of the algorithm by nearly half, thereby obtaining the forms proved correct for the various arithmetics considered in =-=[16, 33, 40, 43, 51, 54]-=-. Proposition: Let a and b be floating point numbers. If the floating point arithmetic is faithful, then the numbers c and d computed by the following algorithm satisfy c + d = a + b and either c = d ... |

48 | MPFUN: A Portable High Performance Multiprecision Package.” NAS Applied Research Office, NASA Ames Research Center, 5 Field, CA 94035
- Bailey
- 1991
(Show Context)
Citation Context ...roblems we can solve accurately, we are naturally led to consider the costs of accurate computations. In section 3, we compare our model with several others which exhibit similar computational power (=-=[41, 2]-=-). As a result, we show that the differences between these models can lead to significant variations in the cost we can predict for certain types of computations given the accuracy we wish to achieve.... |

40 |
A stable and efficient algorithm for the rankone modification of the symmetric eigenproblem
- Gu, Eisenstat
- 1994
(Show Context)
Citation Context ...n cannot be per92 formed, perhaps because the properties upon which it depends are not possessed by the computer we wish to use, then we must resort to a clever approach suggested by Gu and Eisenstat =-=[25]-=-. They avoid the use of multiple precision computation by evaluating the relevant function to working precision, yielding a possibly inaccurate result for the given data, then perturbing the original ... |

35 | Underflow and the reliability of numerical software - Demmel - 1984 |

34 |
Correction d’une somme en arithmétique à virgule flottante
- Pichat
- 1972
(Show Context)
Citation Context ...bsequently, we show that with a few additional hypotheses, we can reduce the cost of the algorithm by nearly half, thereby obtaining the forms proved correct for the various arithmetics considered in =-=[16, 33, 40, 43, 51, 54]-=-. Proposition: Let a and b be floating point numbers. If the floating point arithmetic is faithful, then the numbers c and d computed by the following algorithm satisfy c + d = a + b and either c = d ... |

33 |
Floating-Point Computation
- Sterbenz
- 1974
(Show Context)
Citation Context ...the (1 + ffl) property in the first place. Nevertheless, faithfulness is a much stronger property, as we will demonstrate in the following results. We begin with two simple lemmas. Lemma 1: (Sterbenz =-=[61]-=-) If a and b are t-digit, radix fi floating point numbers such that 1=2sa=bs2, then a \Gamma b is also a t-digit, radix fi floating point number. In particular, in a faithful arithmetic, fl(a \Gamma b... |

32 |
On the orthogonality of eigenvectors computed by divide{and{conquer techniques
- Sorensen, Tang
- 1991
(Show Context)
Citation Context ...omputes its value to full relative accuracy even when jzj is small. If expm1 is not available, a more subtle program may be needed; see [36].) A more relevant example is provided by Sorensen and Tang =-=[60]-=-. They show that by evaluating a certain function to at least twice working precision, a well-known parallel algorithm for computing eigensystems of real symmetric matrices can be guaranteed to give e... |

26 |
A simple but realistic model of floating-point computation
- Brown
- 1981
(Show Context)
Citation Context ...ore importantly, the goal of an axiomatic approach is the unification of hypotheses, and Dekker's distinction between different families of properties contributes little to that goal. The Brown model =-=[11], subseque-=-ntly expressed as a formal specification of floating point arithmetic by Wichmann [63], also specifically includes a definition of float27 ing point arithmetic. Brown defines "model numbers"... |

23 |
Some remarks on the foundations of numerical analysis
- Smale
- 1990
(Show Context)
Citation Context ...thms with exact computations. We begin by defining a model of real number computation and considering the problems which can be solved in such a model. Here we follow some of the development in Smale =-=[58]-=-. The most intuitive notion of a problem is a mapping which assigns to each instance of the problem, or input, the corresponding solution, or output. The inputs and outputs usually lie in distinguishe... |

19 |
A Fortran Multiple Precision Arithmetic Package
- Brent
- 1978
(Show Context)
Citation Context ...l examine more closely the relationship between arbitrary precision arithmetic with expansions and multiple precision floating point arithmetic, such as that provided by a number of software packages =-=[2, 10, 59]-=-. For now we simply remark that, unlike multiple precision floating point, the sparse representation of expansions allows us to bound the cost of a computation solely in terms of the sizes of the inpu... |

19 |
Systematic Programming: An Introduction
- Wirth
- 1973
(Show Context)
Citation Context ...ning under other types of arithmetics which could not possibly possess properties analogous to properties S1--S3, for example. The same deficiency afflicts the axioms suggested by Holm [29] and Wirth =-=[66]-=-, although both specify an otherwise faithful, monotonic, commutative, and anti-symmetric arithmetic. Holm uses only fifteen axioms, compared with van Wijngaarden's 32, by explicitly referring to two ... |

15 |
Algorithm 693: A FORTRAN package for floatingpoint multiple-precision arithmetic
- Smith
- 1991
(Show Context)
Citation Context ...l examine more closely the relationship between arbitrary precision arithmetic with expansions and multiple precision floating point arithmetic, such as that provided by a number of software packages =-=[2, 10, 59]-=-. For now we simply remark that, unlike multiple precision floating point, the sparse representation of expansions allows us to bound the cost of a computation solely in terms of the sizes of the inpu... |

13 |
Computational graphs and rounding error
- Bauer
- 1974
(Show Context)
Citation Context ...umber machine (with exact rational arithmetic operations) can be solved approximately by a numerically stable floating point algorithm. The theorem comes close to answering a question raised by Bauer =-=[4]: &qu-=-ot;A constructive derivation of a benign process [for an arbitrary problem] would be a great success---but it is not even clear whether to every problem that is defined by some process, a benign proce... |

13 |
Experiments on Gram-Schmidt orthogonalization
- Rice
- 1966
(Show Context)
Citation Context ... We caution, however, that such vagueness may lead to some confusion, since subtle variations can produce very different algorithms. Consider the following example, evidently first discovered by Rice =-=[57]-=-. The Gram-Schmidt algorithm is a well-known way to compute a factorization A = QR of a matrix A so that Q is orthogonal and R is upper triangular with non-negative diagonal; the algorithm is often pr... |

12 |
Precision Geometry: A General Technique for Calculating Line and Segment Intersections using Rounded Arithmetic
- Double
- 1989
(Show Context)
Citation Context ...s and can be generalized to any faithful arithmetic by using the more complicated sum-and-roundoff algorithm of chapter 2 together with other algorithms in appendix A. Along similar lines, Milenkovic =-=[50]-=- showed that if line equation coefficients are represented as single precision numbers with t 1 bits and computations are performed in double precision with t 2 bits, where t 2s2t 1 + 1, then the inte... |

11 |
Algorithm 665. machar: A subroutine to dynamically determine machine parameters
- Cody
- 1988
(Show Context)
Citation Context ...on of the input and output functions, the form of a floating point machine is independent of t, the precision of the arithmetic. (Such a machine can compute the precision t and radix fi, however; see =-=[14, 22, 44, 47]-=-, for example.) As in appendix A, we consider the extension of floating point arithmetic to arbitrary precision by representing each number as an unevaluated sum of fixed precision floating point numb... |

11 |
Analysis of some known methods of improving the accuracy of floatingpoint sums
- Linnainmaa
- 1974
(Show Context)
Citation Context |

9 |
A versatile precompiler for nonstandard arithmetics
- Crary
- 1979
(Show Context)
Citation Context ...precisions associated with certain variables. His translator then substitutes the appropriate calls to an MPFUN subroutine for each arithmetic operation involving multiple precision variables. (Crary =-=[15]-=- describes a similar translator which can be used with other multiple precision packages such as Brent's MP package.) Note that in this paradigm, each different precision of arithmetic functions as a ... |

9 |
Compiler Support for Floating-point Computation
- Farnum
- 1988
(Show Context)
Citation Context ... possibly be proved correct in a programming environment that unfathomable, since there is no way to know whether the value substituted for a variable is the same as that last assigned to it. (Farnum =-=[20]-=- and Goldberg [24] note this and other potential problems which compiler writers ideally should avoid. Their advice appears to be too often ignored, however, judging from Cody's account [14] of the pr... |

8 |
Floating-Point Computation of Functions with Maximum Accuracy
- Bohlender
- 1977
(Show Context)
Citation Context ...f course, if the summands are presorted, the doubly compensated summation method given above produces the first component at a cost comparable to two or three passes of Pichat's algorithm.) Bohlender =-=[7]-=- later showed that Pichat's algorithm produces the entire expansion for the sum in at most n \Gamma 1 passes, and he added a stopping criterion for the case where only a given number of leading compon... |

5 |
Stable Evaluation of Polynomials
- Meszteny, Witzgall
- 1967
(Show Context)
Citation Context ...ring the iterative refinement on polynomials which our algorithm can handle without difficulty. Finally we note another approach to accurate polynomial evaluation considered by Mesztenyi and Witzgall =-=[48]-=-. They prove that if the polynomial is given in an appropriate Newton form (i.e., in the form f(x) = P d i=0 c i Q i j=1 (x \Gamma x j )) then evalu87 ation by the obvious modification of synthetic di... |

3 |
Evaluation of Arithmetic Expressions with Maximum Accuracy, in A New Approach to Scientific Computation
- Bohm
- 1983
(Show Context)
Citation Context ...fying rfi 1\Gammaqt ! ffl(jyj \Gamma rfi 1\Gammaqt ) for some jyjsjf(x)j \Gamma rfi 1\Gammaqt ; hence q �� 1 t (1 \Gamma log fi ffljf (x)j (1+2ffl)r ). We note that a similar bound is claimed by B=-=ohm [9]-=- for another polynomial evaluation algorithm. His method first computes an approximation to f(x) using working precision, then repeatedly refines this approximation by first computing interval bounds ... |

3 |
Recipes for Geometry and Numerical Analysis
- Dobkin, Silver
- 1988
(Show Context)
Citation Context ...computed in version 2, so that the overall cost will be slightly less than the sum of the costs of each version alone. (In this respect, our approach is similar to that suggested by Dobkin and Silver =-=[19]-=-.) Moreover, the most pessimistic quantities in the cost estimates arise from the summation procedure, which we have taken to be doubly compensated summation with an initial O(n 2 ) sort; hence, the c... |

3 |
Computer Arithmetic, Appendix A in
- Goldberg
- 1990
(Show Context)
Citation Context ...ion or square root with n-bit operands rounded first to p bits and subsequently to n bits will yield the same value as if the operation were rounded to n bits directly provided ps2n + 1 (see Goldberg =-=[23]). "d-=-estination" of an arithmetic operation within each programming language standard so as to close the loophole mentioned above. Then the IEEE standard would reflect the true behavior of the program... |

3 |
for doubled-precision floating-point computations
- Software
- 1981
(Show Context)
Citation Context ...ared with the (1 + ffl) property: it states that we can compute exactly the roundoff error incurred in the floating point addition of two numbers. As observed by M��ller [51], Dekker [16], Linnain=-=maa [44]-=-, and others, this technique can be used to effectively extend the precision of a floating point computation, in some cases to the point of computing an exact representation of the result (see appendi... |

3 |
Numerical Stability of Simple Geometric Algorithms
- Ottman, Thiemt, et al.
- 1987
(Show Context)
Citation Context ...erent summation algorithm. Therefore, we suggest that the preceding algorithm is not only provably accurate and robust, but could easily be made very efficient as well. We remark that Ottman, et al , =-=[52, 53]-=- show how to compute intersection points to full accuracy using single precision arithmetic, a means of computing inner products to full precision, and interval arithmetic methods based on directed ro... |

2 | Y-MP Floating Point and Cholesky Factorization
- Carter
- 1991
(Show Context)
Citation Context ...0, j = \Gammaa, k = 0, l = 0, and d = 0, but c + d = 0 6= a + b. By these and other counterexamples, we do not intend to suggest that the Cray 2 is less accurate than the Cray X/Y-MP; in fact, Carter =-=[12]-=- presents an analysis that suggests the opposite: because the Cray 2 sometimes errs in its subtraction by subtracting too little and other times errs by subtracting too much, the errors tend to cancel... |

2 |
Interval Arithmetic Implementations Using Floating Point
- Clemmesen
- 1984
(Show Context)
Citation Context ...ng point operations. Even directed roundings can be simulated in any faithful arithmetic that does not provide them, albeit at a cost that would not be tolerable for practical purposes (see Clemmesen =-=[13]-=-). The primary difference between these three models lies in the paradigms for numerical programming that each suggests. Specifically, Bailey's package suggests a paradigm in which we simply implement... |

2 |
More on Algorithms That Reveal Properties of Floating Point Arithmetic Units
- Gentleman, Marovich
- 1974
(Show Context)
Citation Context ...on of the input and output functions, the form of a floating point machine is independent of t, the precision of the arithmetic. (Such a machine can compute the precision t and radix fi, however; see =-=[14, 22, 44, 47]-=-, for example.) As in appendix A, we consider the extension of floating point arithmetic to arbitrary precision by representing each number as an unevaluated sum of fixed precision floating point numb... |

2 | Parallel Algorithms for the Rounding-Exact Summation of Floating-Point Numbers, Computing 28 - Leuprecht, Oberaigner - 1982 |

2 |
Quasi Double Precision
- M��ller
- 1965
(Show Context)
Citation Context ...rength of faithfulness as compared with the (1 + ffl) property: it states that we can compute exactly the roundoff error incurred in the floating point addition of two numbers. As observed by M��l=-=ler [51]-=-, Dekker [16], Linnainmaa [44], and others, this technique can be used to effectively extend the precision of a floating point computation, in some cases to the point of computing an exact representat... |

2 |
Numerical Analysis as an Independent
- Wijngaarden
- 1966
(Show Context)
Citation Context ... of the model which may necessitate a more expensive program or a more complicated analysis. The first attempt to formulate a complete set of axioms for numerical analysis was made by van Wijngaarden =-=[62]-=-. His axioms describe a faithful, monotonic arithmetic (with a few additional axioms specifically dealing with the interaction between real and integer quantities), but one cannot deduce the results o... |

2 |
Towards a Formal Specification of Floating Point
- Wichmann
- 1989
(Show Context)
Citation Context ...s distinction between different families of properties contributes little to that goal. The Brown model [11], subsequently expressed as a formal specification of floating point arithmetic by Wichmann =-=[63], also spe-=-cifically includes a definition of float27 ing point arithmetic. Brown defines "model numbers" to be floating point numbers within a given exponent range, although he allows an arithmetic to... |

1 |
A Stopping Criterion for Polynomial Root
- Adams
- 1967
(Show Context)
Citation Context ...i.e., for the polynomial P d i=0 a i x i , the computed value y can be written y = P d i=0 b i x i where jb i \Gamma a i j ! (2i+1) log(1+fi 1\Gammat )ja i j. Another estimate due to Kahan (see Adams =-=[1]-=-) shows that the same algorithm is stable for bounded coefficients and arguments with respect to a forward absolute error criterion: specifically, jy \Gamma P a i x i j ! 2fi 2\Gammat P js i x i j whe... |

1 |
Do We Need Beyond
- What
- 1990
(Show Context)
Citation Context ...ing components are desired. Many other algorithms which evaluate a sum or an inner product to working precision (i.e., yielding the first component of an expansion) have been developed; see Bohlender =-=[8]-=- for a survey of several approaches. Unfortunately, many of these techniques rely on special features, such as an extra wide fixed point accumulator, directed roundings, or interval arithmetic, which ... |

1 |
Patriot Missile Defense: Software Problem Led to System Failure at Dhahran, Saudi Arabia
- Report
(Show Context)
Citation Context ...eed only be cognizant of a few simple but important properties. The price we might pay for lack of such an understanding is more difficult to estimate. As a start, we might consider the recent report =-=[21]-=- on a Scud missile attack on Dhahran, Saudi Arabia, during the 1991 Persian Gulf war, in which a simple floating point roundoff error led to the failure of a Patriot missile defense system to correctl... |

1 |
Every Computer Scientist Should Know About Floating-Point Arithmetic
- What
- 1991
(Show Context)
Citation Context ...d correct in a programming environment that unfathomable, since there is no way to know whether the value substituted for a variable is the same as that last assigned to it. (Farnum [20] and Goldberg =-=[24]-=- note this and other potential problems which compiler writers ideally should avoid. Their advice appears to be too often ignored, however, judging from Cody's account [14] of the prestidigitation req... |

1 |
Polynomials with Multiple Zeroes
- Guting
- 1967
(Show Context)
Citation Context ...e polynomial with enough precision to guarantee at most that much absolute error. Unfortunately, the bounds upon which this approach and others like it are based tend to be so large (such as Guting's =-=[26]-=- bounds, which are exponential in d, for example) that the resulting precision estimates will almost always yield a computation which is more expensive than simply evaluating the polynomial exactly an... |

1 |
The Accuracy of Floating Point Summation, Numerical Analysis Report
- Higham
- 1991
(Show Context)
Citation Context ...putations, the cost of extra accuracy may be intolerable. Section 1 considers a number of variations on the problem of summing a set of numbers, augmenting the repertoire of methods studied by Higham =-=[28]-=- with an algorithm which produces a sum correct to working precision and another which produces the exact sum in the form of an expansion. In section 2, we apply the techniques in appendix A to severa... |