Results 11 
15 of
15
On Automatic Differentiation
 in Mathematical Programming: Recent Developments and Applications
, 1989
"... In comparison to symbolic differentiation and numerical differencing, the chain rule based technique of automatic differentiation is shown to evaluate partial derivatives accurately and cheaply. In particular it is demonstrated that the reverse mode of automatic differentiation yields any gradien ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
In comparison to symbolic differentiation and numerical differencing, the chain rule based technique of automatic differentiation is shown to evaluate partial derivatives accurately and cheaply. In particular it is demonstrated that the reverse mode of automatic differentiation yields any gradient vector at no more than five times the cost of evaluating the underlying scalar function. After developing the basic mathematics we describe several software implementations and briefly discuss the ramifications for optimization.
Automatic Differentiation: Point And Interval
, 1997
"... e applied. In summary, AD is more accurate than numerical differentiation and requires fewer resources and is more generally applicable than symbolic differentiation. The simplest type of algorithmic definition of a function is a code list , which is similar to the segment of computer code for the ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
e applied. In summary, AD is more accurate than numerical differentiation and requires fewer resources and is more generally applicable than symbolic differentiation. The simplest type of algorithmic definition of a function is a code list , which is similar to the segment of computer code for the evaluation of an expression (i.e., a formula). For illustration, consider the function defined by the formula f(x; y) = (xy + sin x + 4)(3y 2 + 6): An equivalent algorithmic definition of this function by a code list is t 1 = x; t 6 = t 5 + 4; t 2 = y; t 7 = t 2 2 ; t 3 = t 1 t 2 ; t 8 = 3t 7 ; t 4 = s
Computational Divided Differencing and DividedDifference Arithmetics
, 2000
"... Tools for computational differentiation transform a program that computes a numerical function F (x) into a related program that computes F 0 (x) (the derivative of F ). This paper describes how techniques similar to those used in computationaldifferentiation tools can be used to implement other pr ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Tools for computational differentiation transform a program that computes a numerical function F (x) into a related program that computes F 0 (x) (the derivative of F ). This paper describes how techniques similar to those used in computationaldifferentiation tools can be used to implement other program transformations  in particular, a variety of transformations for computational divided differencing . The specific technical contributions of the paper are as follows: It presents a program transformation that, given a numerical function F (x) de ned by a program, creates a program that computes F [x0 ; x1 ], the first divided difference of F(x), where F [x0 ; x1 ] def = F (x 0 ) F (x 1 ) x 0 x 1 if x0 6= x1 d dz F (z); evaluated at z = x0 if x0 = x1 It shows how computational first divided differencing generalizes computational differentiation. It presents a second program transformation that permits the creation of higherorder divided differences of a numerical function de ...
Numerical Solution Of Some Singular Unconstrained Minimization Problems
"... . Numerical solution of a singular unconstrained minimization problem is divided into two steps. Firstly, we consider a continuous analogue of the steepest gradient descent, which leads to a system of Volterra integral equations, and obtain convergence rate estimates. Secondly, the system is solved ..."
Abstract
 Add to MetaCart
. Numerical solution of a singular unconstrained minimization problem is divided into two steps. Firstly, we consider a continuous analogue of the steepest gradient descent, which leads to a system of Volterra integral equations, and obtain convergence rate estimates. Secondly, the system is solved by an approximateiterative method. An estimate of the total absolute error is given as a sum of the inherent error, the error of the numerical method, and the roundo error. The estimate enables one to determine parameters of the computational process and to solve the initial problem with preassigned accuracy. Key words. Unconstrained minimization, singular problems, Volterra integral equation, rate of convergence, error estimate. AMS(MOS) subject classications. 65K05, 65R20, 65G05 1. Introduction. We consider the following unconstrained minimization problem: nd x 2 R n , for which (1) f f(x ) = min x2R n f(x); where R n is the ndimensional real Euclidean space. We a...
Exact Complexity Results and PolynomialTime Algorithms for Derivative Accumulation
"... We discuss the complexity of evaluating certain collections of monic multilinear polynomials using operations in {×, +}. The functions we consider, which are defined on paths in directed acyclic graphs (DAGs), represent derivative computations that are based on the chain rule and have direct applica ..."
Abstract
 Add to MetaCart
We discuss the complexity of evaluating certain collections of monic multilinear polynomials using operations in {×, +}. The functions we consider, which are defined on paths in directed acyclic graphs (DAGs), represent derivative computations that are based on the chain rule and have direct applications in highperformance scientific computing. Our main results concern functions derived from singlesource, singlesink DAGs whose maximal paths all have length three. We derive tight, exact lower bounds for the numbers of multiplications, additions, and total arithmetic operations needed. Moreover, we show that, given such a DAG, an arithmetic circuit (or straightline program) of minimum size that evaluates J (G) can be constructed in polynomial time. In contrast, we show the (perhaps surprising) result that the problem of finding a circuit of minimum size for a given DAG becomes NPhard, even for the restricted class of DAGs considered in this paper, when some subset of the arcs may be labeled with the multiplicative identity 1 rather than an indeterminate. 1