Results 1  10
of
15
On Automatic Differentiation
 IN MATHEMATICAL PROGRAMMING: RECENT DEVELOPMENTS AND APPLICATIONS
, 1989
"... In comparison to symbolic differentiation and numerical differencing, the chain rule based technique of automatic differentiation is shown to evaluate partial derivatives accurately and cheaply. In particular it is demonstrated that the reverse mode of automatic differentiation yields any gradient v ..."
Abstract

Cited by 138 (14 self)
 Add to MetaCart
In comparison to symbolic differentiation and numerical differencing, the chain rule based technique of automatic differentiation is shown to evaluate partial derivatives accurately and cheaply. In particular it is demonstrated that the reverse mode of automatic differentiation yields any gradient vector at no more than five times the cost of evaluating the underlying scalar function. After developing the basic mathematics we describe several software implementations and briefly discuss the ramifications for optimization.
A differential approach to inference in Bayesian networks
 Journal of the ACM
, 2000
"... We present a new approach to inference in Bayesian networks which is based on representing the network using a polynomial and then retrieving answers to probabilistic queries by evaluating and differentiating the polynomial. The network polynomial itself is exponential in size, but we show how it ca ..."
Abstract

Cited by 112 (18 self)
 Add to MetaCart
We present a new approach to inference in Bayesian networks which is based on representing the network using a polynomial and then retrieving answers to probabilistic queries by evaluating and differentiating the polynomial. The network polynomial itself is exponential in size, but we show how it can be computed efficiently using an arithmetic circuit that can be evaluated and differentiated in time and space linear in the circuit size. The proposed framework for inference subsumes one of the most influential methods for inference in Bayesian networks, known as the tree–clustering or jointree method, which provides a deeper understanding of this classical method and lifts its desirable characteristics to a much more general setting. We discuss some theoretical and practical implications of this subsumption. 1.
Path Problems in Graphs
 COMPUTING SUPPL
, 1989
"... A large variety of problems in computer science can be viewed from a common viewpoint as instances of "algebraic" path problems. Among them are of course path problems in graphs such as the shortest path problem or problems of finding optimal paths with respect to more generally defined objective ..."
Abstract

Cited by 27 (0 self)
 Add to MetaCart
A large variety of problems in computer science can be viewed from a common viewpoint as instances of "algebraic" path problems. Among them are of course path problems in graphs such as the shortest path problem or problems of finding optimal paths with respect to more generally defined objective functions; but also graph problems whose formulations do not directly involve the concept of a path, such as finding all bridges and articulation points of a graph; Moreover, there are even problems which seemingly have nothing to do with graphs, such as the solution of systems of linear equations, partial differentiation, or the determination of the regular expression describing the language accepted by a finite automaton. We describe the relation among these problems and their common algebraic foundation. We survey algorithms for solving them: vertex elimination algorithms such as GaußJordan elimination; and iterative algorithms such as the "classical" Jacobi and GaußSeidel iteration.
A Differential Semantics for Jointree Algorithms
"... Darwiche has recently proposed the representation of a belief network as a multivariate polynomial, allowing one to reduce probabilistic inference into a process of evaluating and dierentiating polynomials. ..."
Abstract

Cited by 14 (7 self)
 Add to MetaCart
Darwiche has recently proposed the representation of a belief network as a multivariate polynomial, allowing one to reduce probabilistic inference into a process of evaluating and dierentiating polynomials.
Automatic Differentiation And Spectral Projected Gradient Methods For Optimal Control Problems
, 1998
"... this paper is to show the application of these canonical formulas to optimal control processes being integrated by the RungeKutta family of numerical methods. There are many papers concerning numerical comparisions between automatic differentiation, finite differences and symbolic differentiation. ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
this paper is to show the application of these canonical formulas to optimal control processes being integrated by the RungeKutta family of numerical methods. There are many papers concerning numerical comparisions between automatic differentiation, finite differences and symbolic differentiation. See, for example, [1, 2, 6, 7, 21] among others. Another objective is to test the behavior of the spectral projected gradient methods introduced in [5]. These methods combine the classical projected gradient with two recently developed ingredients in optimization: (i) the nonmonotone line search schemes of Grippo, Lampariello and Lucidi ([24]), and (ii) the spectral steplength (introduced by Barzilai and Borwein ([3]) and analyzed by Raydan ([30, 31])). This choice of the steplength requires little computational work and greatly speeds up the convergence of gradient methods. The numerical experiments presented in [5], showing the high performance of these fast and easily implementable methods, motivate us to combine the spectral projected gradient methods with automatic differentiation. Both tools are used in this work for the development of codes for numerical solution of optimal control problems. In Section 2 of this paper, we apply the canonical formulas to the discrete version of the optimal control problem. In Section 3, we give a concise survey about spectral projected gradient algorithms. Section 4 presents some numerical experiments. Some final remarks are presented in Section 5. 2 CANONICAL FORMULAS The basic optimal control problem can be described as follows: Let a process governed by a system of ordinary differential equations be dx(t) dt = f(x(t); u(t); ); T 0 t T f ; (1) where x : [T 0 ; T f ] ! IR nx , u : [T 0 ; T f ] ! U ` IR nu , U compact, and 2 V ...
Practical quasiNewton methods for solving nonlinear systems
, 2000
"... Practical quasiNewton methods for solving nonlinear systems are surveyed. The definition of quasiNewton methods that includes Newton 's method as a particular case is adopted. However, especial emphasis is given to the methods that satisfy the secant equation at every iteration, which are call ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
Practical quasiNewton methods for solving nonlinear systems are surveyed. The definition of quasiNewton methods that includes Newton 's method as a particular case is adopted. However, especial emphasis is given to the methods that satisfy the secant equation at every iteration, which are called here, as usually, secant methods. The leastchange secant update (LCSU) theory is revisited and convergence results of methods that do not belong to the LCSU family are discussed. The family of methods reviewed in this survey includes Broyden 's methods, structured quasiNewton methods, methods with direct updates of factorizations, rowscaling methods and columnupdating methods. Some implementation features are commented. The survey includes a discussion on global convergence tools and linearsystem implementations of Broyden's methods. In the final section, practical and theoretical perspectives of this area are discussed. 1 Introduction In this survey we consider nonlinear ...
Algorithms for Solving Nonlinear Systems of Equations
, 1994
"... In this paper we survey numerical methods for solving nonlinear systems of equations F (x) = 0, where F : IR n ! IR n . We are especially interested in large problems. We describe modern implementations of the main local algorithms, as well as their globally convergent counterparts. 1. INTRODUC ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
In this paper we survey numerical methods for solving nonlinear systems of equations F (x) = 0, where F : IR n ! IR n . We are especially interested in large problems. We describe modern implementations of the main local algorithms, as well as their globally convergent counterparts. 1. INTRODUCTION Nonlinear systems of equations appear in many real  life problems. Mor'e [1989] has reported a collection of practical examples which include: Aircraft Stability problems, Inverse Elastic Rod problems, Equations of Radiative Transfer, Elliptic Boundary Value problems, etc.. We have also worked with Power Flow problems, Distribution of Water on a Pipeline, Discretization of Evolution problems using Implicit Schemes, Chemical Plant Equilibrium problems, and others. The scope of applications becomes even greater if we include the family of Nonlinear Programming problems, since the firstorder optimality conditions of these problems are nonlinear systems. Given F : IR n ! IR n ; F = (...
Probing a Set of Hyperplanes by Lines and Related Problems
, 1993
"... Suppose that for a set H of n unknown hyperplanes in the Euclidean ddimensional space, a line probe is available which reports the set of intersection points of a query line with the hyperplanes. Under this model, this paper investigates the complexity to find a generic line for H and further to ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Suppose that for a set H of n unknown hyperplanes in the Euclidean ddimensional space, a line probe is available which reports the set of intersection points of a query line with the hyperplanes. Under this model, this paper investigates the complexity to find a generic line for H and further to determine the hyperplanes in H . This problem arises in factoring the uresultant to solve systems of polynomials (e.g., Renegar [12]). We prove that d+1 line probes are sufficient to determine H . Algorithmically, the time complexity to find a generic line and reconstruct H from O(dn) probed points of intersection is important. It is shown that a generic line can be computed in O(dn log n) time after d line probes, and by an additional d line probes, all the hyperplanes in H are reconstructed in O(dn log n) time. This result can be extended to the ddimensional complex space. Also, concerning the factorization of the uresultant using the partial derivatives on a generic line, we touch upon reducing the time complexity to compute the partial derivatives of the uresultant represented as the determinant of a matrix.
GRADIENT: Algorithmic Differentiation in Maple
, 1993
"... Many scientific applications require computation of the derivatives of a function f : IR as well as the function values of f itself. All computer algebra systems can differentiate functions represented by formulae. But not all functions can be described by formulae. And formulae are not alway ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Many scientific applications require computation of the derivatives of a function f : IR as well as the function values of f itself. All computer algebra systems can differentiate functions represented by formulae. But not all functions can be described by formulae. And formulae are not always the most effective means for representing functions and derivatives.
Computation Of Exact Gradients In Distributed Dynamic Systems
 Optimization, Methods and Software
, 1998
"... A new and unified methodology for computing first order derivatives of functions obtained in complex multistep processes is developed on the basis of general expressions for differentiating a composite function. From these results, we derive the formulas for fast automatic differentiation of elem ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
A new and unified methodology for computing first order derivatives of functions obtained in complex multistep processes is developed on the basis of general expressions for differentiating a composite function. From these results, we derive the formulas for fast automatic differentiation of elementary functions, for gradients arising in optimal control problems, nonlinear programming and gradients arising in discretizations of processes governed by partial differential equations. In the proposed approach we start with a chosen discretization scheme for the state equation and derive the exact gradient expression. Thus a unique discretization scheme is automatically generated for the adjoint equation. For optimal control problems, the proposed computational formulas correspond to the integration of the adjoint system of equations that appears in Pontryagin's maximum principle. This technique appears to be very efficient, universal, and applicable to a wide variety of distributed controlled dynamic systems and to sensitivity analysis. Keywords: Fast automatic differentiation; optimal control problem; differentiation of elementary functions; rounding error estimation; parabolic system; hyperbolic system; adjoint equation; sensitivity analysis. 1