Efficient Computation of Gradients and Jacobians by Dynamic Exploitation of Sparsity in Automatic Differentiation

Efficient Computation of Gradients and Jacobians by Dynamic Exploitation of Sparsity in Automatic Differentiation (1996)

Cached

Download Links

[info.mcs.anl.gov]

by Christian H. Bischof , Peyvand M. Khademi , Ali Bouaricha , Alan Carle

Citations:

6 - 1 self

Versions

Version History

Metadata Version 1

Datum	Value	Source
TITLE	y	SVM HeaderParse 0.1
AUTHOR NAME	Christian H. Bischof	SVM HeaderParse 0.1
AUTHOR NAME	Peyvand M. Khademi	SVM HeaderParse 0.1
AUTHOR AFFIL	y	SVM HeaderParse 0.2
AUTHOR NAME	Ali Bouaricha	SVM HeaderParse 0.1
AUTHOR AFFIL	y	SVM HeaderParse 0.2
AUTHOR ADDR	and	SVM HeaderParse 0.1
AUTHOR NAME	Alan Carle	SVM HeaderParse 0.1
AUTHOR AFFIL	z	SVM HeaderParse 0.2
AUTHOR ADDR	Argonne Preprint MCS-P519-0595	SVM HeaderParse 0.1
ABSTRACT	Abstract. Automatic differentiation (AD) is a technique that augments computer codes with statements for the computation of derivatives. The computational workhorse of AD-generated codes for first-order derivatives is the linear combination of vectors. For many large-scale problems, the vectors involved in this operation are inherently sparse. If the underlying function is a partially separable one (e.g., if its Hessian is sparse), many of the intermediate gradient vectors computed by AD will also be sparse, even though the final gradient is likely to be dense. For large Jacobians computations, every intermediate derivative vector is usually at least as sparse as the least sparse row of the final Jacobian. In this paper, we show that dynamic exploitation of the sparsity inherent in derivative computation can result in dramatic gains in runtime and memory savings. For a set of gradient problems exhibiting implicit sparsity, we report on the runtime and memory requirements of computing the gradients with the ADIFOR (Automatic DIfferentiation of FORtran) tool, both with and without employing the SparsLinC (Sparse Linear Combinations) library, and show that SparsLinC can reduce runtime and memory costs by orders of magnitude. We also compute sparse Jacobians using the SparsLinC-based approach---in the process, automatically detecting the sparsity structure of the Jacobian---and show that these Jacobian results compare favorably with those of previous techniques that require a priori knowledge of the sparsity structure of the Jacobian.	SVM HeaderParse 0.1
CITATIONS	27 found	ParsCit 1.0