Abstract

In the basic form of the reverse mode for calculating derivatives, the amount of memory needed to record the intermediate values can become excessively large for problems of practical interest. If sequential checkpointing schemes are used, the memory requirement can be dramatically reduced, but the run time may be significantly increased. Implementing suitable checkpointing schemes on multiprocessor systems can decrease the run time to its theoretical minimum. Among the many possible scheduling strategies, we develop one that minimizes resource requirements. We present different communication structures that depend on the memory architecture of the multiprocessor system and the available resources. We also estimate the limits of the complexity and the memory requirements of the problem function. 1 Introduction Our starting point is a problem P (f; rf; g; (b1; : : : ; b n )) whose solution requires, in addition to the problem function f : R ¯ ! R , its derivative rf . It is assum...

Citations

137	ADOL-C, a package for the automatic differentiation of algorithms written - Griewank, Juedes, et al. - 1996
100	Achieving logarithmic growth of temporal and spatial complexity in reverse automatic differentiation - Griewank - 1992
83	Vaidy Sunderam. PVM: Parallel Virtual Machine – A Users’ Guide and Tutorial for Networked Parallel Computing - Geist, Beguelin, et al. - 1994