Abstract

It is shown how the techniques of automatic differentiation can be viewed in a broader context as an application of analysis on a nonarchimedean field. The rings used in automatic differentiation can be ordered in a natural way and form finite dimensional real algebras which contain infinitesimals. Some of these algebras can be extended to become a Cauchy-complete real-closed nonarchimedean field, which forms an infinite dimensional real vector space and is denoted by L. On this field, a calculus is developed. Rules of differentiation and certain fundamental theorems are discussed. A remarkable property of differentiation is that difference quotients with infinitely small differences yield the exact derivative up to an infinitely small error. This is of historical interest since it justifies the concept of derivatives as differential quotients. But it is also of practical relevance; it turns out that the algebraic operations used to compute derivatives in automatic differentiation are...

Citations