Results 1 
2 of
2
ReverseMode AD in a Functional Framework: Lambda the Ultimate Backpropagator
"... We show that reversemode AD (Automatic Differentiation)—a generalized gradientcalculation operator—can be incorporated as a firstclass function in an augmented lambda calculus, and therefore into a functionalprogramming language. Closure is achieved, in that the new operator can be applied to an ..."
Abstract

Cited by 4 (3 self)
 Add to MetaCart
We show that reversemode AD (Automatic Differentiation)—a generalized gradientcalculation operator—can be incorporated as a firstclass function in an augmented lambda calculus, and therefore into a functionalprogramming language. Closure is achieved, in that the new operator can be applied to any expression in the augmented language, yielding an expression in that language. This requires the resolution of two major technical issues: (a) how to transform nested lambda expressions, including those with freevariable references, and (b) how to support self application of the AD machinery. AD transformations preserve certain complexity properties, among them that the reverse phase of the reversemode AD transformation of a function have the same temporal complexity as the original untransformed function. Firstclass unrestricted AD operators increase the expressive power available to the numeric programmer, and may have significant practical implications for the construction of numeric software that is robust, modular, concise, correct, and efficient.
Nonstandard Interpretations of Probabilistic Programs for Efficient Inference
"... Probabilistic programming languages allow modelers to specify a stochastic process using syntax that resembles modern programming languages. Because the program is in machinereadable format, a variety of techniques from compiler design and program analysis can be used to examine the structure of th ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Probabilistic programming languages allow modelers to specify a stochastic process using syntax that resembles modern programming languages. Because the program is in machinereadable format, a variety of techniques from compiler design and program analysis can be used to examine the structure of the distribution represented by the probabilistic program. We show how nonstandard interpretations of probabilistic programs can be used to craft efficient inference algorithms: information about the structure of a distribution (such as gradients or dependencies) is generated as a monadlike side computation while executing the program. These interpretations can be easily coded using specialpurpose objects and operator overloading. We implement two examples of nonstandard interpretations in two different languages, and use them as building blocks to construct inference algorithms: automatic differentiation, which enables gradient based methods, and provenance tracking, which enables efficient construction of global proposals. 1