Putting the Automatic Back into AD: Part II, Dynamic, Automatic, Nestable, and Fast (CVS: 1.1)

This paper discusses a new AD system that correctly and automatically accepts nested and dynamic use of the AD operators, without any manual intervention. The system is based on a new formulation of AD as highly generalized first-class citizens in a ń-calculus, which is briefly described. Because the ń-calculus is the basis for modern programminglanguage implementation techniques, integration of AD into the ń-calculus allows AD to be integrated into an aggressive compiler. We exhibit a research compiler which does this integration, and uses some novel analysis techniques to accept code involving free dynamic use of nested AD operators, yet performs as well as or better than the most aggressive existing AD systems