Siskind, Jeffrey Mark and Pearlmutter, Barak A. (2016) Efficient Implementation of a Higher-Order Language with Built-In AD. In: AD 2016 Conference: 7th International Conference on Algorithmic Differentiation, September 12-15, 2016, Oxford U.K..

Preview

Download (287kB) | Preview

Abstract

We show that Automatic Differentiation (AD) operators can be provided in a dynamic language without sacrificing numeric performance. To achieve this, general forward and reverse AD functions are added to a simple high-level dynamic language, and support for them is included in an aggressive optimizing compiler. Novel technical mechanisms are discussed, which have the ability to migrate the AD transformations from run-time to compile-time. The resulting system, although only a research prototype, exhibits startlingly good performance. In fact, despite the potential inefficiencies entailed by support of a functional-programming language and a first-class AD operator, performance is competitive with the fastest available preprocessor-based Fortran AD systems. On benchmarks involving nested use of the AD operators, it can even dramatically exceed their performance.

Item Type:	Conference or Workshop Item (Paper)
Additional Information:	Extended abstract presented at the AD 2016 Conference, Sep 2016, Oxford UK. Cite as: arXiv:1611.03416 [cs.PL]
Keywords:	Higher-Order Language; Built-In AD; Automatic Differentiation (AD);
Academic Unit:	Faculty of Science and Engineering > Computer Science
Item ID:	8113
Depositing User:	Barak Pearlmutter
Date Deposited:	03 Apr 2017 15:41
Refereed:	Yes
Funders:	Science Foundation Ireland (SFI)
URI:	https://arxiv.org/abs/1611.03416
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads