Baydin, Atilim Gunes and Pearlmutter, Barak A. and Siskind, Jeffrey Mark (2015) DiffSharp: Automatic Differentiation Library. Working Paper. arXiv.

Preview

Download (129kB) | Preview

Abstract

In this paper we introduce DiffSharp, an automatic differentiation (AD) library designed with machine learning in mind. AD is a family of techniques that evaluate derivatives at machine precision with only a small constant factor of overhead, by systematically applying the chain rule of calculus at the elementary operator level. DiffSharp aims to make an extensive array of AD techniques available, in convenient form, to the machine learning community. These including arbitrary nesting of forward/reverse AD operations, AD with linear algebra primitives, and a functional API that emphasizes the use of higher-order functions and composition. The library exposes this functionality through an API that provides gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products. Bearing the performance requirements of the latest machine learning techniques in mind, the underlying computations are run through a high-performance BLAS/LAPACK backend, using OpenBLAS by default. GPU support is currently being implemented.

Item Type:	Monograph (Working Paper)
Additional Information:	Cite as: arXiv:1511.07727 [cs.MS]
Keywords:	automatic differentiation; backpropagation; optimization; gradient methods;
Academic Unit:	Faculty of Science and Engineering > Computer Science
Item ID:	8144
Depositing User:	Barak Pearlmutter
Date Deposited:	10 Apr 2017 12:20
Publisher:	arXiv
Funders:	Science Foundation Ireland (SFI), US Army Research Laboratory
URI:	https://arxiv.org
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads