MURAL - Maynooth University Research Archive Library



    DiffSharp: Automatic Differentiation Library


    Baydin, Atilim Gunes and Pearlmutter, Barak A. and Siskind, Jeffrey Mark (2015) DiffSharp: Automatic Differentiation Library. Working Paper. arXiv.

    [img]
    Preview
    Download (129kB) | Preview
    Official URL: https://arxiv.org/abs/1511.07727


    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...



    Add this article to your Mendeley library


    Abstract

    In this paper we introduce DiffSharp, an automatic differentiation (AD) library designed with machine learning in mind. AD is a family of techniques that evaluate derivatives at machine precision with only a small constant factor of overhead, by systematically applying the chain rule of calculus at the elementary operator level. DiffSharp aims to make an extensive array of AD techniques available, in convenient form, to the machine learning community. These including arbitrary nesting of forward/reverse AD operations, AD with linear algebra primitives, and a functional API that emphasizes the use of higher-order functions and composition. The library exposes this functionality through an API that provides gradients, Hessians, Jacobians, directional derivatives, and matrix-free Hessian- and Jacobian-vector products. Bearing the performance requirements of the latest machine learning techniques in mind, the underlying computations are run through a high-performance BLAS/LAPACK backend, using OpenBLAS by default. GPU support is currently being implemented.

    Item Type: Monograph (Working Paper)
    Additional Information: Cite as: arXiv:1511.07727 [cs.MS]
    Keywords: automatic differentiation; backpropagation; optimization; gradient methods;
    Academic Unit: Faculty of Science and Engineering > Computer Science
    Item ID: 8144
    Depositing User: Barak Pearlmutter
    Date Deposited: 10 Apr 2017 12:20
    Publisher: arXiv
    Funders: Science Foundation Ireland (SFI), US Army Research Laboratory
    URI:

    Repository Staff Only(login required)

    View Item Item control page

    Downloads

    Downloads per month over past year