Efficient Differentiable Programming in a Functional Array-Processing Language
This work addresses the need for efficient differentiable programming in functional languages, which is incremental as it builds on existing automatic differentiation methods but with optimizations.
The authors tackled the problem of automatic differentiation in functional array-processing languages by developing a system that supports both source-to-source differentiation and global optimizations, resulting in performance that surpasses state-of-the-art tools on real-world machine learning and computer vision benchmarks.
We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations. Thanks to this feature, we demonstrate how for some real-world machine learning and computer vision benchmarks, the system outperforms the state-of-the-art automatic differentiation tools.