A Common Interface for Automatic Differentiation
This work addresses the problem of AD system integration for researchers and developers in scientific machine learning, offering a modular solution but is incremental as it builds on existing backends.
The paper tackles the challenge of selecting and using Automatic Differentiation (AD) systems for scientific machine learning by introducing DifferentiationInterface.jl, a Julia package that provides a common frontend to a dozen AD backends, enabling easy comparison and modular development with features like sparsity handling.
For scientific machine learning tasks with a lot of custom code, picking the right Automatic Differentiation (AD) system matters. Our Julia package DifferentiationInterface$.$jl provides a common frontend to a dozen AD backends, unlocking easy comparison and modular development. In particular, its built-in preparation mechanism leverages the strengths of each backend by amortizing one-time computations. This is key to enabling sophisticated features like sparsity handling without putting additional burdens on the user.