Reverse Faà di Bruno's Formula for Cartesian Reverse Differential Categories
This work provides a foundational mathematical framework for higher-order reverse differentiation in automatic differentiation, which is incremental as it extends existing categorical axioms.
The paper tackles the problem of expressing higher-order reverse derivatives for composition in categorical automatic differentiation by deriving a reverse differential analogue of Faà di Bruno's Formula, establishing a higher-order reverse chain rule in Cartesian reverse differential categories.
Reverse differentiation is an essential operation for automatic differentiation. Cartesian reverse differential categories axiomatize reverse differentiation in a categorical framework, where one of the primary axioms is the reverse chain rule, which is the formula that expresses the reverse derivative of a composition. Here, we present the reverse differential analogue of Faa di Bruno's Formula, which gives a higher-order reverse chain rule in a Cartesian reverse differential category. To properly do so, we also define partial reverse derivatives and higher-order reverse derivatives in a Cartesian reverse differential category.