Algorithmic Differentiation of Linear Algebra Functions with Application in Optimum Experimental Design (Extended Version)
For practitioners needing exact derivatives of linear algebra functions in optimization or sensitivity analysis, this provides a practical, accurate implementation in ALGOPY.
This paper derives algorithms for higher-order derivative computation of QR decomposition and eigenvalue decomposition using algorithmic differentiation, achieving near machine-precision accuracy. The methods are applied to compute gradients for optimum experimental design objectives.
We derive algorithms for higher order derivative computation of the rectangular $QR$ and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of algorithmic differentiation (AD) using univariate Taylor propagation of matrices (UTPM). Linear algebra functions are regarded as elementary functions and not as algorithms. The presented algorithms are implemented in the BSD licensed AD tool \texttt{ALGOPY}. Numerical tests show that the UTPM algorithms derived in this paper produce results close to machine precision accuracy. The theory developed in this paper is applied to compute the gradient of an objective function motivated from optimum experimental design: $\nabla_x Φ(C(J(F(x,y))))$, where $Φ= \{λ_1 : λ_1 C\}$, $C = (J^T J)^{-1}$, $J = \frac{\dd F}{\dd y}$ and $F = F(x,y)$.