Looking for Narrow Interfaces in Automatic Differentiation.

Abstract

Automatic differentiation is a powerful technique for evaluating derivatives of functions given in the form of a high-level programming language such as Fortran, C, or C++. This technique is superior, in terms of accuracy, to numerical differentiation because it avoids the truncation error involved in divided difference approximations. In automatic differentiation, the program is treated as a potentially very long composition of elementary functions to which the chain rule of differential calculus is applied over and over again. Because of the associativity of the chain rule, there is room for different strategies computing the same numerical results but whose computational cost may vary significantly. Several strategies exploiting high-level structure of the underlying computer code are known to reduce computational cost as opposed to blindly applying automatic differentiation. An example includes ``interface contraction'' where one takes advantage of the fact that the number of variables passed between subroutines is small compared with the number of propagated directional derivatives. Unfortunately, these so-called narrow interfaces are not immediately available. The present study investigates the use of the VCG graph drawing tool to recognize narrow interfaces in the computational graph, a certain directed acyclic graph used to represent data dependences of variables in the underlying computer code.