Hierarchical algorithms for automatic differentiation.




Under a wide range of circumstances, the performance of computer programs generated in a black-box fashion by automatic differentiation (AD) often rivals or even exceeds that of numerical approaches based on divided differences. Yet, a little insight into the structure of the program to which automatic differentiation is applied may further increase the resulting AD code significantly. In the present work, the term ``structure'' is interpreted in the sense of a hierarchical approach to AD where a suitable layer of abstraction is determined by data dependence information or the semantics of the original program.