RIOS: Efficient I/O in reverse direction.

Abstract

The reverse mode of automatic differentiation executes the adjoint statements induced by each statement in the original program in the reverse order of the original program flow. This program flow reversal commonly requires storage of information on the control flow of the original program. In addition, intermediate values of variables that are overwritten have to be recorded, as these values may later be needed to compute the partial derivatives of the corresponding statement. The stored information will be accessed in reverse order of being written. This runs contrary to many assumptions made in standard implementations of file systems, operating systems, and inputoutput (IO) libraries. A common buffering strategy aimed at speeding up future read requests is to employ read-ahead. This strategy is useful for accesses in forward direction but is considered to be harmful to the performance of the reverse mode. To increase the performance of the reverse mode, it is also advantageous to interleave computations with the data storage and retrieval operations, which can be achieved using multithreading. To this end, we design and implement a novel software called reverse-mode IO stream (RIOS) that is adapted to these particular requirements of the reverse mode. We show the advantages of RIOS in two empirical case studies, an artificially constructed example of a typical IO pattern in the reverse mode and a real-world example arising from fluid mechanics, which is studied in Fortran90 and in Matlab where the reverse mode is generated via the automatic differentiation tools Tapenade and ADiMat, respectively.