Using automatic differentiation to differentiate functions offers high precision and ease of use. However, due to not available source-code transformation tools in the past, automatic differentiation was not feasible for large scale GPU applications. With the newest version of Tapenade (a source-code transformation tool from INRIA), this is now getting addressed. In this project we aim to bring automatic differentiation to GPU-based applications (such as Machine-Learning and physics based simulations). We aim to use the latest mixed-precision capabilities of recent GPUs coupled with Mutli-GPU setups, to get performance boosts for these applications.