Compiled by Peter Sharpe
Lectures, Videos, GitHub repos, blog posts:
- YouTube: "What is Automatic Differentiation" - a "3Blue1Brown"-style video introducing AD
- Medium: "Automatic Differentiation Step by Step"
- GitHub: "Differentiation for Hackers" - includes runnable examples in Julia
- Lecture: "Automatic differentiation" - by Matthew J. Johnson, a huge contributor in the current AD landscape (developer of autograd, JAX).
- Lecture: "Intuition behind reverse mode algorithmic differentiation" - by Joris Gillis, one of the developers of CasADi (the AD library underneath AeroSandbox).
- Blog: "Reverse-mode automatic differentiation: a tutorial"
- Lecture: "Large-Scale Multidisciplinary Design Optimization of Aerospace Systems" - by Joaquim Martins. Section on AD starts at ~17:00.
Academic literature:
- Griewank and Walther, "Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation", 2008. Great comprehensive overview.
- "Automatic Differentiation in Machine Learning: a Survey" on ArXiV by Baydin et. al. is another great read.
- "Automatic differentiation of algorithms" by Bartholomew-Biggs, et. al.