Reverse Derivative Categories

  • It's rare to see a CS paper, let alone a CS paper with category theory, with 7 authors!

    It's fairly mathematical, so I'm going to quote the abstract for some context as to why you might care about this:

    The reverse derivative[0] is a fundamental operation in machine learning and automatic differentiation. This paper gives a direct axiomatization of a category with a reverse derivative operation, in a similar style to that given by Cartesian differential categories for a forward derivative. Intriguingly, a category with a reverse derivative also has a forward derivative, but the converse is not true. In fact, we show explicitly what a forward derivative is missing: a reverse derivative is equivalent to a forward derivative with a dagger structure[1] on its subcategory of linear maps. Furthermore, we show that these linear maps form an additively enriched category with dagger biproducts.

    [0] This is usually called the reverse mode of differentiation in the automatic differentiation literature: https://en.wikipedia.org/wiki/Automatic_differentiation#The_...

    [1] https://ncatlab.org/nlab/show/dagger+category