Microsoft CNTK Automatic Differentiation

131 Views Asked by At

According to Microsoft, CNTK includes automatic differentiation. For better understanding the source (which I've successfully built) I'd like to know which C++ classes implement AD and how it is implemented in CNTK?

1

There are 1 best solutions below

0
On

CNTK class Function implements the AD (via Gradients method, to be precise). Neural networks are represented as multiple Function compositions like g(f(x)). Then derivative of function g is computed with respect to f like this:

                                                                     derivative