I have a vector valued function that is fixed in input and output size, but is called in a loop and I would like to calculate the gradient and hessian of all output entries.
In addition, each call slightly changes additional inputs to that function (that is an index is changed but I am not interested in the gradient wrt to that index and that input does not change the function in a meaningful way wrt gradient calculations).
This is what I am currently doing (as described here: https://juliadiff.org/ForwardDiff.jl/stable/user/advanced/#Hessian-of-a-vector-valued-function)
Outside Loop
jcfg = ForwardDiff.JacobianConfig(y->f(y,param,k),
x[1:LenIndex],
ForwardDiff.Chunk(zeros(LenIndex)))
hcfg = ForwardDiff.JacobianConfig(
z->ForwardDiff.jacobian(y->f(y,param,k),z),
x[1:LenIndex],
ForwardDiff.Chunk(zeros(LenIndex)))
Inside loop:
H = ForwardDiff.jacobian(y -> f(y,param,k),x[Index],jcfg,Val{false}())::Matrix{Float64}
HH = reshape(
ForwardDiff.jacobian(z ->
ForwardDiff.jacobian(y -> f(y,param,k), z),
x[Index],
hcfg,
Val{false}())::Matrix{Float64},
4,LenIndex,LenIndex)
This works and removing tag checks and using the configs noticably improves speed.
However, I assume that there is more to be gained by also using a config for the "inner" Jacobian in the Hessian. Of course, I cannot specify the same things because the dual number magic will be the input. I tried some things like zeros(ForwardDiff.Dual, LenIndex) but that did not work and I honestly have no idea what I am doing. I tried finding a guide on how to use the dual number type, but what I find confuses me more than it aids me, because suddenly there is something like Dual{S,V,M} and I have no clue what S and V should be or if I need to define tags and whatnot.
Anyways, how would one configure the inner jacobian? Can I disable tag checking in there or is that exactly the point where things go bananas? And can I use the DiffResults thing to get function, gradient and hessian in one swoop also here?