I'm trying to implement a slightly different version of SGD with pytorch and test it on some datasets. I need to write a custom optimizer on which to train my model, however I cannot find any guide which explains how to do so. Is anyone able to tell me how I can proceed or suggest me any useful reference?
Thanks in advance.