How do I train a Tensorflow model using Adam optimizer that decays learning rate during the trining with Tensorflow.JS! (not python) I cannot find that the library provides an exponential decay scheduler nor decay parameter in tf.train.adam(...) constructor:
/**
* Constructs a `tf.AdamOptimizer` that uses the Adam algorithm.
* See [https://arxiv.org/abs/1412.6980](https://arxiv.org/abs/1412.6980)
*
* @param learningRate The learning rate to use for the Adam gradient
* descent algorithm.
* @param beta1 The exponential decay rate for the 1st moment estimates.
* @param beta2 The exponential decay rate for the 2nd moment estimates.
* @param epsilon A small constant for numerical stability.
*
* @doc {heading: 'Training', subheading: 'Optimizers', namespace: 'train'}
*/
static adam(learningRate?: number, beta1?: number, beta2?: number, epsilon?: number): AdamOptimizer;
I can see tho that adamax optimizer has decay parameter but does it work the same way as adam does?
/**
* Constructs a `tf.AdamaxOptimizer` that uses the Adamax algorithm.
* See [https://arxiv.org/abs/1412.6980](https://arxiv.org/abs/1412.6980)
*
* @param learningRate The learning rate to use for the Adamax gradient
* descent algorithm.
* @param beta1 The exponential decay rate for the 1st moment estimates.
* @param beta2 The exponential decay rate for the 2nd moment estimates.
* @param epsilon A small constant for numerical stability.
* @param decay The learning rate decay over each update.
*
* @doc {heading: 'Training', subheading: 'Optimizers', namespace: 'train'}
*/
static adamax(learningRate?: number, beta1?: number, beta2?: number, epsilon?: number, decay?: number): AdamaxOptimizer;