In a Neural Network, should bias have a momentum term?

1.3k Views Asked by At

Should the momentum be added also to the bias term of every node in the network or preferably only on weights?

1

There are 1 best solutions below

1
On

Bias and weights. If you just applied it to the weights, the bias would lag the weights, artificially increasing the error and slowing convergence.

Think of bias as simply one more weight -- an extra input that's always 1.