Number of backprops as performance metric for neural networks

Question

I have been reading article about SRCNN and found that they are using "number of backprops" for evaluating how well network is performing, i.e. what network is able to learn after x backprops (as I understand). I would like to know what number of backprops actually means. Is this just the number of training data samples that there used during the training? Or maybe the number of mini-batches? Maybe it is one of the previous numbers multiplied by number of learnable parameters in the network? Or something completely different? Maybe there is some other more common name for this that I could loop up somewhere and read more about it because I was not able to find anything useful by searching "number of backprops" or "number of backpropagations"?

Bonus question: how widely this metric is used and how good is it?

Answer 1

I would assume that after it means how many the network has learned after back-propagating n times. Its more likely interchangeable with "after training over n samples..."

This maybe a bit different if they are using a recurrent network, as they could have more samples run in forward prop then in backwardprop. (For whatever reason I can't get the link to the paper to load, so unsure).

Based on your number of questions I think you might be overthinking this :)

Number of backprops is not a metric used commonly. Perhaps they use it here to showcase the speed of training based upon whatever optimization method's they are using. But for most common instances, it is not a relevant metric.

Answer 2

I read their Paper from 2016:

• author={C. Dong and C. C. Loy and K. He and X. Tang},
• journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
• title={Image Super-Resolution Using Deep Convolutional Networks},

Since they don't even mention batches I assume they are doing a backpropagation to update their weights after each sample / image.

In other words their batchsize (mini-batchsize) is equal to 1 sample.

So number of backpropagations means amount of batches after all, which is quite a common metric, viz. in the paper PSNR (loss) over amount of batches (or usually loss over epochs).

Bonus question: I come to the conclusion they just didn't stick to the common thesaurus of machine learning, or deep learning.

BonusBonus question: They use the metric of loss after n batches to showcase how much the different network architectures could learn on trainigdatasets with different size.