The meaning of batch_size in ptb_word_lm (LSTM model of tensorflow)

Question

I am new to tensorflow, i am now a little confused about the meaning of batch_size. As commonly known that the meaning of batch_size is the number of samples for each batch, but according to the code in ptb_word_lm, it seems not:

reader.py:

data_len = tf.size(raw_data) #the number of words in dataset
batch_len = data_len // batch_size 

What does batch_len mean? The number of batches?

ptb_word_lm.py:

self.epoch_size = ((len(data) // batch_size) - 1) // num_steps

What does epoch_size mean? The number of sequences in each batch?

But if batch_size means the number of batches, then everything make sense. have I misunderstood something?

Answer 1

There are a few different concepts here: epoch, step, batch, and unroll steps for LSTM.

At the highest level, you train a network with multiple epochs. In each epoch, you will go through and use all training data (usually in a random order) by steps; In each step, you train a batch of samples.

I think the confusion here added by LSTM is that: each step, you will train a sequence of batches, instead of a single batch. The length of sequence is the number of unroll steps (num_steps).