Why does my tensorboard plot have multiple lines?

Question

I'm only plotting a single sequence of 2d points. Can someone explain why my graph looks like this?

I understand there might be actual + smooth, but why are there are mess of like 5 different lines here?

Answer 1

weird connections

You see weird connections, because you restarted you training (or resumed) with the same name without setting the epoch (or step) to the correct resumed value. If your last datapoint is at step=10 you should continue saving data at step=11 not step=0. Otherwise the last value at step=1o will be conntect to a new value at step=0 which causes the observed connection. On first training with clean log files, this will not happen. Either choose a new name, or resume with proper step or epoch.

smoothing

It's not a bug, it's a feature! Often training progress fluctuates a lot, making it hard to observe a general trend. Smoothing the time series allows you to quickly understand where the training is going.

Look at this GAN Generator training. On the first image, without smoothing, its hard to tell what is happening.

Activating smoothing allows easier interpretation.

Answer 2

I think you are seeing both the actual values and the smoothed values. Drag the smoothing slider all the way down (it defaults to about 0.6).