updates in mean-field variational inference

Question

I'm wondering if ELBO should be increasing monotonically as every update of the variational parameter q_i? I understand that it should increase at every iteration, but is this also true within one iteration for each q_i?

Answer 1

Yes, it is also true. Thinks it is a coordinate ascent method:

q_i^t = argmax_{q_i} \cal{L}(q_1^{t}, ...,q_{i-1}^{t}, q_i, q_{i+1}^{t-1}, ..., q_n^{t-1}). 

so you are always maximizing.