So I trained a Bayesian U-Net with p = 0.5 during training stages and turned off dropout during inference. Shouldn't this essentially behave like a normal neural network? For some reason I am getting a major dip in accuracy when I do this. The same network with MC dropout at inference is performing well. Any reason why this is happening?
I tried testing with no dropout and validation score dropped significantly but I expected it to behave like a normal network that essentially cannot quantify uncertainty