Android shared element transition produces image flicker with imageswitcher

Question

Activity A has a small ImageSwitcher that shows a stream with images and information about the current image.

When pressing the current image activity B opens with the image in fullscreen. You can continue to watch the image stream here.

Pressing the image here finishes this activity and goes back to A.

There is a image transition from A to B where the smaller image becomes bigger to fit the image in B.

When going back to A from B the image becomes smaller to fit in A.

The transitions works smoothly with just using

Intent intent = new Intent(this, BActivity.class);
ActivityOptionsCompat options = ActivityOptionsCompat.makeSceneTransitionAnimation(this, imageSwitcher,"imageInStream");
startActivity(intent, options.toBundle());

But if the image shown in activity B has changed to the next in the stream, then when going back to activity A the animation ends with a brief flicker of the previous image.

I have tried with delaying the transition with

protected void onResume() {
    super.onResume();
    postponeEnterTransition();
    // loading the image
    imageSwitcher.setImageDrawable(new BitmapDrawable(getResources(), imageBitmap));
    startPostponedEnterTransition();
}

I have also tried to put the startPostponedEnterTransition call in the post method of imageSwitcher, but the flickering persists.

So for some reason it seems like the imageSwitcher has not changed image yet, but I have no idea how to resolve this.