Changing ItemsPanelTemplate with VisualStateManager disables scrollbars

Question

I have a GridView with an ItemPanel like this:

<ItemsPanelTemplate x:Key="PanelTemplateLandscape">
    <WrapGrid Orientation="Vertical" />
</ItemsPanelTemplate>

Now if the app is changing the mode to "Snapped", I'd like to change the orientation of this WrapGrid. One way is to create a second ItemsPanelTemplate like this:

<ItemsPanelTemplate x:Key="PanelTemplatePortrait">
    <WrapGrid Orientation="Horizontal" />
</ItemsPanelTemplate>

And then use the VisualStateManager with the VisualState "Snapped" to change the ItemsPanel Template like this:

<ObjectAnimationUsingKeyFrames Storyboard.TargetName="DataGrid" Storyboard.TargetProperty="ItemsPanel">
    <DiscreteObjectKeyFrame KeyTime="0" Value="{StaticResource PanelTemplatePortrait}"/>
</ObjectAnimationUsingKeyFrames>

For completeness, here's the GridView markup:

<GridView x:Name="DataGrid" Grid.Row="1" ItemTemplate="{StaticResource Standard250x250ItemTemplate}" ItemsPanel="{StaticResource PanelTemplateLandscape}"
    ScrollViewer.HorizontalScrollBarVisibility="Auto" ScrollViewer.VerticalScrollBarVisibility="Auto"
    ScrollViewer.HorizontalScrollMode="Auto" ScrollViewer.VerticalScrollMode="Auto">
</GridView>

I then filled the GridView with 100 items and upon startup, it looks good and has a horizontal scrollbar which can be used.

Now if I "Snap" to the left, the orientation is changed and now there is a vertical scrollbar which can be used.

Everything as expected so far.

But if I now go back to FullscreenMode, the orientation is changed correctly, but the horizonzal scrollbar is visible but not usable. Same if I then switch back to the Snapped View, the scrollbar is visible but not usable.

Are there known problems with changing Templates on runtime? Are there ways around this issue? Is it wrong to change Templates with the VisualStateManager?

Thanks for your help