WPF filter Items Observable Collection without deleting views

Question

I've an ItemsControl and I'm binding the ItemsSource to an Observable Collection of ViewModels. This is how I'm binding to it...

<ItemsControl Grid.Row="1" ItemsSource="{Binding ObsvColl}" Background="Black">
<ItemsControl.ItemsPanel>
    <ItemsPanelTemplate>
        <Layout:CustomPanel />
    </ItemsPanelTemplate>
</ItemsControl.ItemsPanel>

<ItemsControl.ItemTemplate>
    <DataTemplate DataType="{x:Type VM:ItemViewModel}">
        <Border BorderThickness="3">
            <DMC:UserControl 
                Center="{Binding DataContext.Center, Source={x:Reference Name=SummaryView}}"
                ID="{Binding ID}" 
                ClipToBounds="True"/>
        </Border>
    </DataTemplate>
</ItemsControl.ItemTemplate>
        

The UserControl is sorta like an image control. When the user double clicks on a usercontrol, I'm removing all items from the observable collection except the one which was double clicked. This will make the clicked UserControl maximize. When the user double clicks again, I'm readding all the removed ViewModels to the observable collection.

Now when I remove ViewModels from the Observable collection, all the other usercontrols are deleted. When I'm adding the viewmodels again, I gets rerendered. Is there any way to not remove Views but hide them and display the selected usercontrol and bring them back when needed.

I tried collapsing the usercontrols but it just leaves empty spaces in place. Is there a way to do this?