Flowlayoutpanel scrolling many controls with mouse

Question

I have a list that it's populated with usercontrols (~300 usercontrols // every UC has a PictureBox and 10 Labels). I display all of them in a Flowlayoutpanel but when I scroll down or up it is very slow and kinda unresponsive. I'm already using a DoubleBuffered Flowlayoutpanel and that 'trick' with SuspendLayout and ResumeLayout. What I'm trying to do: I scroll down with mouse -> when the first two usercontrols are not in the view anymore, remove them -> add two new usercontrols from the list at the bottom. And that should work also in the opposite way: scroll up with mouse -> when the last two are not in the view anymore, remove them -> add two new usercontrols from the list at the top. Picture

I checked and tried everything that I found but it's not working... Some links that I checked:

• Fake-scrolling containers with very many controls
• https://social.msdn.microsoft.com/Forums/en-US/bc881e4d-6770-44f0-85ce-51456920f938/adding-4000-child-controls-to-flowlayoutpanel-freezez-the-scrolling?forum=winforms
• Populating a FlowLayoutPanel with a large number of controls and painting thumbnails on demand
• How can i implement the paging effect in a FlowLayoutPanel control?
• Winforms: Scrollable FlowLayoutPanel with thousands of user controls - How to prevent memory leaks and dispose objects the right way?
• https://www.dreamincode.net/forums/topic/381141-flowlayoutpanel-paging-effect/

 private void FlowLayoutPanel_MouseMove(object sender, MouseEventArgs e)
        {
            // move flowlayoutpanel with mouse
            if (e.Button != MouseButtons.Left)
                return;

            Point pointDifference = new Point(Cursor.Position.X + mouseDownPoint.X, Cursor.Position.Y - mouseDownPoint.Y);
            if ((mouseDownPoint.X == Cursor.Position.X) && (mouseDownPoint.Y == Cursor.Position.Y))
                return;

            Point currAutoScroll = FlowLayoutPanel.AutoScrollPosition;
            FlowLayoutPanel.AutoScrollPosition = new Point(Math.Abs(currAutoScroll.X) - pointDifference.X, Math.Abs(currAutoScroll.Y) - pointDifference.Y);
            mouseDownPoint = Cursor.Position;

            
            //
            Point locationOnForm = UClist[0].FindForm().PointToClient(
            UClist[0].Parent.PointToScreen(new Point(UClist[0].Location.X,UClist[0].Location.Y+300)));
            if (locationOnForm.Y < 190)
            {
                FlowLayoutPanel.SuspendLayout();
                FlowLayoutPanel.Controls.Clear();
                FlowLayoutPanel.Controls.AddRange(UClist.Skip(2).Take(2).ToArray());
                FlowLayoutPanel.ResumeLayout();
            }   
        }

How should I do?

EDIT This is what I was talking 'bout