Flutter: Update scrollView physic without rebuild

Question

I have created a custom SliverPersistentHeader that expands by stretching (over scroll) the scrollview to cover entire screen.

Red border is device screen.

SliverPersitentDelegate observes the scroll position and if it reached the trigger offset, it ask the _RenderFleixbleHeader to open the map. _RenderFleixbleHeader is a subclass of RenderSliverPinnedPersistentHeader and is CustomSliverPersistentHeader's RenderSliver. it uses an animation controller and in each tick it calls markNeedsLayout to performLayout for new height.

the problem is when I drag over header, the scrollView still drag. I want to lock the scrollview and only by tapping on the close button reset the layout.

so as you see I don't call setState to change widget configuration. so I can not rebuild the CustomScrollView widget which container slivers with new NeverScrollableScrollPhysics physic to prevent scrolling.

in SliverPersitentDelegate I have access to Scrollable ScrollPosition and ScrollController.

I'm wondering is it possible to inject new physic through ScrollController?

I used this but no success:

final scrollController = Scrollable.of(context)?.widget.controller;
scrollController.detach(_oldPosition);
final newPosition = scrollController.createScrollPosition(
          NeverScrollableScrollPhysics(), Scrollable.of(context), _oldPosition);
scrollController.attach(newPosition);

any idea?