How to scroll away SliverAppBar without a body

Question

What I'm trying to achieve:

Picture 1: View should be scrollable and the NoData container should fill up the remaining space.

• SliverFillRemaining just scrolls under the SliverAppBar which is not wanted - it should stop at the top which seems not to be possible.

Picture 2: You should be able to scroll away the SliverAppBar, Background, TabBar, etc and view the No Data in full like there never was a SliverAppBar there (at least the expanded portion of it).

Picture 3: Now there's ONE content - I still want the view to behave the same.

• Currently this just means the screens looks like Picture #1 with a Row inserted. Eh? Ugly.

Picture 4: Now there's many rows of content - the view should scroll away the SliverAppBar and continue with the scroll view.

But this is not what it looks like. If there's no "body" in the Widget the SliverAppBar just sits there. Adding a secondary scrolling view for the Rows just means it scrolls by itself while the SliverAppBar still just sits there. Using a SliverFillRemaining scrolls below the SliverAppBar even though there's just one row of content? That's not the remaining space? That's remaining space + AppBar - which is more than what's remaining.

In this case the SliverAppBar consists of this:

SliverAppBar(
    expandedHeight: 512,
    collapsedHeight: kToolbarHeight,
    toolbarHeight: kToolbarHeight,
    pinned: true,
    floating: false,
    snap: false,
    elevation: 0,
    backgroundColor: theme.primaryColor,
    leading: CloseButton(onPressed: onBackButton),
    title: Text("Title"),
    automaticallyImplyLeading: false,
    stretch: true,
    flexibleSpace: const FlexibleSpaceBar(
        background: getBackground(), // Background Image with some "Info" in it
    ),
    bottom: getBottom(), // TabBar
)

I've tried calculating the screen size remaining once the "SliverAppBar" is scrolled away but then there's the scrolling issue that it won't scroll both views at the same time. Regardless of primary: true/false, shrinkWrap: true/false. Also adding this "calculated" Container to a SliverFillRemaining is just plain ignored. I can set it to any size it still ignores the height set.

But the biggest question is - why would anyone want a SliverAppBar that's not always "scroll away-able" (for lack of a better term). If there's no content in the screen it just looks weird. I at least want the screen excluding the SliverAppBar to always take up the space of the screen with scrolling intact.

Is SliverListDelegate the only alternative? But Row, Row, Row in Picture #4 may not be the only scenario this applies to. In that case it doesn't feel correct to add a ListBuilder everytime you want to add a few Widgets. If I know there will be 4 widgets I'm not sure why I need a ListBuilder for that. But if I'm scrolling 200 items I agree.

In short, most solutions for this seems like hacks as of now.

Suggestions?