Nested ForEach + ScrollView Header causes Glitches

Question

Inside of a Scrollview I use a LazyVStack with a pinned header, and based on the scroll position I manipulate the scale of that header.

In the LazyVStack, I have a ForEach iterating over some list of items. However, if I use a nested ForEach loop (to have, say, items grouped together by month), the scrollview becomes extremely glitchy/jumpy.

Minimum reproducible code:

struct Nested: View {
    @State var yValueAtMinScale: Double = 100 // y value at which header is at minimum scale (used for linear interpolation)
    @State var headerScale = 1.0
    var restingScrollYValue = 240.0 // y value of scroll notifier when scrollview is at top
    
    var body: some View {
        ZStack {
            ScrollView {
                LazyVStack(pinnedViews: [.sectionHeaders]) {
                    Section(
                        header:
                            Circle()
                                .fill(Color.red)
                                .frame(width: 150, height: 150)
                                .scaleEffect(CGFloat(headerScale), anchor: .top)
                                .zIndex(-1)
                    ) {
                        
                        // scroll position notifier
                        // i set the header's scale based on this view's global y-coordinate
                        GeometryReader { geo -> AnyView in
                            let frame = geo.frame(in: .global)
                            print("miny", frame.minY)
                            DispatchQueue.main.async {
                                self.headerScale = calculateHeaderScale(frameY: frame.minY)
                            }
                            return AnyView(Rectangle()) // hack
                        }
                        .frame(height: 0) // zero height hack
                        
                        
                        ForEach(1...10, id: \.self) { j in
                            Section {
                                Text("\(j)")
                                // works without nested loop
                                ForEach(1...3, id: \.self) { i in
                                    Rectangle()
                                        .frame(height: 50)
                                        .padding(.horizontal)
                                        .id(UUID())
                                }
                            }
                        }
                    }
                }
            }
        }
    }
    
    // interpolates some linear value bounded between 0.75 and 1.5x, based on the scroll value
    func calculateHeaderScale(frameY: CGFloat) -> Double {
        let minScale = 0.75
        let linearValue = (1-minScale) * (Double(frameY) - yValueAtMinScale) / (restingScrollYValue - yValueAtMinScale) + minScale
        return max( 0.75, min(1.5, linearValue) )
    }
}

Removing the inner nested ForEach loop removes the problem. What could be going on here? I figured updating the header scale on every scroll update would be too many view updates and cause glitches, but that doesn't explain why it works with one ForEach loop.