SwiftUI: NavigationLink and ScrollView drag gesture colliding

Question

I'm trying to implement a UICollectionView like View in SwiftUI which basically works fine. But when I scroll/drag inside the scroll view to scroll down, the tap is recognized on the NavigationLink and then navigating to the detail view, even if I just wanted to scroll down.

Any ideas what might be causing this? Additional info: The whole NavigationView is opened from a .sheet from another view (as you might notice in the screenshot). I tried adding a "manual" link by setting the tag property on the link and setting the tag within a TapGesture, but this doesn't work either.

Here is a short example where the error can be reproduced. Scrolling down will activate a tap on one of the white rectangles.

View before drag gesture on scroll view

View during drag gesture, tap is already recognized on NavigationLink

Navigation view pushes to detail view after drag gesture

    var body: some View {
        NavigationView {
            ScrollView {
                VStack {
                    ForEach(0..<6) { i in
                        NavigationLink(destination: Text("Detail")) {
                            Rectangle()
                                .background(Color.red)
                                .frame(width: 365, height: 100, alignment: .center)
                        }
                    }
                }
            }
            .navigationViewStyle(StackNavigationViewStyle())
            .navigationBarTitle("Items")
        }
    }