CollectionView compositional layout with estimated height only shows _UICollectionViewOrthogonalScrollerEmbeddedScrollView

Question

I have a complex layout and most of the time it loads fine. After reloading it a few times though everything disappears (this does not always happen) and I only see an empty collectionView. If I inspect it I can see that there is only empty spacie and multiple _UICollectionViewOrthogonalScrollerEmbeddedScrollView (can see the layout inspector below).

The only warning in the console is related to item insets, but I'm not sure if it's even related since the insets won't apply on a compositional collectionView with vertical scrolling, but I would not think that it could cause other issues outside of that. The warning is still there even when the collectionView shows up fine.

When the issue happens the collectionView is set up correctly and even inspecting it you can see that the number of sections and rows are correctly set up. If I put a breakpoint where the cell gets populated though it does not get called.

An example of a section's layout:

func generateSideBySideAdaptiveLayout(isWide: Bool, height: CGFloat, withHeader header: Bool, numberOfItems: Int, withFooter footer: Bool = false, direction: UICollectionViewAdaptiveLayoutSectionDirection = .vertical) -> NSCollectionLayoutSection {

    let itemSize = NSCollectionLayoutSize(widthDimension: .fractionalWidth(1.0),
                                          heightDimension: .estimated(height))
    let item = NSCollectionLayoutItem(layoutSize: itemSize)

    // Show one item on narrow screens, two items on wider screens
    let groupFractionalWidth = isWide ? 0.5 : 1
    let groupHeight = isWide ? height : height * CGFloat(numberOfItems)
    let groupSize = NSCollectionLayoutSize(
        widthDimension: .fractionalWidth(CGFloat(groupFractionalWidth)),
        heightDimension: .estimated(CGFloat(groupHeight)))
    let group: NSCollectionLayoutGroup
    switch direction {
    case .vertical:
        group = isWide ? NSCollectionLayoutGroup.vertical(layoutSize: groupSize, subitem: item, count: numberOfItems / 2) : NSCollectionLayoutGroup.vertical(layoutSize: groupSize, subitem: item, count: numberOfItems)
    case .horizontal:
        group = isWide ? NSCollectionLayoutGroup.horizontal(layoutSize: groupSize, subitem: item, count: numberOfItems) : NSCollectionLayoutGroup.horizontal(layoutSize: groupSize, subitem: item, count: numberOfItems)
    }

    let section = NSCollectionLayoutSection(group: group)
    
    var supplementary: [NSCollectionLayoutBoundarySupplementaryItem] = []
    
    let headerSize = NSCollectionLayoutSize(widthDimension: .fractionalWidth(1.0),
                                            heightDimension: .estimated(44))
    if header {
        
        let sectionHeader = NSCollectionLayoutBoundarySupplementaryItem(
            layoutSize: headerSize,
            elementKind: SectionHeader.title.rawValue, alignment: .top)
        supplementary.append(sectionHeader)
    }
    if footer {
        let sectionFooter = NSCollectionLayoutBoundarySupplementaryItem(
            layoutSize: headerSize,
            elementKind: SectionFooter.actionButton.rawValue,
            alignment: .bottom)
        supplementary.append(sectionFooter)
    }
    section.boundarySupplementaryItems = supplementary
    section.orthogonalScrollingBehavior = .groupPaging

    return section
}

An example of set up code for the data:

var snapshot = NSDiffableDataSourceSnapshot<FeedCollectionSection, FeedCollectionItem>()
    guard sections.count > 0 else {
        return snapshot
    }
    
    snapshot.appendSections(sections)
    
    for each in sections {
        let items = getItemsForSection(each)
        snapshot.appendItems(items, toSection: each)
    }
    
    return snapshot