Mat Cards not rearranging to same line after ngIf filter

Question

I'm trying to understand what part of my code is causing the mat-cards I have on my dashboard to not fill the same row after the toggle button is clicked to fulfill the *ngIf condition. I used the inspect tool on the browser and I think the issue is that it still thinks there's a card there when there really isn't. Firing off another API call to update the cards doesn't sound right so i'm guessing using the *ngIf was the right method but I can't seem to understand why it doesn't actually remove and display the cards I want on the same line. It might also have something to do with the Flex layout responsiveness? Using an ng-container seemed to resolve the issue of the cards being on different rows but it removed any of the Flex layout responsiveness that I need to keep the dashboard neat. I'm guessing there's something i'm just not doing right here, would appreciate some help!

This is the code for the container of the mat-card

<div style="padding-left: 60px; padding-top: 16px;" fxLayout="row wrap" fxLayoutGap="8px">
        <ng-container *ngFor="let pvsTrip of pvsTrips; let i = index" >
          <div *ngIf="(milesToggle && pvsMilesVariance.includes(pvsTrip.scheduleNum)) || !milesToggle" fxFlex.gt-md="32%" fxFlex.lt-sm="100%" fxFlex.md="48%"
            fxLayout.lt-sm="column">
              <div *ngIf = "(cToggle && !ncToggle && !neToggle && compliance.includes(pvsTrip.scheduleNum)) || (!cToggle && ncToggle && !neToggle && compliance.includes(pvsTrip.scheduleNum)) || (!cToggle && !ncToggle && neToggle && compliance.includes(pvsTrip.scheduleNum)) || (!cToggle && !ncToggle && !neToggle)">
                    <mat-card> 
                    </mat-card>

Apologies in advance for the messy screenshots. I have data on there that I can't expose.