Curious difference in behavior between compiled and regular binding

Question

I'm trying to create a MenuFlyout with ToggleMenuFlyoutItems where one and only one toggle is checked at any given moment. The toggles corresponds to ToggleViewModels, binding the IsChecked property of the toggle to an IsSelected property of the ToggleViewModel. Because I want to uncheck the previously checked toggle whenever a new toggle is checked I relay the setting of the IsSelected property to the MainViewModel that holds the collection of ToggleViewModels.

Button with flyout defined in MainPage.xaml

<Button Content="Asdf">
    <Button.Flyout>
        <MenuFlyout>
            <ToggleMenuFlyoutItem 
                Text="{x:Bind _viewModel.ToggleCollection[0].Name}"
                IsChecked="{x:Bind _viewModel.ToggleCollection[0].IsSelected, Mode=TwoWay}" />
            <ToggleMenuFlyoutItem 
                Text="{x:Bind _viewModel.ToggleCollection[1].Name}"
                IsChecked="{x:Bind _viewModel.ToggleCollection[1].IsSelected, Mode=TwoWay}" />
            <ToggleMenuFlyoutItem 
                Text="{x:Bind _viewModel.ToggleCollection[2].Name}"
                IsChecked="{x:Bind _viewModel.ToggleCollection[2].IsSelected, Mode=TwoWay}" />
        </MenuFlyout>
    </Button.Flyout>
</Button>

MainPageViewModel:

public class MainViewModel : BindableBase
{
    public MainViewModel()
    {
        ToggleCollection = new ObservableCollection<ToggleViewModel>();

        var selectToggleAction = new Action<ToggleViewModel>(param => SetToggleSelection(param));

        for (int i = 0; i < 3; i++)
        {
            ToggleCollection.Add(new ToggleViewModel($"Item {i}", selectToggleAction));
        }
    }

    public ObservableCollection<ToggleViewModel> ToggleCollection { get; private set; }

    private void SetToggleSelection(ToggleViewModel toggle)
    {
        var selectedToggle = ToggleCollection.SingleOrDefault(t => t.IsSelected);
        if (selectedToggle != toggle)
        {
            selectedToggle?.SetSelection(false);
            toggle.SetSelection(true);
        }
    }
}

ToggleViewModel:

public class ToggleViewModel : BindableBase
{
    private Action<ToggleViewModel> _selectToggleAction;
    private bool _isSelected;

    public ToggleViewModel(string name, Action<ToggleViewModel> action)
    {
        Name = name;
        _selectToggleAction = action;
    }

    public string Name { get; set; }

    public bool IsSelected
    {
        get { return _isSelected; }
        set
        {
            if (_isSelected != value)
            {
                _selectToggleAction(this);
                base.OnPropertyChanged();
            }
        }
    }

    public void SetSelection(bool selected)
    {
        _isSelected = selected;
        base.OnPropertyChanged("IsSelected");
    }
}

Now all the code above works very well. The problem occurs when I try to use regular bindings instead of compiled ones:

<ToggleMenuFlyoutItem 
    Text="{Binding ToggleCollection[0].Name}"
    IsChecked="{Binding ToggleCollection[0].IsSelected, Mode=TwoWay}" />

Binding the properties like this I'm suddenly able to uncheck the currently checked toggle so that none is selected. This is due to the getter of the IsSelected property not being called when I raise the OnPropertyChanged in the setter of the IsSelected property (the reason for using regular bindings is that I want to create the toggles dynamically in code behind, but to illustrate the problem XAML works just as well).

Can anyone explain to me why the {x:Bind} in this case works but not the {Binding}?