iOS: didReceiveRemoteNotification is not being called when the app is in background

Question

Despite having "remote notification" in background modes, the app does not seem to do anything when the app is in the background. All I want to do is updating the badge number when the app is closed or terminated (like what Twitter app does).

func application( _ application: UIApplication, didReceiveRemoteNotification userInfo: [AnyHashable: Any], fetchCompletionHandler completionHandler: @escaping (UIBackgroundFetchResult) -> Void) {
    UIApplication.shared.applicationIconBadgeNumber = 99 // just to test
    completionHandler(.newData)
}

Background Mode settings:

When the app is in foreground it updates the badge, this is the payload:

{
  "aps": {
    "badge": 4,
    "content-available": 1,
    "mutable-content": 0
  }
}

But when it's in background it does nothing. Should it not update the badge?