Google Batch and Instance reservation

Question

The created Job in Google Batch ignores the reserved capacities with GPUs, tries to create a new instance and gets a quota limit. How to make Google Batch take a reserved instance with GPU ?

1. I created a reservation using the command:

  gcloud compute reservations create "reservation-for-google-batch" \
      --project="my-project-name" \
      --zone="us-central1-f" \
      --vm-count=1 \
      --machine-type=n1-highcpu-8 \
      --require-specific-reservation \
      --accelerator=count=1,type=nvidia-tesla-t4

2. I am trying to submit a task to the Google Batch service: Command: gcloud batch jobs submit test-reservation-gpu --location us-central1 --config test.json

test.json

{
  .....
  "allocationPolicy": {
    "instances": [
      {
        "policy": {
          "machineType": "n1-highcpu-8",
          "accelerators": [
            {
              "type": "nvidia-tesla-t4",
              "count": 1
            }
          ],
          "reservation": "projects/my-project-name/reservations/reservation-for-google-batch"
        }
      }
    ],
    "location": {
      "allowedLocations": [
        "zones/us-central1-f"
      ]
    }
  },
  .....
}

3. The task is created, but a message is displayed in the events

Quota checking process decided to delay scheduling for the job test-reservation-g-... due to inadequate quotas [Quota: NVIDIA_T4_GPUS, limit: 1, usage: 1, wanted: 1.]

But a simple instance (separate from google batch) is successfully created

 gcloud compute instances create test-create-from-reservation \
    --project=my-project-name \
    --zone=us-central1-f \
    --machine-type=n1-highcpu-8 \
    --accelerator=count=1,type=nvidia-tesla-t4 \
    --reservation-affinity=specific \
    --reservation=projects/my-project-name/reservations/reservation-for-google-batch

Answer 1

This problem is known, as a solution is suggested to disable quota checking

https://cloud.google.com/batch/docs/known-issues#underestimated-reservations-quota

{
  "taskGroups": [
   .....
  ],
  "allocationPolicy": {
    ...
  },
  "labels": {
    "goog-batch-skip-quota-check": "true"
  },
  "logsPolicy": {
    ...
  }
}