Gmail API using service account is getting delegation denied error message

Question

I am developing a web-based application using Google API. I am using server-to-server which is obviously with a service account. I am almost done.

I can now insert new users through Google api. I can even create and/or give newly created users access to a Team Drive through Google API. Meaning, I have a service-account that can impersonate all domain users.

The Problem: But why am I getting errors when trying to impersonate a user through Gmail API? I need to add Gmail Labels and Filters to newly created users but I am getting an error message stating "Deligation denied". Am I missing something?

Obviously I have a working service-account settings already with proper scopes in G Suite. I guess userID "me" is not an option since my application needs to impersonate a newly created G Suite user. Please correct me if I am wrong.

$service = new \Google_Service_Gmail($this->client($scopes));

$serviceLabel = new \Google_Service_Gmail_Label();
$serviceLabel->setLabelListVisibility('labelShow');
$serviceLabel->setMessageListVisibility('show');
$serviceLabel->setName($label_name);

try {
    $resp = $service->users_labels->create($email, $serviceLabel);    
} catch (Google_Service_Exception $e) {
   return false;
}

Here is the error I am getting

{#398 ▼
  +"statusCode": 403
  +"errors": array:1 [▼
    0 => array:3 [▼
      "domain" => "global"
      "reason" => "forbidden"
      "message" => "Delegation denied for [email protected]"
    ]
  ]
  +"message": """
    {
     "error": {
      "errors": [
       {
        "domain": "global",
        "reason": "forbidden",
        "message": "Delegation denied for [email protected]"
       }
      ],
      "code": 403,
      "message": "Delegation denied for [email protected]"
     }
    }
    """
}

A Gmail Label is expected to be added automatically for each user being created through the application.