GCP project creation via API doesn't enable Service Usage API

Question

I'm trying to automate the entire process of project creation using the official Google SDK for Node.js. For project creation, I use the Resource Manager SDK:

const resource = new Resource();
const project = resource.project(projectName);
const [, operation,] = await project.create();

I also have to enable some services in order to use them in the process. When I run:

const client = new ServiceUsageClient();
const [operation] = await client.batchEnableServices({
  parent: `projects/${projectId}`,
  serviceIds: [
    "apigateway.googleapis.com",
    "servicecontrol.googleapis.com",
    "servicemanagement.googleapis.com",
  ]
});

I receive:

Service Usage API has not been used in project 1014682171642 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=1014682171642 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

I find it suspicious that Service Usage API isn't enabled by default when I create a project via API. Obviously, it takes the benefit of using APIs if I had to enable something manually. When I create a project via Could Console, Service Usage API is enabled by default, so this issue affects only the API. Maybe there's some other way to enable Service Usage API programmatically.

I would appreciate any form of help.

Answer 1

As described in GCP docs:

When you create a Cloud project using the Cloud Console or Cloud SDK, the following APIs and services are enabled by default...

In your case, you're creating a project with a Client Library. The doc needs improvement as when it mentioned Cloud SDK, they actually meant the CLI tool, not the Client libraries.

To clarify, projects currently created with Client Libraries or with REST don't have any APIs enabled by default.

You can't call Service Usage to enable Service Usage on a project, as making the call would require Service Usage to already be enabled on the resource project.

My suggestion is to follow this flow:

1. Some process, using application project X (where Service Usage API is enabled), creates the new project Y.
2. The same process, using application project X, batch enables API services on project Y.

Or:

Automate the process of project creation on some sort of a bash script and create them using gcloud projects create commands.

Answer 2

I wrote a complete block of code, which worked for me. I apologize in advance if the code quality suffers (I probably butchered it) - literally do not know any nodejs - I compiled it from your code and couple of examples on the Internet.

const {Resource} = require('@google-cloud/resource-manager');
const {ServiceUsageClient} = require('@google-cloud/service-usage');

const projectId = '<YOUR PROJECT ID>';
const orgId = '<YOUR ORG ID>'; // I had to use org for my project

const resource = new Resource();
async function create_project() {
    await resource
      .createProject(`${projectId}`, {
        name: `${projectId}`,
        parent: { type: "organization", id: `${orgId}` }
      })
      .then(data => {
        const operation = data[1];
        return operation.promise();
      })
      .then(data => {
        console.log("Project created successfully!");
        enable_apis();
      });
}

const client = new ServiceUsageClient();
async function enable_apis() {
  const [operation] = await client.batchEnableServices({
    parent: `projects/${projectId}`,
    serviceIds: [
      "serviceusage.googleapis.com",
      "servicecontrol.googleapis.com",
      "servicemanagement.googleapis.com",
    ]
  })
}

create_project();

This successfully creates the project and enables three APIs. I would make sure the project is fully created before trying to enable apis (this is just a theory).

Regarding the link, you've mentioned earlier, I am going to speculate here, but I think what they meant by Cloud SDK is gcloud CLI tool, which is a part of Cloud SDK.

Answer 3

I wanted to accomplish the same thing, fully programmatic setup of a new project, including API enablement.

This is basically the only result online that kind of touches on this need, I think everyone just uses clickops or gcloud.

Well, I did a --log-http on gcloud while creating a project and found that it POSTS {} to: https://serviceusage.googleapis.com/v1/projects/<project-id>/services/cloudapis.googleapis.com:enable?alt=json

Even though this uses the serviceusage endpoint, I had service usage API off on this project. I guess maybe they allow the cloudapis.googleapis.com meta service to be enabled even when service usage is not yet enabled.

When I sent a post to that endpoint using curl for a fresh project made via API, it indeed enabled the basic API set, including the service usage API.

$ curl -X POST -H "Authorization: Bearer \"$(gcloud auth print-access-token)\"" -H "Content-Type: application/json; charset=utf-8" -d "{}" https://serviceusage.googleapis.com/v1/projects/<project-id>/services/cloudapis.googleapis.com:enable?alt=json

{
  "name": "operations/acat.p2-3333333333-5555fffe-88888-472c-b280-444444444444"
}

$ curl -H "Authorization: Bearer \"$(gcloud auth print-access-token)\"" https://serviceusage.googleapis.com/v1/operations/acat.p2-3333333333-5555fffe-88888-472c-b280-444444444444?alt=json

{
  "name": "operations/acat.p2-3333333333-5555fffe-88888-472c-b280-444444444444",
  "metadata": {
    "@type": "type.googleapis.com/google.protobuf.Empty"
  },
  "done": true,
  "response": {
    "@type": "type.googleapis.com/google.api.serviceusage.v1.EnableServiceResponse",
    "service": {
      "name": "projects/11111111111111/services/cloudapis.googleapis.com",
      "config": {
        "name": "cloudapis.googleapis.com",
        "title": "Google Cloud APIs",
        "documentation": {
          "summary": "This is a meta service for Google Cloud APIs for convenience. Enabling this service enables all commonly used Google Cloud APIs for the project. By default, it is enabled for all projects created through Google Cloud Console and Google Cloud SDK, and should be manually enabled for all other projects that intend to use Google Cloud APIs. Note: disabling this service has no effect on other services.\n"
        },
        "quota": {},
        "authentication": {},
        "usage": {
          "requirements": [
            "serviceusage.googleapis.com/tos/cloud"
          ]
        },
        "monitoring": {}
      },
      "state": "ENABLED",
      "parent": "projects/11111111111111"
    }
  }
}

I was also able to accomplish this using the nodejs @google-cloud/service-usage library:

const svcclient = new ServiceUsageClient()

async function enableServices(id) {
  const [op] = await svcclient.enableService({ name: `projects/${id}/services/cloudapis.googleapis.com` })
  const [resp] = await op.promise()
  console.log(`Enabled services on project.`)
}