Google Vertex AI Prediction Request Failing

Question

I am trying to migrate a Google Cloud ML classification request from Language AutoML (now deprecated) to Vertex AI.

I've uploaded my dataset to VertexAI, trained, the model, and verified that predictions are working properly within the Cloud Console environment.

I've created an endpoint and it was successful and received an email saying

Hello Vertex AI Customer,

Vertex AI finished creating endpoint "{Endpoint Name}".
Additional Details:
Operation State: Succeeded
Resource Name: 
projects/{Project ID}/locations/us-central1/endpoints/{Endpoint ID}

Note: I'm using the same principal as I used for the AutoML predictions.

My old AutoML code used to look like this:

string authjson = @"
{
    ""type"": ""service_account"",
    ""project_id"": ""{projectname}"",
    ""private_key_id""
    ... etc..
}

GoogleCredential credential = GoogleCredential.FromJson(authjson).CreateScoped(PredictionServiceClient.DefaultScopes);

var client = new PredictionServiceClientBuilder { JsonCredentials = authjson }.Build();

var predictionrequest = new PredictRequest()
{
    ModelName = new ModelName("{Project ID}", "us-central1", "{Model ID}"),
    Payload = new ExamplePayload
    {
        TextSnippet = new TextSnippet
        {
            Content = content_input,
            MimeType = "text/plain"
        },
    },

};

var response = client.Predict(predictionrequest);

and it worked just fine. Now, with Vertex AI, I need to make the call to an endpoint, not the model. OK, my new VertexAI code (now using endpoints) looks like this

string projectID = "1234";
string endpointID = "5678";
string string_input_for_prediction = "blah blah blah";



string authjson = @"
{
    ""type"": ""service_account"",
    ... etc..
}

GoogleCredential credential = GoogleCredential.FromJson(authjson).CreateScoped(PredictionServiceClient.DefaultScopes);

PredictionServiceClientBuilder ClientBuilder = new PredictionServiceClientBuilder
{
    Credential = credential,
    Endpoint = "projects/" + projectID + "/locations/us-central1/endpoints/" + endpointID;
};

PredictionServiceClient client = ClientBuilder.Build();

var structVal = Google.Protobuf.WellKnownTypes.Value.ForStruct(new Struct
{
    Fields =
    {
    ["mimeType"] = Google.Protobuf.WellKnownTypes.Value.ForString("text/plain")
    ,
    ["content"] = Google.Protobuf.WellKnownTypes.Value.ForString(string_input_for_prediction)
    }
 });

PredictRequest predictionrequest = new PredictRequest()
{
    Instances = { structVal }
};

PredictResponse response = client.Predict(predictionrequest);

and getting the following error:

Grpc.Core.RpcException
  HResult=0x80131500
  Message=Status(StatusCode="Unavailable", Detail="Error connecting to subchannel.", DebugException="System.Net.Sockets.SocketException: No such host is known.")
  Source=Grpc.Net.Client

I've tried various flavors of the endpoint including "https://us-central1-aiplatform.googleapis.com/projects/{ProjectId}/....etc.. to no avail.

I'm sure this is a stupid error but I'm too stupid to see it.

Thanks for any help

Answer 1

I see that kind of error ("Error connecting to subchannel") mainly in GRPC, like "grpc/grpc-dotnet issue 2173".
That would point an incorrect or improperly formatted endpoint URL.

+----------------+      +---------------------+      +------------------+
| .NET Client    | ---> | Google Cloud Vertex | ---> | Vertex AI Model  |
| (Your Machine) |      | AI Endpoint         |      | (Cloud)          |
+----------------+      +---------------------+      +------------------+

Make sure the Endpoint in PredictionServiceClientBuilder is just the base URL of the API, not the full path to your specific endpoint.
See this example for illustration (in Java, but the same idea applies)

In the PredictRequest, include the full path to your specific endpoint, including the project ID, location, and endpoint ID. This assumes your environment has internet access to reach the Google Cloud services.

string projectID = "1234";
string endpointID = "5678";
string location = "us-central1"; // Location of the endpoint
string string_input_for_prediction = "foo bar";

string authjson = @"
{
    ""type"": ""service_account"",
    etc..
}"

GoogleCredential credential = GoogleCredential.FromJson(authjson).CreateScoped(PredictionServiceClient.DefaultScopes);

// Correct the endpoint format
string apiEndpoint = $"us-central1-aiplatform.googleapis.com";

PredictionServiceClientBuilder ClientBuilder = new PredictionServiceClientBuilder
{
    Credential = credential,
    Endpoint = apiEndpoint
};

PredictionServiceClient client = ClientBuilder.Build();

var structVal = Google.Protobuf.WellKnownTypes.Value.ForStruct(new Struct
{
    Fields =
    {
        ["mimeType"] = Google.Protobuf.WellKnownTypes.Value.ForString("text/plain"),
        ["content"] = Google.Protobuf.WellKnownTypes.Value.ForString(string_input_for_prediction)
    }
});

// Correct the endpoint in the PredictRequest
PredictRequest predictionrequest = new PredictRequest()
{
    Endpoint = $"projects/{projectID}/locations/{location}/endpoints/{endpointID}",
    Instances = { structVal }
};

PredictResponse response = client.Predict(predictionrequest);

---

The OP Mike Smith confirms in the comments:

The fact that I needed 2 endpoints (one for the PredictionServiceClientBuilder and another for the was the main problem here).

Exactly:

• API Endpoint for PredictionServiceClientBuilder
  That is the base URL of the Vertex AI Prediction API. In this case, it is something like us-central1-aiplatform.googleapis.com. That endpoint is used by the PredictionServiceClientBuilder to create a PredictionServiceClient instance.

• Specific Endpoint for PredictRequest
  That endpoint is the full path to the specific Vertex AI endpoint where the model is deployed. It includes details like project ID, location, and endpoint ID, formatted as projects/{projectID}/locations/{location}/endpoints/{endpointID}. That specific endpoint is used in the PredictRequest to direct the prediction request to the correct model endpoint.

The other thing I discovered is that you have to ADD the Vertex Service AI Agent role to the principal making the request. I only had AutoML Predictor role.

True: the principal (the identity making the request, typically a service account in this context) requires an appropriate role for accessing Vertex AI services.
Initially, the principal had the "AutoML Predictor" role, which was suitable for the now-deprecated AutoML services but not sufficient for Vertex AI.
You need the "Vertex AI Service Agent" role to the principal. That role provides the necessary permissions to access Vertex AI resources, including making prediction requests to deployed models.

Fixing those things made this all work.

So that involves both correctly configuring service endpoints and ensuring that the principal making the requests has the appropriate roles and permissions for the intended service.

Answer 2

Double-check the endpoint URL in your code for accuracy. Ensure it matches the URL provided in the Vertex AI console. Consider using environment variables to manage URLs for easier configuration. Verify internet connectivity and DNS resolution. If behind a firewall, allow necessary outbound connections to Google Cloud services. Use tools like ping or nslookup to test hostname resolution. Confirm the Vertex AI endpoint is running and deployed correctly in the console. Check for any errors or logs indicating issues with the endpoint itself.

Also, Ensure you're using the correct credentials and authentication method. Verify that your application has the necessary permissions to access the endpoint. I hope the above hints will help you to troubleshoot this error: