ModelNotFoundException while using Vertex AI online prediction

Question

I am attempting to deploy a finetuned GPT-2 model (pytorch) on Vetex AI. While importing the model, I did not set up any predict schemata. The model should accept tensors, and return either tensors or a string (not sure yet because I haven't gotten it to work).

Here is my code:

from typing import Dict, List, Union

from google.cloud import aiplatform
from google.protobuf import json_format
from google.protobuf.struct_pb2 import Value


def predict_custom_trained_model_sample(
    project: str,
    endpoint_id: str,
    instances: Union[Dict, List[Dict]],
    location: str = $REGION,
    api_endpoint: str = $API-ENDPOINT,
):
    """
    `instances` can be either single instance of type dict or a list
    of instances.
    """
    # The AI Platform services require regional API endpoints.
    client_options = {"api_endpoint": api_endpoint}
    # Initialize client that will be used to create and send requests.
    # This client only needs to be created once, and can be reused for multiple requests.
    client = aiplatform.gapic.PredictionServiceClient(client_options=client_options)
    # The format of each instance should conform to the deployed model's prediction input schema.
    instances = instances if isinstance(instances, list) else [instances]
    instances = [
        json_format.ParseDict(instance_dict, Value()) for instance_dict in instances
    ]
    parameters_dict = {}
    parameters = json_format.ParseDict(parameters_dict, Value())
    endpoint = client.endpoint_path(
        project=project, location=location, endpoint=endpoint_id
    )
    response = client.predict(
        endpoint=endpoint, instances=instances, parameters=parameters
    )
    print("response")
    print(" deployed_model_id:", response.deployed_model_id)
    # The predictions are a google.protobuf.Value representation of the model's predictions.
    predictions = response.predictions
    for prediction in predictions:
        print(" prediction:", dict(prediction))


# [END aiplatform_predict_custom_trained_model_sample]

predict_custom_trained_model_sample(
    project=$PROJECT,
    endpoint_id=$ENDPOINT_ID,
    location=$REGION,
    instances={"tensor": "[1024, 2059, 23, 4829]"}
)

This code can be found here. When I run my code though, I receive this error:

NotFound: 404 {
  "code": 404,
  "type": "ModelNotFoundException",
  "message": "Model not found: model"
}

I know I am using the correct values for project and endpoint, but I suspect somewhere along the way something got corrupted or something got weird and now things can't find stuff. Has anyone else run into this problem? Have I made some glaring error somewhere I will only realize after posting this question? Thank you all.

Answer 1

It's worth double checking that your model is deployed to the correct region.

I.e., if you uploaded your finetuned model to us-central1, you should make sure the endpoint you create is in that same region and that you're making your requests to the Vertex API endpoint for that region.