How can I register a local model.mar to a running torchserve service?

Question

I have a running torchserve service. According to the docs, I can register a new model at port 8081 with the ManagementAPI. When running curl -X OPTIONS http://localhost:8081, the output also states for the post request on /models:

    ...
    "post": {
        "description": "Register a new model in TorchServe.",
        "operationId": "registerModel",
        "parameters": [
          {
            "in": "query",
            "name": "url",
            "description": "Model archive download url, support local file or HTTP(s) protocol. For S3, consider use pre-signed url.",
            "required": true,
            "schema": {
              "type": "string"
            }
          },
     ...

Now, I do have a local model.mar file and want to use it with the file protocol. I tried:

curl -X POST "http://localhost:8081/models?url=file:///path/to/model.mar"

But got:

{
  "code": 400,
  "type": "DownloadArchiveException",
  "message": "Failed to download archive from: file:///path/to/model.mar"
}

Is there anything that I am missing? How can I properly register a local model.mar to a running torchserve service?

Answer 1

I just figured it out. Everything I stated was completely correct and under normal circumstances, all of this would have worked.

The error is arising because I am running the torchserve instance in a docker container and the curl command is sent to this container which then looks in his local files for the model.mar. I thought for whatever reason that I can pass a file path from the machine that the docker container is running on.
Instead, I have to copy the file into the docker container first (or mount a directory) and then execute the registering command with the file path from the model.mar inside the docker container.