protoc has trouble with input file

Question

I have this shell script:

#!/usr/bin/env bash

dir_name="$(cd "$(dirname "${BASH_SOURCE}")" && PWD)"

protoc --go_out="$dir_name" --go-grpc_out="$dir_name" "${dir_name}/myservice.proto"

I get this strange error message:

alex@alex yoda % /Users/alex/codes/yoda/yoda.ores.int/src/proto/generate.sh
/Users/alex/codes/yoda/yoda.ores.int/src/proto
/Users/alex/codes/yoda/yoda.ores.int/src/proto/myservice.proto: File does not reside within any path specified using --proto_path (or -I).  You must specify a --proto_path which encompasses this file.  Note that the proto_path must be an exact prefix of the .proto file names -- protoc is too dumb to figure out when two paths (e.g. absolute and relative) are equivalent (it's harder than you think).

here is the error more legibly:

File does not reside within any path specified using --proto_path (or -I). You must specify a --proto_path which encompasses this file. Note that the proto_path must be an exact prefix of the .proto file names -- protoc is too dumb to figure out when two paths (e.g. absolute and relative) are equivalent (it's harder than you think).

the input file(s), output files, and generate script are all in the same folder. Can someone help me figure out what I am doing wrong?

The single input file currently looks like this:

// myservice.proto

syntax = "proto3";

package myservice;

// Define your gRPC service.
service MyService {
  rpc SendMessage (MessageRequest) returns (MessageResponse);
}

// Define your request and response message types.
message MessageRequest {
  string message_data = 1;
  // Add more fields as needed
}

message MessageResponse {
  string status = 1;
  // Add more fields as needed
}

Answer 1

protoc is challenging with respect to import paths.

Because (!) you reference myservice.proto prefixed with ${dir_name}, you must include --proto_path=${dir_name} too:

protoc \
--proto_path="$dir_name" \
--go_out="$dir_name" \
--go-grpc_out="$dir_name" \
"${dir_name}/myservice.proto"

Your use of $dir_name is unclear to me; generally I'd use ${PWD}:

protoc \
--proto_path=${PWD} \
--go_out=${PWD} \
--go-grpc_out=${PWD} \
${PWD}/myservice.proto

If service.proto is in the ${PWD}, you can simplify this (but I prefer the more explicit version above) as:

protoc \
--go_out=${PWD} \
--go-grpc_out=${PWD} \
myservice.proto

Once you remove the ${PWD} (or equivalent) from myservice.proto, you no longer need --proto_path (in this case).

However (!) ...

myservice.proto is defined to be within a package, specifically package myservice. It's curious that it's not enforced but the convention is to represent protobuf (!) package hierarchies as folders (under proto_path roots), i.e.:

${PWD}
└── myservice
    └── myservice.proto

You can see this in Google's Well-Known Types usually distributed with protoc in the include folder:

.
└── google
    └── protobuf
        ├── any.proto
        ├── api.proto
        ├── compiler
        │   └── plugin.proto
        ├── descriptor.proto
        ├── duration.proto
        ├── empty.proto
        ├── field_mask.proto
        ├── source_context.proto
        ├── struct.proto
        ├── timestamp.proto
        ├── type.proto
        └── wrappers.proto

If pick any of these, e.g. timestamp.proto, you'll see that its package is google.protobuf and it's represented in the include folder as google/protobuf/timestamp.proto.

Using this convention, your command would be:

protoc \
--proto_path=${PWD} \
--go_out=${PWD} \
--go-grpc_out=${PWD} \
${PWD}/myservice/myservice.proto

And, lastly, Go is going to moan about being "unable to determine Go import path".

There are actually different ways to do this.

Commonly you'll see --go_opt=paths=source_relative but my preference is to use --go_opt=module=${MODULE} as follows:

NOTE You'll need to add both --go_opt and go_grpc_opt flags.

Assuming:

MODULE="github.com/foo/bar"

go mod init ${MODULE}

# This is the protobuf root folder
mkdir protos

And:

.
├── go.mod
└── protos
    └── myservice
        └── myservice.proto

Then update myservice.proto to include the value (!) of:

option go_package = "${MODULE}/protos";

And:

protoc \
--proto_path=${PWD}/protos \
--go_out=${PWD} \
--go_opt=module=${MODULE} \
--go-grpc_out=${PWD} \
--go-grpc_opt=module=${MODULE} \
${PWD}/protos/myservice/myservice.proto

Yielding:

.
├── go.mod
└── protos
    ├── myservice
    │   └── myservice.proto
    ├── myservice_grpc.pb.go
    └── myservice.pb.go