Incorrect/incomplete python pb2 file generation with buf

40 Views Asked by At

I cannot figure out what is wrong with the protobuf Python code generation (I did check a couple of other similar questions)...let me share some details below.

I have the following folder structure:

├── ors-proto
│   ├── bin
│   │   ├── buf-Linux-x86_64
│   │   └── grpc_python_plugin
│   ├── buf.yaml
│   ├── config
│   │   └── assembly
│   │       └── proto.xml
│   ├── ors-proto.iml
│   ├── pom.xml
│   ├── src
│   │   └── main
│   │       └── proto
│   │           └── test
│   │               └── protobuf
│   │                   └── ors
│   │                       ├── common
│   │                       │   └── v1
│   │                       │       └── common_model.proto
│   │                       ├── ous
│   │                       │   └── v1
│   │                       │       └── ous_api.proto
│   │                       └── server
│   │                           └── v1
│   │                               ├── ors_api.proto
│   │                               └── ors_model.proto

The buf.yaml looks as follows:

version: v1beta1
build:
  roots:
    - ors-proto/src/main/proto
lint:
  use:
    - DEFAULT
  except:
    - FIELD_LOWER_SNAKE_CASE
    - PACKAGE_DIRECTORY_MATCH
    - RPC_REQUEST_STANDARD_NAME
    - RPC_RESPONSE_STANDARD_NAME
    - SERVICE_SUFFIX
  enum_zero_value_suffix: _UNKNOWN
breaking:
  use:
    - FILE

The common_model.proto is as follows:

syntax = "proto3";
package test.protobuf.ors.common.v1;

option java_package = "com.test.protobuf.ors.common.v1";
option java_multiple_files = true;
option java_outer_classname = "CommonModelProto";


message Field {
  string platform_field_name = 1;
  string field_value = 2;
  string default_value = 3;
  string platform_field_description = 4;
}


enum InternalField {
  INTERNAL_FIELD_UNKNOWN = 0;
  INTERNAL_FIELD_SYMBOL = 1;
}

The relevant pom.xml sections of the proto module look as follows:

......................................................................................
<properties>
        <protobuf-maven-plugin.version>0.6.1</protobuf-maven-plugin.version>
        <exec-maven-plugin.version>3.2.0</exec-maven-plugin.version>
        <os-maven-plugin.version>1.7.1</os-maven-plugin.version>

        <protobuf-sources.directory>src/main/proto</protobuf-sources.directory>
        <generated-sources.directory>${project.build.directory}/generated-sources</generated-sources.directory>
        <protoc-dependencies.directory>${project.build.directory}/protoc-dependencies</protoc-dependencies.directory>
        <protoc-dependencies-sources.directory>
            ${protoc-dependencies.directory}/com/google/protobuf/protobuf-java/${protobuf.version}/protobuf-java-${protobuf.version}.jar
        </protoc-dependencies-sources.directory>
        <protoc-plugins.directory>${project.build.directory}/protoc-plugins</protoc-plugins.directory>
        <working.directory>${project.basedir}</working.directory>
        <bin.directory>${project.basedir}/bin</bin.directory>
        <skip-on-deploy>false</skip-on-deploy>
    </properties>
................................................................................
<!-- Generate protobuf and gRPC code -->
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>${protobuf-maven-plugin.version}</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier}
                    </protocArtifact>
                    <protoSourceRoot>${protobuf-sources.directory}</protoSourceRoot>
                    <protocPluginDirectory>${protoc-plugins.directory}</protocPluginDirectory>
                    <temporaryProtoFileDirectory>${protoc-dependencies.directory}</temporaryProtoFileDirectory>
                    <hashDependentPaths>false</hashDependentPaths>
                    <clearOutputDirectory>false</clearOutputDirectory>
                </configuration>
                <executions>
                    <execution>
                        <id>Generate Java Protobuf sources</id>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${generated-sources.directory}/java</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>Generate Java gRPC sources</id>
                        <goals>
                            <goal>compile-custom</goal>
                        </goals>
                        <configuration>
                            <pluginId>grpc-java</pluginId>
                            <pluginArtifact>io.grpc:protoc-gen-grpc-java:${grpc.version}:exe:${os.detected.classifier}
                            </pluginArtifact>
                            <outputDirectory>${generated-sources.directory}/java</outputDirectory>
                        </configuration>
                    </execution>

                    <execution>
                        <id>Generate Python Protobuf sources</id>
                        <goals>
                            <goal>compile-python</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${generated-sources.directory}/python</outputDirectory>
                        </configuration>
                    </execution>
                    <execution>
                        <id>Generate Python gRPC sources</id>
                        <goals>
                            <goal>compile-custom</goal>
                        </goals>
                        <configuration>
                            <pluginId>grpc-python</pluginId>
                            <pluginExecutable>${bin.directory}/grpc_python_plugin</pluginExecutable>
                            <outputDirectory>${generated-sources.directory}/python</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

The result I get after running mvn clean install is below:

# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: test/protobuf/ors/common/v1/common_model.proto
# Protobuf Python Version: 4.25.3
"""Generated protocol buffer code."""
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import symbol_database as _symbol_database
from google.protobuf.internal import builder as _builder
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n>test/protobuf/ors/common/v1/common_model.proto\x12\"test.protobuf.ors.common.v1\"\x94\x01\n\x05\x46ield\x12+\n#platform_field_name\x18\x01 \x01(\t\x12\x13\n\x0b\x66ield_value\x18\x02 \x01(\t\x12\x15\n\rdefault_value\x18\x03 \x01(\t\x12\x32\n*platform_field_description\x18\x04 \x01(\t*F\n\rInternalField\x12\x1a\n\x16INTERNAL_FIELD_UNKNOWN\x10\x00\x12\x19\n\x15INTERNAL_FIELD_SYMBOL\x10\x01\x42\x44\n&com.test.protobuf.ors.common.v1B\x18CommonModelProtoP\x01\x62\x06proto3')

_globals = globals()
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals)
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'test.protobuf.ors.common.v1.common_model_pb2', _globals)
if _descriptor._USE_C_DESCRIPTORS == False:
  _globals['DESCRIPTOR']._options = None
  _globals['DESCRIPTOR']._serialized_options = b'\n&com.test.protobuf.ors.common.v1B\030CommonModelProtoP\001'
  _globals['_INTERNALFIELD']._serialized_start=253
  _globals['_INTERNALFIELD']._serialized_end=323
  _globals['_FIELD']._serialized_start=103
  _globals['_FIELD']._serialized_end=251
# @@protoc_insertion_point(module_scope)

...the pb2 file does not contain the actual message definitions.

Could someone please advise on what might be wrong in my setup? (I have to mention that I see no issues with the generated Java files).

Thank you in advance.

0

There are 0 best solutions below