Dependency conflict when upgrading Beam version for Cloud Dataflow Template (Java SDK)

Question

I am running a customized Dataflow PubsubToBigQuery template. It is a Java SDK template.

Right now, I am trying to move from Beam version from 2.36.0 to 2.43.0.

My template uses a Cloud storage library which is declared in the pom file as:

<dependency>
      <groupId>com.google.apis</groupId>
      <artifactId>google-api-services-storage</artifactId>
      <version>v1-rev20181013-1.27.0</version>
</dependency>

I change the Beam version to 2.43.0 as mentioned and run the deploy command:

mvn compile exec:java ^
-Dexec.mainClass=com.google.cloud.teleport.templates.PubSubToBigQuery ^
-Dexec.cleanupDaemonThreads=false ^
-Dexec.args="[PRIVATE ARGS]"

The deploy task fails with the following error:

"You are currently running with version 2.0.0 of google-api-client. You need at least version 1.15 of google-api-client to run version 1.27.0 of the Cloud Storage JSON API library".

Detail: google-api-client is not explicit in my pom. Thus no "version 2.0.0" explicitly installed. Another question is that message seems totally bonkers, since if I have 2.0.0 and 1.15 at least is required, then I would be more recently than it?

I already tried to declare the google-api-client with a more recent version than 2.0.0, but then it breaks in one of my code lines.

Anyone have ideas on how to overcome this?

Answer 1

It was solved by adding this to the pom.

   <dependency>
    <groupId>com.google.api-client</groupId>
    <artifactId>google-api-client</artifactId>
    <version>1.33.2</version>
</dependency>