How to provide credentials for Spanner integration tests with Spring Boot, R2DBC and TestContainers (Spanner Emulator)

Question

Here is the error from the execution of the integration tests:

com.google.cloud.spanner.SpannerException: 
UNAUTHENTICATED: com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: 
UNAUTHENTICATED: Request is missing required authentication credential. 
Expected OAuth 2 access token, login cookie or other valid authentication credential. 
See https://developers.google.com/identity/sign-in/web/devconsole-project.

code (simplified for easier and faster reproducing):

@ExtendWith(SpringExtension.class)
@SpringBootTest
@ActiveProfiles("it")
@DirtiesContext
public class SpannerIT {

    static final String PROJECT_ID = "emulator-config";
    static final String INSTANCE_ID = "test-instance";
    static final String DATABASE_NAME = "test-database";

    static SpannerEmulatorContainer spannerContainer;

    @Autowired
    private R2dbcEntityTemplate template;

    @DynamicPropertySource
    static void properties(DynamicPropertyRegistry r) {
        r.add("spring.cloud.gcp.spanner.emulator-host", spannerContainer::getEmulatorGrpcEndpoint);
        r.add("spring.r2dbc.url", () -> "r2dbc:cloudspanner://" + spannerContainer.getEmulatorHttpEndpoint() +
            "/projects/" + PROJECT_ID + "/instances/" + INSTANCE_ID + "/databases/" + DATABASE_NAME);
    }

    @BeforeAll
    public static void beforeAll() {
        spannerContainer = new SpannerEmulatorContainer(
            DockerImageName.parse("gcr.io/cloud-spanner-emulator/emulator").withTag("1.4.1"));
        spannerContainer.start();
    }

    @AfterAll
    public static void afterAll() {
        spannerContainer.stop();
    }

    @Test
    void test() {
        StepVerifier.create(
                template.select(Query.query(CriteriaDefinition.empty()), SomeClazz.class)
            )
            .verifyComplete();

    }

    @Data
    @Table("test")
    public class SomeClazz {

        @Column("column")
        private String column;
    }
}

and the configs (application-it.yml):

spring:
  cloud:
    gcp:
      spanner:
        project-id: emulator-config

  r2dbc:
    url: overwritten_in_tests
    properties:
      usePlainText: true
      autoConfigEmulator: true

By setting the usePlainText to the r2dbc URL in theory we should bypass the credentials issue. The credentials provider is correctly configured to NoCredentials (SpannerConnectionFactoryProvider#extractCredentials).

Github repo with the complete code:
https://github.com/magiccrafter/spanner-spring-boot-r2jdbc-app

Answer 1

Here you can find a test container example of gcloud spanner:

https://github.com/saturnism/testcontainers-gcloud-examples/blob/main/springboot/spanner-example/src/test/java/com/example/springboot/spanner/SpannerIntegrationTests.java

and here:

https://www.testcontainers.org/modules/gcloud/#spanner

Answer 2

Thanks to elefeint's hint:

You would also have to set up an environment variable export SPANNER_EMULATOR_HOST=localhost:9010, so that the Spanner client library picks up the non-production host, similar to jdbc driver.

Upvote #200 for allowing programmatic customization of emulator.

for more details check the Github issue #200

Here is the workaround I came up with to bypass the requirement for SPANNER_EMULATOR_HOST env variable prior to the integration tests run:

@Testcontainers
@ExtendWith(SystemStubsExtension.class)
@ExtendWith(SpringExtension.class)
@SpringBootTest
@ActiveProfiles("it")
@DirtiesContext
public class SpannerIT {

    static final String PROJECT_ID = "nv-local";
    static final String INSTANCE_ID = "test-instance";
    static final String DATABASE_NAME = "trades";

    @Container
    private static final SpannerEmulatorContainer spannerContainer =
        new SpannerEmulatorContainer(
            DockerImageName.parse("gcr.io/cloud-spanner-emulator/emulator").withTag("1.4.1"));

    @SystemStub
    private static EnvironmentVariables environmentVariables;

    @Autowired
    ConnectionFactory connectionFactory;

    @DynamicPropertySource
    static void properties(DynamicPropertyRegistry r) {
        environmentVariables.set("SPANNER_EMULATOR_HOST", spannerContainer.getEmulatorGrpcEndpoint());
        r.add("spring.r2dbc.url", () -> "r2dbc:cloudspanner://" +
            "/projects/" + PROJECT_ID + "/instances/" + INSTANCE_ID + "/databases/" + DATABASE_NAME);
    }

This way, we can have the Spanner emulator in Testcontainers without starting the spanner emulator separately and mapping a concrete port. The complete source code can be found here.

Answer 3

The error you are getting is actually hinting that your credentials options are working, but that something is trying to connect to Cloud Spanner (or some other Cloud product) without credentials. The emulator would normally not return an Unauthenticated error, as it does not expect any credentials. Could you check:

1. That your tests are really trying to connect to the emulator endpoint in all cases, and not real Cloud Spanner.
2. That your tests are not accidentally trying to access some other Cloud product with NoCredentials.