How do I authenticate my Java application with Google Cloud Storage?

Question

I am writing a Java application to interact with files in Google Cloud Storage. I found gcloud-java which I'm trying to get working.

Looking at their examples it seems I should be able to simply run them after having logged-in with gcloud, but it doesn't seem to be working. I'm trying to run StorageExample, which says "logged-in project will be used if not supplied", but despite logging in I cannot access my project whether I specify it or not.

$ gcloud auth login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?....


Saved Application Default Credentials.

You are now logged in as [...].
Your current project is [...].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

$ java -cp GCloudDemo.jar com.google.gcloud.examples.storage.StorageExample list
Exception in thread "main" java.lang.IllegalArgumentException: A project ID is required for this service but could not be determined from the builder or the environment.  Please set a project ID using the builder.
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
        at com.google.gcloud.ServiceOptions.<init>(ServiceOptions.java:317)
        ...
        at com.google.gcloud.examples.storage.StorageExample.main(StorageExample.java:566)

$ java -cp GCloudDemo.jar com.google.gcloud.examples.storage.StorageExample ... list
Exception in thread "main" com.google.gcloud.storage.StorageException: Login Required
        at com.google.gcloud.spi.DefaultStorageRpc.translate(DefaultStorageRpc.java:94)
        at com.google.gcloud.spi.DefaultStorageRpc.list(DefaultStorageRpc.java:148)
        ...
        at com.google.gcloud.examples.storage.StorageExample$ListAction.run(StorageExample.java:1)
        at com.google.gcloud.examples.storage.StorageExample.main(StorageExample.java:579)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
{
  "code" : 401,
  "errors" : [ {
    "domain" : "global",
    "location" : "Authorization",
    "locationType" : "header",
    "message" : "Login Required",
    "reason" : "required"
  } ],
  "message" : "Login Required"
}
        at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:145)
        ...
        at com.google.gcloud.spi.DefaultStorageRpc.list(DefaultStorageRpc.java:138)
        ... 10 more

Clearly something is wrong with my environment, but what do I have to do to fix it? I'd like to correct my environment rather than make changes to the example.

Notes:

• I'm running in Cygwin 2.0.4 on Windows 10
• I'm not building via Maven, but I assume that isn't the source of the problem

Answer 1

The reason gcloud auth login isn't working is because Java doesn't play well with Cygwin, specifically with regards to absolute paths.

gcloud stores your settings and credentials in your Cygwin home directory, ~/.config/gcloud, but Java is going to look for them in your Windows home directory via System.getProperty("user.home"). You can specify where the application should look for this directory via the CLOUDSDK_CONFIG environment variable, like so (be sure to only specify it when calling java; setting it for your shell will in turn break gcloud):

$ CLOUDSDK_CONFIG=$(cygpath -w ~/.config/gcloud) \
  java -cp GCloudDemo.jar com.google.gcloud.examples.storage.StorageExample list 
Exception in thread "main" com.google.gcloud.storage.StorageException: Login Required
    at ....

Notice that that we didn't have to specify a project, but we're still not logged in for some reason. It turns out the demo app doesn't do a great job of reporting authentication errors. Adding an explicit call to AuthCredentials.createApplicationDefaults() at the start of the main() method reports a much more helpful error message:

Exception in thread "main" java.io.IOException: The Application Default Credentials are
not available. They are available if running in Google Compute Engine. Otherwise, the
environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a
file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials
for more information.

So now we at least have the GOOGLE_APPLICATION_CREDENTIALS environment variable to go on. But why isn't gcloud configuring things for us? It looks like that's actually a bug in the library; it should now be able to find our login token.

In the meantime, we can workaround it by explicitly specifying the credentials path as well:

$ CLOUDSDK_CONFIG=$(cygpath -w ~/.config/gcloud) \
  GOOGLE_APPLICATION_CREDENTIALS=$(cygpath -w ~/.config/gcloud)/application_default_credentials.json \
  java -cp GCloudDemo.jar com.google.gcloud.examples.storage.StorageExample list
Bucket{name=...}

Success!