How to Create Google Cloud Storage Signed Urls on App Engine Python

Question

I couldn't find a simple example on how to implement Google Cloud Storage Signed Urls on Google App Engine with Python. Please write a step by step guide. :)

Answer 1

Here's how we made it work:

Step 1: Get p12 file/certificate

Download p12 file from https://console.developers.google.com/ “APIs & auth / Credentials” tab.

Step 2: Convert p12 file to DER format

Find a Linux computer open and connect using Terminal Command: openssl pkcs12 -in -nodes -nocerts > # The current Google password for the p12 file is notasecret

Command: openssl rsa -in -inform PEM -out -outform DER

Step 3: Convert DER file to base64 encoded string

Python console:

private_key = open(‘<filename.der>’, 'rb').read()
print private_key.encode('base64')

Copy and paste into App engine script.

Step 4: Enable PyCrypto in AppEngine

app.yaml must have a line to enable PyCrypto:

- name: pycrypto
  version: latest

Step 5: Python code to create Signed URL

import Crypto.Hash.SHA256 as SHA256
import Crypto.PublicKey.RSA as RSA
import Crypto.Signature.PKCS1_v1_5 as PKCS1_v1_5

der_key = “””<copy-paste-the-base64-converted-key>”””.decode('base64')

bucket = <your cloud storage bucket name (default is same as app id)>
filename = <path + filename>

valid_seconds = 5
expiration = int(time.time() + valid_seconds)

signature_string = 'GET\n\n\n%s\n' % expiration
signature_string += bucket + filename



# Sign the string with the RSA key.
signature = ''
try:
  start_key_time = datetime.datetime.utcnow()
  rsa_key = RSA.importKey(der_key, passphrase='notasecret')
  #objects['rsa_key'] = rsa_key.exportKey('PEM').encode('base64')
  signer = PKCS1_v1_5.new(rsa_key)
  signature_hash = SHA256.new(signature_string)
  signature_bytes = signer.sign(signature_hash)
  signature = signature_bytes.encode('base64')

  objects['sig'] = signature
except:
  objects['PEM_error'] = traceback.format_exc()

try:
  # Storage
  STORAGE_CLIENT_EMAIL = <Client Email from Credentials console: Service Account Email Address>
  STORAGE_API_ENDPOINT = 'https://storage.googleapis.com'

  # Set the query parameters.
  query_params = {'GoogleAccessId': STORAGE_CLIENT_EMAIL,
                'Expires': str(expiration),
                'Signature': signature}


  # This is the signed URL:
  download_href = STORAGE_API_ENDPOINT + bucket + filename + '?' + urllib.urlencode(query_params)

except:
  pass

Sources

How to get the p12 file.

Signing instructions.

Inspiration for how to sign the url.

Answer 2

I created this repo: https://github.com/voscausa/appengine-gcs-signed-url

Using GAE Pyrthon app_identity.get_service_account_name() and app_identity.sign_blob() makes creating signed url's very easy, without using a PEM key. The app shows how to download a GCS file.

But if you use the SDK to test the app, you have to use:

1. --appidentity_email_address
2. --appidentity_private_key_path

because creating a signed url is not part of the GCS client.

Answer 3

The other solutions work but there is a simpler way using generate_signed_url. This method does the same thing as @voscausa's answer but is less tedious and has custom exceptions and support for other environments.

def sign_url(obj, expires_after_seconds=60):

    client = storage.Client()
    default_bucket = '%s.appspot.com' % app_identity.get_application_id()
    bucket = client.get_bucket(default_bucket)
    blob = storage.Blob(obj, bucket)

    expiration_time = int(time.time() + expires_after_seconds)

    url = blob.generate_signed_url(expiration_time)

    return url

What vascausa said regarding local development server testing

But if you use the SDK to test the app, you have to use:

--appidentity_email_address

--appidentity_private_key_path

because creating a signed url is not part of the GCS client.

still holds.