Google Cloud Storage generate upload/download URL fails with unicode characters in blobname

Question

I have been testing uploading & downloading on GCS lately, I was able to generate upload & download URLs without problems as long as the blobname or "path" was all English letters including spaces and special characters. However once I use unicode characters in the file name (blobname) it fails with KeyError

The docs mentions that unicode is supported:

The name of the blob. This corresponds to the unique path of the object in the bucket. If bytes, will be converted to a unicode object. Blob / object names can contain any sequence of valid unicode characters, of length 1-1024 bytes when UTF-8 encoded.

Code is a few lines:

generate_download_signed_url_v4("storage_files", "202209/جراج.png")

def generate_download_signed_url_v4(bucket_name, blob_name):
    storage_client = storage.Client.from_service_account_json('s_account.json')
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(blob_name)
    
    url = blob.generate_signed_url(
        expiration=datetime.timedelta(minutes=10), # This URL is valid for 10 minutes
        method="GET", # Allow GET requests using this URL.
    )
    
    print "Generated GET signed URL:", url
    return url

Error is referring to a keyerror along with the unicode value of the first character in the unicode file name portion (blobname)

  Traceback (most recent call last):
  File "/Users/k/Documents/app30/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/Users/k/Documents/app30/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/Users/k/Documents/app30/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/Users/k/Documents/app30/gae/users_app/helpers.py", line 68, in dispatch
    webapp2.RequestHandler.dispatch(self) # dispatch the main handler
  File "/Users/k/Documents/app30/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/Users/k/Documents/app30/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/Users/k/Documents/app30/gae/users_app/helpers.py", line 431, in check_admin
    return myfunction(self, *args, **kwargs)
  File "/Users/k/Documents/app30/gae/users_app/main.py", line 10796, in get
    my_file_link = self.generate_download_signed_url_v4(root_bucket_name, "572680296//202209/جراج.png")
  File "/Users/k/Documents/app30/gae/users_app/main.py", line 10863, in generate_download_signed_url_v4
    method="GET", # Allow GET requests using this URL.
  File "/Users/k/Documents/app30/gae/users_app/lib/google/cloud/storage/blob.py", line 338, in generate_signed_url
    quoted_name=quote(self.name))
  File "/Users/k/Documents/app30/google-cloud-sdk/platform/google_appengine/google/appengine/dist27/urllib.py", line 1294, in quote
    return ''.join(map(quoter, s))
KeyError: u'\u062c'
*******> Exception: u'\u062c'

Worth mentioning that this is on the local dev server v.340

Answer 1

This issue was fixed in 2018. Looking at the line number and code in

 File "/Users/k/Documents/app30/gae/users_app/lib/google/cloud/storage/blob.py", line 338, in generate_signed_url
    quoted_name=quote(self.name))

confirms are using a version from before that fix (they match this which was changed by this commit). Upgrade to a version of the library later than 1.11.0.