How to use gcs-resumable-upload with signed url

Question

I want to be able to do resumable uploads to Google Cloud Storage in a node.js client application using the gcs-resumable-upload package, in conjunction with signed urls (since the client app is invoked by unauthenticated users).

My server generates a signed url by calling getSignedUrl with {action: 'resumable'}. The server then sends a POST to the signed url with header { 'x-goog-resumable': 'start' } and an empty body, and receives a response with a location header that looks something like the following:

https://storage.googleapis.com/<bucket_name/<file_path>?GoogleAccessId=<service_account>&Expires=<expiry_time>&Signature=<signature>&upload_id=<upload_id>

My question is: If I return the above location header to my client, can the client use it to perform a resumable upload using gcs-resumable-upload, and if so, how exactly? If anyone has an example, that would be greatly appreciated!

Answer 1

If you are using java then.

According to the google doc here It is clear how to create a signed URL to upload the object to a bucket.

But you need to add an extra header for resumable upload ("x-goog-resumable: start"). which is not mentioned in the doc.

import com.google.auth.oauth2.GoogleCredentials;
import com.google.auth.oauth2.ServiceAccountCredentials;
import com.google.cloud.storage.*;

import java.io.*;
import java.net.URL;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;

public class GenerateV4PutObjectSignedUrl {

    public static void generateV4GPutObjectSignedUrl(
            String projectId, String bucketName, String objectName) throws StorageException, IOException {
        File initialFile = new File("src/main/java/credentials.json");
        InputStream serviceAccountJson = new FileInputStream(initialFile);

        ServiceAccountCredentials credentials = (ServiceAccountCredentials)
                GoogleCredentials.fromStream(serviceAccountJson);
        Storage storage = StorageOptions.newBuilder().setProjectId(projectId).setCredentials(credentials).build().getService();

        // Define Resource
        BlobInfo blobInfo = BlobInfo.newBuilder(BlobId.of(bucketName, objectName)).build();

        // Generate Signed URL
        Map<String, String> header = new HashMap<>();
        header.put("x-goog-resumable", "start");

        URL url =
                storage.signUrl(
                        blobInfo,
                        15,
                        TimeUnit.MINUTES,
                        Storage.SignUrlOption.httpMethod(HttpMethod.POST),
                        Storage.SignUrlOption.withExtHeaders(header),
                        Storage.SignUrlOption.withV4Signature());


        System.out.println("Generated PUT signed URL:");
        System.out.println(url);
    }

    public static void main(String[] args) throws IOException {
        generateV4GPutObjectSignedUrl("projectId", "bucketName", "objectName");
    }
}

Use the generated URL to make a POST call. I used cURL for the request.
The response would something like this.

Host: storage.googleapis.com
> User-Agent: curl/7.54.0
> Accept: */*
> x-goog-resumable: start
>

< HTTP/2 201
< content-type: text/plain; charset=utf-8
< x-guploader-uploadid: some-id
< location: session URL for actual resumable upload
< content-length: 0
< date: Mon, 07 Sep 2020 12:18:00 GMT
< server: UploadServer

Now use the location to make a PUT call to upload the object. For more info on which headers to use on interruption see link

Answer 2

According to this post, it is possible.

1. Client requests a signature so it can do a PUT

2. Your server does creates and returns a signed URL.

3. Your server makes a POST request to initiate the resumable upload

4. Your server returns both the URL and the Upload ID to the client

5. Client does one or more PUTs using the provided URL and Upload ID