Google Cloud Storage upload triggers python app alternatives to Cloud Function

Question

I'd like to find out what's the best architecture for a python app that gets triggered when a file is uploaded to Google Cloud Storage, does some processing and outputs a file to Google Drive?

I've tried using Cloud Functions but I'm getting a Function invocation was interrupted. Error: memory limit exceeded. in the logs.

I've also followed this tutorial Trigger Cloud Run with events from Eventarc so I know that one way is with EventArc and Cloud Audit logs.

2 questions:

1. What other methods are there since I require higher memory limits?
2. How do I get the bucket name and file name from cloud audit logs? through protoPayload.resourceName?

Answer 1

You can use PubSub. You can create a PubSub notification and create a push subscription to the service that you want.

• Http cloud function
• App Engine
• Cloud Run
• Any HTTP service running somewhere (VM, Kubernetes, on prem,...)

EventArc is mainly a wrapper of this process and can call only Cloud Run service (for now)