Redeploying Google cloud function looses python dependencies

Question

I have a simple cloud function that contains the main.py file, and a requirements.txt.

This is the content of the main.py:

from superpackage.subpackage import TheSuperClass

def watch_function(event, context):
    print(f"Event ID: {context.event_id}")
    print(f"Event type: {context.event_type}")
    print("Bucket: {}".format(event["bucket"]))
    print("File: {}".format(event["name"]))
    print("Metageneration: {}".format(event["metageneration"]))
    print("Created: {}".format(event["timeCreated"]))
    print("Updated: {}".format(event["updated"]))
    a = TheSuperClass()

And this is is the content of the requirements.txt file, it just references a repo that contains a python package. Using this package locally works fine.

-e git+https://github.com/someorg/[email protected]#egg=superpackage

Using normal deploy the function works fine, and the superpackage.subpackage can be used. This is initial command for deploy.

gcloud functions deploy peter-function \
  --runtime python311 \
  --trigger-resource the-bucket-we-watch  \
  --entry-point watch_function \
  --trigger-event google.storage.object.finalize

Uploading file to the-bucket-we-watch triggers the function, and all is fine.

Now we come to the problem. I make a small change in the main.py, and redeploy the function, with the same name, and the same command as above. It fails to deploy with error. It somehow does not include the requirements.txt on redeploy. Deploying the change with new function name works as expected. I have updated the gcloud sdk to the latest version and tried again before asking this question :)

Deploying function (may take a while - up to 2 minutes)...failed.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Function failed on loading user code. This is likely due to a bug in the user code. Error message: Traceback (most recent call last):
  File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
    sys.exit(_cli())
             ^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/functions_framework/_cli.py", line 37, in _cli
    app = create_app(target, source, signature_type)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/layers/google.python.pip/pip/lib/python3.11/site-packages/functions_framework/__init__.py", line 288, in create_app
    spec.loader.exec_module(source_module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/workspace/main.py", line 1, in <module>
    from superpackage.subpackage import TheSuperClass
ModuleNotFoundError: No module named 'superpackage'. Please visit https://cloud.google.com/functions/docs/troubleshooting for in-depth troubleshooting documentation.

Answer 1

Make sure that you are following the same code structure from the documentation when using Cloud Storage triggers. Take a look at this sample main.py

from cloudevents.http import CloudEvent

import functions_framework


# Triggered by a change in a storage bucket
@functions_framework.cloud_event
def hello_gcs(cloud_event: CloudEvent) -> tuple:
    """This function is triggered by a change in a storage bucket.

    Args:
        cloud_event: The CloudEvent that triggered this function.
    Returns:
        The event ID, event type, bucket, name, metageneration, and timeCreated.
    """
    data = cloud_event.data

    event_id = cloud_event["id"]
    event_type = cloud_event["type"]

    bucket = data["bucket"]
    name = data["name"]
    metageneration = data["metageneration"]
    timeCreated = data["timeCreated"]
    updated = data["updated"]

    print(f"Event ID: {event_id}")
    print(f"Event type: {event_type}")
    print(f"Bucket: {bucket}")
    print(f"File: {name}")
    print(f"Metageneration: {metageneration}")
    print(f"Created: {timeCreated}")
    print(f"Updated: {updated}")

    return event_id, event_type, bucket, name, metageneration, timeCreated, updated

Also, You can also use this gcloud CLI as a reference in deploying your function.

gcloud functions deploy python-finalize-function \
--gen2 \
--runtime=python311 \
--region=REGION \
--source=. \
--entry-point=hello_gcs \
--trigger-event-filters="type=google.cloud.storage.object.v1.finalized" \
--trigger-event-filters="bucket=YOUR_BUCKET_NAME"

Note:

If you do not specify the --source flag:

• The current directory will be used for new function deployments.

• If the function was previously deployed using a local filesystem path, then the function's source code will be updated using the current directory.

• If the function was previously deployed using a Google Cloud Storage location or a source repository, then the function's source code will not be updated.

References:

• MissingTargetException when deploying Google Cloud Function

• Cloud Storage Tutorial (2nd gen)

• gcloud functions deploy