Airflow, copy file to Google Drive

47 Views Asked by At

I want to use Airflow to move file from GCS to Drive every day. The Operator GCSToGoogleDriveOperator works fine, but each time it creates new file with the same name and I have many files with the same name.

Is it possible to update an existing file?

I have:

  • Day 1: create new file in Drive.
  • Day 2: create new file in Drive (with the same name)
  • and so on.

I want:

  • Day 1: create new file in Drive.
  • Day 2: update the file created on Day 1.
  • and so on.

My code:

    move_file_from_gcs_to_drive = GCSToGoogleDriveOperator(
        task_id="move_file_from_gcs_to_drive",
        gcp_conn_id=CONFIGURATION['drive']['gcp_conn_id'],
        source_bucket=CONFIGURATION['drive']['source_bucket'],
        source_object=CONFIGURATION['drive']['source_object'],
        destination_object=CONFIGURATION['drive']['destination_object'],
        destination_folder_id=CONFIGURATION['drive']['destination_folder_id']
    )
0

There are 0 best solutions below