How can create an Amazon Event Bus rule to handle an AWS Data Pipeline event?

Question

1. We have an AWS Data Pipeline that copies data from S3 into Redshift (RedshiftCopyActivity).
2. We are looking to call a Lambda function when the copy is complete.

My understanding so far is:

1. Amazon Event Bus is the recommended way to handle the event and call the function
2. Data Pipeline doesn't raise events directly, but does trigger CloudTrail API calls

There is a line in the CloudTrail + Event Bridge page: "If you want to customize the event pattern, choose Edit, make your changes, and choose Save."

Are there any details on how exactly to write this rule match a specific Data Pipeline Id, RedshiftCopyActivity, and status of "FINISHED"?

Or if there is a simpler way to accomplish this I welcome it, thanks.

Answer 1

Indeed, for some reason, I can't find any documentation or sample event for the AWS Data Pipeline events on the EventBridge service.

However, I suggest You do the following:

1. Create a rule on the EventBridge service, for the the AWS Pipeline service.
2. Choose the appropriate details (AWS, Data Pipeline, All Events)

Once You have it, connect it to a Lambda and just log the calls and event you received.

Then You can actually have some example of the different events and develop the appropriate logic to handle the required.

Also, later on, You may change the event pattern within the Rule (in the EventBridge service), so You will call Lambda only on the specific event.