I have AWS infrastructure that is managed by CloudFormation. I have a CI pipeline that spins up an envionment with all the bells and whistles of a customer-facing environment and runs integration tests against the environment, then CI tears down the environment once this is done. The pipeline is triggered whenever there is a pull request open to merge code to the main branch. The integration tests are currently built in Postman and dynamically configured in the pipeline. If there is an error from running the tests, they provide a generic summary of the fault code, and requires further investigation into the Cloudwatch logs to get the source of the error caused by the new code. Since the Cloudwatch logs are built as part of the environment, they get teared down shortly after the integration tests have finished running. This introduces a timing issue with investigating a test error as the logs would get teared down very quickly after the tests are run.
Therefore, I am currently configuring the Cloudwatch log groups to be streamed to S3 using Kinesis Firehose. This is being done by adding Cloudwatch Subscription Filters to each of the log groups of interest using a custom resource, I am wondering how I would ensure the logs are are in S3 before the pipeline goes to tear down the whole environment, does anyone have any ideas?
- Possibly thinking of scripting a way to listen for the S3 key that Firehose creates for the environment as an option.
- We have several cloudformation stacks, possibly there is a way to put a "DependsOn" attribute on the custom resource for all the stacks?