How do I specify an AWS Elastic Search Ingestion Pipeline in a SAM template as a YAML string?

166 Views Asked by At

I have a sam template with an OpenSearch Ingestion Pipeline resource defined below. According to the documentation PipelineConfigurationBody is meant to be a string of YAML. But this string could be huge, with a lot of string substitutions needed as well. Is there a way to somehow include YAML from another file with string substitutions?

PortfolioDataPipeline:
    Type: AWS::OSIS::Pipeline
    Properties:
      PipelineConfigurationBody: <this is meant to be a yaml string>
1

There are 1 best solutions below

0
Karl Panganiban On
  PipelineConfigurationBody: !Sub
    - |
      version: "2"
      s3-pipeline:
        source:
          s3:
            notification_type: "sqs"
            codec:
              newline: null
            sqs:
              queue_url: "${WAFLogSQSUrl}"
            compression: "none"
            aws:
              region: "ap-northeast-1"
              # IAM role that the pipeline assumes to read data from the queue. This role must be the same as the pipeline role.
              sts_role_arn: "${PipelineRoleArn}"
        processor:
          - date:
              destination: "@timestamp"
              from_time_received: true
        sink:
          - opensearch:
              hosts:
                [
                  "https://${OpenSearchDomainEndpoint}"
                ]
              index: "aws-waf-logs-%{yyyy-MM-dd}"
              aws:
                # IAM role that the pipeline assumes to access the domain sink
                sts_role_arn: "${PipelineRoleArn}"
                region: "${AWS::Region}"
    - OpenSearchDomainEndpoint: !ImportValue opensearch-domain-endpoint
      PipelineRoleArn: !GetAtt PipelineRole.Arn
      WAFLogSQSUrl: !ImportValue waf-log-sqs-queue-url

I was looking for the same solution and did it this way.. Use !Sub and | together.