AWS AppFlow CloudFormation Template Size Limit - Mapping 500+ fields

Question

I am trying to create an AWS::AppFlow::Flow that can map a large number (500+) of fields from a Salesforce source object into an S3 object. The list of fields must be specified (rather than simply slurping all fields).

Please correct me if I'm wrong here, but it seems that my template MUST have a TaskType: Filter with a ConnectorOperator: PROJECTION and the full list of SourceFields I wish to copy.
Further, for each field, I seem to need a separate TaskType: Map defined for that field...

I used this example template to get started, with the relevant bits:

      Tasks:
        - TaskType: Filter
          SourceFields:
            - FieldA
            - FieldB
            - FieldN
          ConnectorOperator:
            Salesforce: PROJECTION
        - TaskType: Map
          SourceFields:
            - FieldA
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: string
        - TaskType: Map
          SourceFields:
            - FieldB
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: datetime
        - TaskType: Map
          SourceFields:
            - FieldN
          TaskProperties:
            - Key: SOURCE_DATA_TYPE
              Value: integer

Unfortunately, the list of fields I need to map is very large, and the template exceeds the maximum size allowed for CloudFormation templates (51200 bytes). I imagine most of the space is used by the repetitive Map task blocks for each field.

I am aware that you can upload the template file to S3, allowing a larger file size quota; however, even if I could pursue this option (I can't), the file size would also exceed the CloudFormation limit!

Is there another way to do this that would save template space?

I've found one or two mentions of a Map_all TaskType, which sounds promising on the surface; however, I can't find any concrete documentation on how to use it.

PS: For what it's worth, just the list of field names, comma-separated, is ~14KB.
PPS: I didn't design this schema, and I have no ability to change it.

Answer 1

i would recommend on excluding the columns thats not required , in the exclude column list , which will reduce the template size