cloudformation sceptre: How to parameterize a list of json objects

Question

I am using sceptre and troposphere to generate my architecture.

In AWS Batch Job Definition, I want to parameterize the Environment of the job definition's container properties:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-jobdefinition-containerproperties.html

The environment is a list of https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-batch-jobdefinition-environment.html.

But in the context of sceptre, it doesn't seem like I can declare something like this:

self.JobDefinitionEnvironment = t.add_parameter(Parameter(
    'JobDefinitionEnvironment',
    Type='List<AWS::Batch::JobDefinition::Environment>'
))

In sceptre:

1. Do I have to write my own resolver to solve this problem? Even if I write my own resolver to resolve JobDefintion::Environment, the resolver returns only a string, not a list of objects.

2. Am I not supposed to parameterize the container properties and hardcode the values in the job-definition.py?

3. I am currently working around by "flatten" out the object, so I can supply the values. Kinda ugly.

Please help!

Answer 1

I'm not sure what you can do with sceptre, but with stacker (http://stacker.readthedocs.io/en/latest) you would just make the definition list a variable, and then use that variable in your blueprint (which is written in troposphere).

An example that might make sense is the open source ECS blueprint:

https://github.com/cloudtools/stacker_blueprints/blob/master/stacker_blueprints/ecs.py#L69-L74 (Where the variable is defined)

https://github.com/cloudtools/stacker_blueprints/blob/master/stacker_blueprints/ecs.py#L69-L74 (where we turn the variable dictionary into a list of objects like ECS expects)

https://github.com/cloudtools/stacker_blueprints/blob/master/stacker_blueprints/ecs.py#L296 (where that is used in the container definition)

Sorry I don't know sceptre well though. Not sure if you're stuck using it, or if stacker is potentially useful to you.

Answer 2

Cloudformation doesn't support List<AWS::Batch::JobDefinition::Environment> (see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html)

There are a few approaches to your problem

Note i'm more familiar with using yaml & jinja, but the bellow should work with troposphere too, and i will add examples when i get the chance

Sceptre_user_data

in config/<stack_name>.yaml add

sceptre_user_data:
  JobDefinitionEnvironment:
    - <list-goes-here-as-yaml/json>

in cfn/<template>.yaml.j2 add

  myBatchJob
    Type: AWS::Batch::JobDefinition
    Properties:
      Environment:
        {{ sceptre_user_data.JobDefinitionEnvironment }}

Native

To do it natively, you can use Type: CommaDelimitedList and select, this is a bit cumbersome

Parameters:
  BatchEnvironmentSetting1:
    Type: CommaDelimitedList
    Default: MySetting,MyValue

Resources:
  myBatchJob:
    Type: AWS::Batch::JobDefinition
    Properties:
      Environment:
       - Name: !Select [0, BatchEnvironmentSetting]
         Value: !Select [1, BatchEnvironmentSetting]

With Conditions and loops

If the native approach works for you then you can make the values optional (and using loops makes this less painful

Parameters:
  {% for i in range(20) %}
  BatchEnvironmentSetting{{ i }}:
    Type: CommaDelimitedList
    Default: ''
  {% endfor %}

Conditions:
  {% for i in range(20) %}
  UseBatchEnvironmentSetting{{ i }}: !Not [!Equals [BatchEnvironmentSetting{{ i }}, '']] 
  {% endfor %}


Resources:
  myBatchJob:
    Type: AWS::Batch::JobDefinition
    Properties:
      Environment:
       {% for i in range(20) %}
       - !If 
         - UseBatchEnvironmentSetting{{ i }}
         - Name: !Select [0, BatchEnvironmentSetting{{ i }}]
           Value: !Select [1, BatchEnvironmentSetting{{ i }}]
         - !Ref AWS::NoValue
      {% endfor %}

This kind of looping is one of the places where troposphere might make the code cleaner as you could have a AddEnviornmentSetting function that implements all 3 loops