argo env parameters and inheritance

Question

I wasn't really sure how to label this question, because i'll be good with any of the solutions above (inheritance of containers or defining parameters for the entire workflow without explicitly setting them in each step template).

i am currently working with argo yaml's and i want to define certain values that will be inputted once (and also, be optional), and will be used by every pod in the yaml.

i'm sure there's a better way to do this than what i found by now, but i can't find anything in the docs. currently the way i saw was defining that parameter as a workflow argument, and then for each container defined - defining it as an input parameter/env parameter.

my question is this - isn't there a way to define those 'env' variables at the top level? of the workflow? so that every pod will use them without me explicitly telling it to?

or - maybe even create one container that has those arguments defined, so that every other container i define inherits from that container and i wouldn't have to write those parameters as input/env for each one i add?

i wouldn't want to add these three values to each container i define. it makes the yaml very big and hard to read and maintain.

    container:
      env:
      - name: env_config
        value: "{{workflow.parameters.env_config}}"
      - name: flow_config
        value: "{{workflow.parameters.flow_config}}"
      - name: flow_type_config
        value: "{{workflow.parameters.flow_type_config}}"

would love to get your input, even if it's pointing me at the direction of the right doc to read, as i haven't found anything close to it yet.

Thanks!

Answer 1

You could use a templating tool like Kustomize or Helm to cut down on the duplication.

You could also write the params to a JSON file, pull it into each Pod as an artifact, and then have a script loop over the values and assign them to env vars. But for this to be worth the additional write step and artifacts yaml, you'd need to be dealing with a lot of env vars.

If you're using the exact same inputs for a large number of steps, it's probably worth considering whether those steps are similar enough to abstract out to one parameterized template. Perhaps you could loop over an array like ["mode1", "mode2", "mode3"...] instead of writing the steps out in series.

Honestly, duplication isn't the worst thing ever. An IDE with a nice find/replace feature should make it simple enough to make changes as necessary.

Answer 2

just realised i haven't updated, so for anyone interested, what i ended up doing is setting an anchor inside a container template:

templates:
#this template is here to define what env parameters each container has using an anchor.
  - name: env-template
    container:
      env: &env_parameters
      - name: env_config
        value: "{{workflow.parameters.env_config}}"
      - name: flow_config
        value: "{{workflow.parameters.flow_config}}"
      - name: run_config
        value: "{{workflow.parameters.run_config}}"

and than using that anchor in each container.

    container:
      image: image
      imagePullPolicy: Always
      env: *env_parameters