Having trouble with yaml render in python ( multi line string )

Question

Here is my python function which i use to render a yaml template with any variables. It works fine up untill i pass multi line string to the sample file.

def render_template(template_path, context):
    """
    Render Jinja2 template using the provided context.
    """
    with open(template_path, 'r') as template_file:
        template = Template(template_file.read())
        return template.render(context)

Here is place from where i call the above function

        if centralized_data is not None:
            print("centralized_data is not None")

            rendered_template = render_template(input_template_path, context)
            sample_data = yaml.safe_load(rendered_template)

            # Ensure both sample_data and centralized_data are dictionaries
            if isinstance(sample_data, dict) and isinstance(centralized_data, dict):
                final_data = merge_data(sample_data, centralized_data)
            else:
                print("Error: Either sample_data or centralized_data is not a dictionary")
                # Handle the error condition appropriately
                final_data = {}
        else:
            print("centralized_data is None, using sample_data directly")
            with open(input_template_path, 'r') as template_file:
                sample_data = yaml.safe_load(template_file)
            final_data = sample_data

THe problem i believe is happening at sample_data = yaml.safe_load(rendered_template)

this is where my multi string goes for a toss and looks something like this

input

deployment:
  annotations:
    proxy.istio.io/config: |
      concurrency: 1
      concurr222ency: 1
      concurrenc33y: 1
      f: 3

and output

deployment:
  annotations:
    proxy.istio.io/config: 'concurrency: 1

      concurr222ency: 1

      concurrenc33y: 1

      f: 3

      '

I also tried stuff like, but nope it does not help

proxy.istio.io/config: |>
proxy.istio.io/config: >-

Answer 1

It seems you are using PyYAML, and not only is it not really built to do this kind of round-tripping YAML, as you experienced. It also only supports a subset of YAML 1.1, which was outdated in 2009.

I recommend using ruamel.yaml instead (disclaimer: I am the author of that package):

import sys
from pathlib import Path

import ruamel.yaml

template_path = Path('template.yaml')
    
yaml = ruamel.yaml.YAML()
# yaml.preserve_quotes = True  # by default superfluous quotes are stripped
data = yaml.load(template_path)
yaml.dump(data, sys.stdout)

which gives:

deployment:
  annotations:
    proxy.istio.io/config: |
      concurrency: 1
      concurr222ency: 1
      concurrenc33y: 1
      f: 3

As you can see the literal style scalar ( the three lines after the |) is preserved as is.

It also looks that that that literal style scalar contains valid YAML, if you load that after loading the "template", you can reinsert the dump of that, but it will need some care in order not to lose the literal scalar style.

Your input doesn't look like a Jinja2 template to me, those usually have {{ and }}. And because of those you usually cannot load a Jinja2 template with a YAML parser, as there are Jinja2 constructs that make the template invalid. ruamel.yaml has a plugin, especially for these that converts the template to valid YAML so you can update the parts that you want, and then converts it back to Jinja2. After that you can render it and load the rendered text as valid YAML ( see also this or this answer )