Preserve quotes in YAML when using yq

Question

Given config.yaml:

APP_ID: APP158
APP_NAME: "My Application"

And process-config.sh which reads the YAML entries and converts them to BASH syntax of KEY=VALUE in order to be set as environment variables:

#!/bin/bash

export $(yq e "to_entries | map(.key + \"=\" +  .value) | join(\" \")" config.yaml)

echo $APP_ID
echo $APP_NAME

The following output is produced:

APP158
My

How do I retain the quotes around APP_NAME's values so that its value isn't truncated (as shown in output)?

Answer 1

You may have issues with the overall concept. Assume that you successfully keep the quotes around "My Application", the string you are trying to construct will do the same thing in Bash.

If you try:

bash-5.2$ export $(echo APP_NAME="My Application")
bash-5.2$ echo "$APP_NAME"
My    

You can see that adding "My Application" in quotes does not fix the problem if you are just using export.

However, if you use source to read a crafted export command, you can get what you are looking for:

bash-5.2$ source <(echo "export APP_NAME=My\ Application")
bash-5.2$ echo "$APP_NAME"
My Application

The easiest way to do this might be in Ruby or Perl or jq or yq where you can escape shell meaningful words.

An example in Ruby (I assume Perl, jq, yq are the same result...):

source <(ruby -r yaml -r shellwords -e '
puts YAML.load($<.read).map{|k,v| "export #{k}=#{Shellwords.escape(v)}" }
' file)

Then:

bash-5.2$ echo "$APP_ID"
APP158
bash-5.2$ echo "$APP_NAME"
My Application

Answer 2

Use the @sh encoder (available since v4.31.1) to quote a string so that it is interpretable by the shell:

yq e 'to_entries | map(.key + "=" + (.value | @sh)) | join(" ")'

APP_ID=APP158 APP_NAME=My' Application'

Note that since version 4.18.1, "yq's 'eval/e' command is the default command and no longers needs to be specified."

Answer 3

You don't need to use to_entries at all, just output the variables in a shell compatible format and have it exported at your shell.

For recent bash versions, use process-substitution <(..) technique to expose a FIFO, that source can read as a file input from.

set -a
source <(yq -o=shell config.yaml)
set +a

In older versions of bash, e.g. 3.2 (baked into macOS by default), exporting from source doesn't work directly, you can use a work-around as below (See Why source command doesn't work with process substitution in bash 3.2?)

set -a
source /dev/stdin <<< $(yq -o=shell config.yaml)
set +a

The -/+ flags around a, serve as a placeholder, within which newly created/modified variables are exported.

Answer 4

In bash you could simply do:

$ IFS=, read -r ID NAME<<<$(yq e '[.APP_ID,.APP_NAME] | join(",")' config.yaml)
$ echo "$ID"
APP158
$ echo "$NAME"
My Application