Add content to yaml file using yq from mike farah

Question

Add content to yaml file using yq from mike farah

This is a basic question but I am struggling to make this work as I have no experience in bash.

Lets say I have a directory directory_1 and inside this directory I have cen1.yaml and cen2.yaml. cen1.yaml has a field called name

name:xxxx

and cen2.yaml has content that can vary (example)

service:ser1
image:
  field1:field1
  field2:field2
.
.
.

In the first iteration I have an empty final.yaml. But I want this file to have the name from cen1.yaml and all content from cen2 should go under name like this this.

xxxx:
  service:ser1
  image:
    field1:field1
    field2:field2
  .
  .
  .

Next I will clone another directory with same file names (cen1.yaml and cen2.yaml). Cen1.yaml again has name field

name:another

and cen2.yaml will have its own content.

tv:lcd
name:
  field1:field1
  field2:field2
.
.
.

Now I need to append to the file that already exists keeping the same structure

xxxx:
  service:ser1
  image:
    field1:field1
    field2:field2
  .
  .
  .
another
  tv:lcd
  mount:
    field1:field1
    field2:field2
  .
  .
  .

So I need a command that works for both cases when file is empty and it writes the first iteration, or when the file already has content and it will append to it. How can I achieve this?

Asked By: Andre Silva

||

Source

Answer 1

The following is an expression for Mike Farah’s yq that loads the name from cen1.yaml and puts it into the internal yq variable $name. It then creates a new YAML document and loads cen2.yaml into it, under the key given by $name:

load("cen1.yaml").name as $name | .[$name] = load("cen2.yaml")

We can use this to loop over several subdirectories, applying yq in each, and collecting the output in final.yaml. The code assumes that the directories all match the filename globbing pattern directory_*:

for dir in directory_*; do
(
    cd "$dir" &&
    yq -n 'load("cen1.yaml").name as $name | .[$name] = load("cen2.yaml")'
)
done >final.yaml

We use -n with yq since we don’t use any files given on the command line.

Given the data in your question (corrected by removing the dotted lines and inserting spaces after each :), this would produce the following YAML file:

xxxx:
  service: ser1
  image:
    field1: field1
    field2: field2
another:
  tv: lcd
  name:
    field1: field1
    field2: field2

If all directories are not available at once, you can build your final.yaml file by appending bits to it:

rm -f final.yaml

# Get directory_1, and then...
(
    cd directory_1 &&
    yq -n 'load("cen1.yaml").name as $name | .[$name] = load("cen2.yaml")'
) >>final.yaml

# Get directory_2, and then...
(
    cd directory_2 &&
    yq -n 'load("cen1.yaml").name as $name | .[$name] = load("cen2.yaml")'
) >>final.yaml

# etc.

… essentially running each iteration of the loop manually and appending the output to the result file in each step.

The same thing, but with Andrey Kislyuk’s yq (a wrapper around jq):

for dir in directory_*; do
(
    cd "$dir" &&
    yq -n -y 'input.name as $name | .[$name] = inputs' cen1.yaml cen2.yaml
)
done >final.yaml

Answered By: Kusalananda