How to add content to a file before the last } character using Busybox utilities?

There is a file with contents

{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null
}

How to add content to a file before the last } character using Busybox utilities so that the file content becomes like below?

{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null,
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
}

However, the } character is not necessarily the last character in the file and is not necessarily located on the last line.

So far I have found only this solution

tac file2 | sed '0,/}/s/}/}n"field4": "value4"n"field3": "value3",n"field2": "value2",n"field1": "value1",n,/' | tac>tmp_file && mv tmp_file file2
Asked By: Maxim

||

Using the ed utility (tested in a BusyBox container):

ed file<<EOF
$ i
  ,
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
.
w
q
EOF
cat file

The output file pass the test.

Answered By: Gilles Quénot

If the last } is the last line then this can be done with head command.
Include the last } in your added text:

$ cat addit
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
}

And what you want is:

$ (head -n -1 input_file ;cat addit)

You leave out the last } from your added text and instead pipe the above output to this sed:

$ sed '$ a }'
Answered By: user9101329

If you are very confident that the file ends with }n (or just } no newline at end of file), you could truncate the last 2 bytes (or 1 byte). Use tail -c to verify beforehand.

After truncate, the file would be missing the last } but be otherwise intact. At this point simply >> append new text (including the } you truncated).

Unfortunately, busybox truncate does not understand relative sizes (-2) so you need to do the math yourself and use stat -c %s to determine the size in bytes.

This method would also work with very large files, without needing to read all the contents first. awk, ed, head -n, etc. all read the entire file.

Answered By: frostschutz

Using any awk and any bourne-like shell:

$ cat tst.sh
#!/usr/bin/env bash

new='
"field1": "value1",
"field2": "value2",
"field3": "value3",
"field4": "value4"
'

awk '
    { lines[NR] = $0 }
    $0 == "}" { last = NR - 1 }
    END {
        for ( i=1; i<last; i++ ) {
            print lines[i]
        }
        print lines[i] ","

        indent = lines[i]
        sub(/[^ t].*/,"",indent)
        gsub(/(^|n)/,"&"indent,new)
        print new

        for ( ++i; i<=NR; i++ ) {
            print lines[i]
        }
    }
' new="$new" file

$ ./tst.sh file
{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null,
  "field1": "value1",
  "field2": "value2",
  "field3": "value3",
  "field4": "value4"
}

Note that it sets the indent for the new block to be whatever the indent is for the line immediately before it so you don’t have to hard-code that indent and it’ll work even if you have lines of comments or blank lines after the last } line.

Answered By: Ed Morton

Note: This answer does not restrict itself to using only utilities built into the Busybox multi-binary but uses the jq utility. This well-known JSON processing tool is available as a static binary for most common architectures, requiring no escalation of permission to install or use. You can also use it via a public Docker image.

Since Busybox multi-binaries may contain a wildly varying number of built-in utilities, depending on exactly what Busybox is being used (302 utilities on Alpine Linux, but an additional 100 extra in the public Busybox Docker image), and since this was not mentioned in the question, I’m assuming that the restriction is purely artificial and not terribly important.


Once you have installed jq you may use it in different ways to add your data, depending on in what form you provide the new data.

Assuming separate keys and unencoded strings, you would use jq with --arg key value to encode each string and create internal jq variables. For values that are already appropriately encoded (numbers, booleans, JSON strings, or JSON fragments) use --argjson instead of --arg. You may then use the key names prepended by $ ($key1, $var2, $lastname etc.) inside the jq expression to access the individual pieces of data, or you may access all such variables using the special $ARGS.named object:

jq  --arg field1 value1 
    --arg field2 'value2 ("temporarily")' 
    --arg field3 value3 
    --argjson field4 true 
    --argjson field5 '{ "some json fragment": [ "goes", "here" ] }' 
    '. += $ARGS.named' file.json

With your first example document in file.json, this would produce the following:

{
  "first_name": "John",
  "last_name": "Smith",
  "is_alive": true,
  "age": 27,
  "address": {
    "street_address": "21 2nd Street",
    "city": "New York",
    "state": "NY",
    "postal_code": "10021-3100"
  },
  "phone_numbers": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "office",
      "number": "646 555-4567"
    }
  ],
  "children": [
    "Catherine",
    "Thomas",
    "Trevor"
  ],
  "spouse": null,
  "field1": "value1",
  "field2": "value2 ("temporarily")",
  "field3": "value3",
  "field4": true,
  "field5": {
    "some json fragment": [
      "goes",
      "here"
    ]
  }
}

You may obviously decide where the data is inserted in the document by specifying a path different from . in the expression above. For example, to add the data to any input object that represents a person who has at least one home phone number starting with "212":

select(
    .phone_numbers |
    map(.type == "home" and (.number | startswith("212"))) |
    any
) += $ARGS.named
Answered By: Kusalananda
Categories: Answers Tags: , , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.