jq: Printing multiple values from multiple arrays at once

Question

jq: Printing multiple values from multiple arrays at once

The default functionality of jq is to send each object from an array one at a time, though the join operator can merge those values. My problem is in trying to print all the values from multiple arrays at once. Taking this example:

{
    "key1": {
        "list1": [
            "val1",
            "val2",
            "val3"
        ]
    },
    "key2": {
        "list1": [
            "val4",
            "val5"
        ]
    },
    "key3": {
        "list1": [
            "val6"
        ]
    }
}

I’d like to print:

val1 val2 val3 val4 val5 val6

And so far have this:

jq -r 'to_entries[] | { list: .value.list1 } | .list | join(" ")' test.json

(More verbose than necessary to help reviewers.)

which gives:

val1 val2 val3
val4 val5
val6

Is there a way to gather all the values together in one command?

Asked By: T145

||

Source

Answer 1

$ jq -r '[ .[].list1[] ] | join(" ")' file
val1 val2 val3 val4 val5 val6

Create a new array with all the elements of each list1 array from each top-level key. Then, join its elements with spaces. This would give you the values in the order they occur in the input file.

An alternative (and arguably neater) approach is with map(.list1) which returns an array of arrays that you may flatten and join up:

$ jq -r 'map(.list1) | flatten | join(" ")' file
val1 val2 val3 val4 val5 val6

Your attempt generates one joined string per top-level key due to .list being one of the list1 arrays in turn. Your approach would work if you encapsulated everything up to the last pipe symbol in a [ ... ] (and expand the .list with .list[]) to generate a single array that you then join. This is what I do in my first approach above; only I use a slightly shorter expression to generate the elements of that array.

$ jq -r '[ to_entries[] |  { list: .value.list1 } | .list[] ] | join(" ")' file
val1 val2 val3 val4 val5 val6

Answered By: Kusalananda

Answer 2

Using Raku (formerly known as Perl_6)

~$ raku -MJSON::Tiny -e 'my %hash = from-json($_) given lines;  
                         my @a = %hash.values.map({ $_.values if $_{"list1"} }); 
                         .say for @a.sort.join(" ");'  file

OR:

~$ raku -MJSON::Tiny -e 'my  %hash = from-json($_) given lines; 
                         for %hash.values.sort() { print .values.sort ~ " " if $_{"list1"} };
                         put "";'  file

Raku is a programming language in the Perl-family that provides high-level support for Unicode. Like Perl, Raku has associative arrays (hashes and/or maps) built-in. The above code is admittedly rather verbose (first example), but you should be able to get the flavor of the language from both examples above:

Raku’s community-supported JSON::Tiny is called at the command line,
All lines are given as one data element to the from-json function, which decodes the input and stores it in %hash,
First Example: Using a map, the values of the hash are searched through for "list1" keys. If (if) found, these are stored in the @a array. Then the @a array is printed.
Second Example: the %hash is iterated through using for, searched through for "list1" keys, and if found the associated values are printed (with at end-of-line). A final put call adds a newline.

Sample Input (includes bogus "list2" elements)

{
    "key1": {
        "list1": [
            "val1",
            "val2",
            "val3"
        ]
    },
    "key2": {
        "list1": [
            "val4",
            "val5"
        ]
    },
    "key3": {
        "list1": [
            "val6"
        ]
    },
    "key4": {
        "list2": [
            "val7"
        ]
    }
}

Sample Output:

val1 val2 val3 val4 val5 val6

Finally, in any programming solution it is often instructive to look at intermediate data-structures. So here’s what the %hash looks like after decoding JSON input:

~$ raku -MJSON::Tiny -e 'my %hash = from-json($_) given lines;  .say for %hash.sort;'  file
key1 => {list1 => [val1 val2 val3]}
key2 => {list1 => [val4 val5]}
key3 => {list1 => [val6]}
key4 => {list2 => [val7]}

https://raku.land/cpan:MORITZ/JSON::Tiny
https://docs.raku.org/language/hashmap
https://raku.org

Answered By: jubilatious1