jq: sed for JSON - Programmer Tools

| jq '.pods[] | .name, .replication'
​jq​​ ​​'._all.primaries | { count: .docs.count, size: .store.size_in_bytes }'​

https://michaelheap.com/extract-keys-using-jq/

The first thing to do was to step in to the wrapper, something.

cat data.json | jq '.something'

Next, we convert to a structure we can select data from by using to_entries, and unwrap it from it’s array by removing the []

cat data.json | jq '.something | to_entries[]'

You can create JSON structures using jq by using braces and providing keys

cat data.json | jq '.something | to_entries[] | {"key": .key, "value": .value.foo}'

Finally, we can zip this up so that it becomes a key/value object like we started with. To do this, we need to wrap our return data with square brackets so that it becomes a list again:

cat data.json | jq '[.something | to_entries[] | {"key": .key, "value": .value.foo}]'

Then we use the from_entries function to remove the key and value keys, zipping them up in to an object. This does the opposite of to_entries

cat data.json | jq '[.something | to_entries[] | {"key": .key, "value": .value.foo}] | from_entries'

https://shapeshed.com/jq-json/
jq '.' file.json

cat file.json | jq '.' | less

jq '.name' dog.json
jq '.breed,.age' dog.json

To search for nested objects chain values using the dot operator just as you would in JavaScript.

jq '.owner.name'

How to find items in an array

jq '.likes[0]'
"bones"

echo '["a","b","c","d","e"]' | jq '.[2:4]'

jq '.[] | .name' names.json

https://stedolan.github.io/jq/
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

https://stedolan.github.io/jq/manual/

.

The absolute simplest (and least interesting) filter is .. This is a filter that takes its input and produces it unchanged as output.

Since jq by default pretty-prints all output, this trivial program can be a useful way of formatting JSON output from, say, curl.

.foo, .foo.bar

The simplest useful filter is .foo. When given a JSON object (aka dictionary or hash) as input, it produces the value at the key “foo”, or null if there’s none present.

If the key contains special characters, you need to surround it with double quotes like this: ."foo$".

A filter of the form .foo.bar is equivalent to .foo|.bar.

.foo?

Just like .foo, but does not output even an error when . is not an array or an object.

.[<string>], .[2], .[10:15]

You can also look up fields of an object using syntax like .["foo"] (.foo above is a shorthand version of this). This one works for arrays as well, if the key is an integer. Arrays are zero-based (like javascript), so .[2] returns the third element of the array.

The .[10:15] syntax can be used to return a subarray of an array or substring of a string. The array returned by .[10:15] will be of length 5, containing the elements from index 10 (inclusive) to index 15 (exclusive). Either index may be negative (in which case it counts backwards from the end of the array), or omitted (in which case it refers to the start or end of the array).

The .[2] syntax can be used to return the element at the given index. Negative indices are allowed, with -1 referring to the last element, -2 referring to the next to last element, and so on.

The .foo syntax only works for simply keys i.e. keys that are all alphanumeric characters. .[<string>]works with keys that contain special characters such as colons and dots. For example .["foo::bar"]and .["foo.bar"] work while .foo::bar and .foo.bar would not.

The ? “operator” can also be used with the slice operator, as in .[10:15]?, which outputs values where the inputs are slice-able.

.[]

If you use the .[index] syntax, but omit the index entirely, it will return all of the elements of an array. Running .[] with the input [1,2,3] will produce the numbers as three separate results, rather than as a single array.

jq '.[]'

.[]?

Like .[], but no errors will be output if . is not an array or object.

,

If two filters are separated by a comma, then the input will be fed into both and there will be multiple outputs: first, all of the outputs produced by the left expression, and then all of the outputs produced by the right. For instance, filter .foo, .bar, produces both the “foo” fields and “bar” fields as separate outputs.

|

The | operator combines two filters by feeding the output(s) of the one on the left into the input of the one on the right. It’s pretty much the same as the Unix shell’s pipe, if you’re used to that.

If the one on the left produces multiple results, the one on the right will be run for each of those results. So, the expression .[] | .foo retrieves the “foo” field of each element of the input array.

jq '.[] | .name'

length

The builtin function length gets the length of various different types of value:

• The length of a string is the number of Unicode codepoints it contains (which will be the same as its JSON-encoded length in bytes if it’s pure ASCII).
• The length of an array is the number of elements.
• The length of an object is the number of key-value pairs.
• The length of null is zero.

--color-output / -C and --monochrome-output / -M:

By default, jq outputs colored JSON if writing to a terminal. You can force it to produce color even if writing to a pipe or a file using -C, and disable color with -M.

tostring

The tostring function prints its input as a string. Strings are left unchanged, and all other values are JSON-encoded.

jq -c .Destinations[]

join(str)

Joins the array of elements given as input, using the argument as separator. It is the inverse of split: that is, running split("foo") | join("foo") over any input string returns said input string.

Numbers and booleans in the input are converted to strings. Null values are treated as empty strings. Arrays and objects in the input are not supported.

jq 'join(", ")'Input["a","b,c,d","e"]Output"a, b,c,d, e"

split(str)

Splits an input string on the separator argument.

curl "" | jq '.'
curl "" | jq '.collections|join(" ")''

select(boolean_expression)

The function select(foo) produces its input unchanged if foo returns true for that input, and produces no output otherwise.

It’s useful for filtering lists: [1,2,3] | map(select(. >= 2)) will give you [2,3].

jq '.[] | select(.id == "second")'
Input	[{"id": "first", "val": 1}, {"id": "second", "val": 2}]
Output	{"id": "second", "val": 2}

map(x), map_values(x)

For any filter x, map(x) will run that filter for each element of the input array, and produce the outputs a new array. map(.+1) will increment each element of an array of numbers.

Similarly, map_values(x) will run that filter for each element, but it will return an object when an object is passed.

map(x) is equivalent to [.[] | x]. In fact, this is how it’s defined. Similarly, map_values(x) is defined as .[] |= x.

jq 'map(in([0,1]))'
Input	[2, 0]
Output	[false, true]

https://robots.thoughtbot.com/jq-is-sed-for-json
brew install jq
jq is built around filters. The simplest filter is ., which echoes its input, but pretty-printed:

The simplest useful filter is .field, which pulls field out of each record:
$ echo '{"hello":{ "greetings":"to you"}}' | jq .hello
{
  "greetings": "to you"
}

jq ".[] | .text" 1.json
we need to go “into” the array first with .[]. Then we can pass each object in the array to the next filter with |, and grab the text field from each object using .text:

jq ".[] | .text" 1.json | sed 's/h/H/g'

Transforming input
jq ".[] | { the_user: .user, the_text: .text }" 1.json
The .[] goes into the array, giving us an array of objects. Then we pass those objects to the next filter using |, and use the {} filter to construct a new object using fields from each object. We also rename user to the_user and text to the_text.

To wrap something in an array, put brackets around an expression:
jq "[.[] | { the_user: .user, the_text: .text }]" 1.json

Dealing with more than one file
jq --slurp will smush together the arrays of messages in 1.json and 2.json and deal with them as one giant array.
jq --slurp "[.[] | .[] | { text: .text }]" 1.json 2.json

https://zerokspot.com/weblog/2013/07/18/processing-json-with-jq/
cat file.json | python -m 'json.tool' > file.clean.json
cat file.json | jq '.[] | .name'
The first filter receives an array object and is instructed to pass each element on its own down to the next one. So ‘.name’ now receives the store object as input and send the value of the “name” property out.

But I actually wanted all the data for a store with a specific name. For this, jq offers the select filter:
cat file.json | jq '.[] | select(.name=="Graz")'

jq ' .response|.docs|.[0]|.field | length ' file.json
https://stedolan.github.io/jq/tutorial/

https://stackoverflow.com/questions/33247228/how-to-use-jq-in-a-shell-pipeline

You need to supply a filter as an argument. To pass the JSON through unmodified other than the pretty printing jq provides by default, use the identity filter .:

curl -s https://api.github.com/users/octocat/repos | jq '.' | cat

https://github.com/stedolan/jq/issues/1110