iREADME - json2tsv - JSON to TSV converter Err codemadness.org 70 hgit clone git://git.codemadness.org/json2tsv URL:git://git.codemadness.org/json2tsv codemadness.org 70 1Log /git/json2tsv/log.gph codemadness.org 70 1Files /git/json2tsv/files.gph codemadness.org 70 1Refs /git/json2tsv/refs.gph codemadness.org 70 1README /git/json2tsv/file/README.gph codemadness.org 70 1LICENSE /git/json2tsv/file/LICENSE.gph codemadness.org 70 i--- Err codemadness.org 70 iREADME (2121B) Err codemadness.org 70 i--- Err codemadness.org 70 i 1 json2tsv Err codemadness.org 70 i 2 -------- Err codemadness.org 70 i 3 Err codemadness.org 70 i 4 Convert JSON to TSV or separated output. Err codemadness.org 70 i 5 Err codemadness.org 70 i 6 json2tsv reads JSON data from stdin. It outputs each JSON type to a TAB- Err codemadness.org 70 i 7 Separated Value format per line by default. Err codemadness.org 70 i 8 Err codemadness.org 70 i 9 Err codemadness.org 70 i 10 Build and install Err codemadness.org 70 i 11 ----------------- Err codemadness.org 70 i 12 Err codemadness.org 70 i 13 $ make Err codemadness.org 70 i 14 # make install Err codemadness.org 70 i 15 Err codemadness.org 70 i 16 Err codemadness.org 70 i 17 Dependencies Err codemadness.org 70 i 18 ------------ Err codemadness.org 70 i 19 Err codemadness.org 70 i 20 - C compiler (C99). Err codemadness.org 70 i 21 - libc Err codemadness.org 70 i 22 Err codemadness.org 70 i 23 Err codemadness.org 70 i 24 Optional dependencies Err codemadness.org 70 i 25 --------------------- Err codemadness.org 70 i 26 Err codemadness.org 70 i 27 - POSIX make(1) (for Makefile). Err codemadness.org 70 i 28 - mandoc for documentation: https://mdocml.bsd.lv/ Err codemadness.org 70 i 29 Err codemadness.org 70 i 30 Err codemadness.org 70 i 31 TAB-Separated Value format Err codemadness.org 70 i 32 -------------------------- Err codemadness.org 70 i 33 Err codemadness.org 70 i 34 By default the output format per line is: Err codemadness.org 70 i 35 Err codemadness.org 70 i 36 nodenametypevalue Err codemadness.org 70 i 37 Err codemadness.org 70 i 38 The nodename and value are escaped (\n, \t and \\). Control-characters are Err codemadness.org 70 i 39 removed. Err codemadness.org 70 i 40 Err codemadness.org 70 i 41 The type field is a single byte and can be: Err codemadness.org 70 i 42 Err codemadness.org 70 i 43 a for array Err codemadness.org 70 i 44 b for bool Err codemadness.org 70 i 45 n for number Err codemadness.org 70 i 46 o for object Err codemadness.org 70 i 47 s for string Err codemadness.org 70 i 48 ? for null Err codemadness.org 70 i 49 Err codemadness.org 70 i 50 Filtering on the first field "nodename" is easy using awk for example. Err codemadness.org 70 i 51 Err codemadness.org 70 i 52 See the json2tsv(1) man page for the full documentation of the options. Err codemadness.org 70 i 53 Err codemadness.org 70 i 54 Err codemadness.org 70 i 55 Example Err codemadness.org 70 i 56 ------- Err codemadness.org 70 i 57 Err codemadness.org 70 i 58 An usage example to parse JSON posts of reddit.com and format them to a Err codemadness.org 70 i 59 plain-text list using awk: Err codemadness.org 70 i 60 Err codemadness.org 70 i 61 Err codemadness.org 70 i 62 #!/bin/sh Err codemadness.org 70 i 63 curl -s -H 'User-Agent:' 'https://old.reddit.com/.json?raw_json=1&limit=100' | \ Err codemadness.org 70 i 64 json2tsv | \ Err codemadness.org 70 i 65 awk -F '\t' ' Err codemadness.org 70 i 66 function show() { Err codemadness.org 70 i 67 if (length(o["title"]) == 0) Err codemadness.org 70 i 68 return; Err codemadness.org 70 i 69 print n ". " o["title"] " by " o["author"] " in r/" o["subreddit"]; Err codemadness.org 70 i 70 print o["url"]; Err codemadness.org 70 i 71 print ""; Err codemadness.org 70 i 72 } Err codemadness.org 70 i 73 $1 == ".data.children[].data" { Err codemadness.org 70 i 74 show(); Err codemadness.org 70 i 75 n++; Err codemadness.org 70 i 76 delete o; Err codemadness.org 70 i 77 } Err codemadness.org 70 i 78 $1 ~ /^\.data\.children\[\]\.data\.[a-zA-Z0-9_]*$/ { Err codemadness.org 70 i 79 o[substr($1, 23)] = $3; Err codemadness.org 70 i 80 } Err codemadness.org 70 i 81 END { Err codemadness.org 70 i 82 show(); Err codemadness.org 70 i 83 }' Err codemadness.org 70 i 84 Err codemadness.org 70 i 85 Err codemadness.org 70 i 86 The following awk function can be used to unescape the json2tsv escape Err codemadness.org 70 i 87 sequences \t, \n and \\: Err codemadness.org 70 i 88 Err codemadness.org 70 i 89 awk ' Err codemadness.org 70 i 90 BEGIN { Err codemadness.org 70 i 91 FS = OFS = "\t"; Err codemadness.org 70 i 92 } Err codemadness.org 70 i 93 function unescape(s) { Err codemadness.org 70 i 94 # use the character "\x01" as a temporary replacement for "\". Err codemadness.org 70 i 95 gsub("\\\\\\\\", "\x01", s); Err codemadness.org 70 i 96 gsub("\\\\n", "\n", s); Err codemadness.org 70 i 97 gsub("\\\\t", "\t", s); Err codemadness.org 70 i 98 gsub("\x01", "\\", s); # restore "\x01" to "\". Err codemadness.org 70 i 99 return s; Err codemadness.org 70 i 100 } Err codemadness.org 70 i 101 $2 == "s" && index($3, "\\") { Err codemadness.org 70 i 102 $3 = unescape($3); Err codemadness.org 70 i 103 } Err codemadness.org 70 i 104 { Err codemadness.org 70 i 105 print $3; Err codemadness.org 70 i 106 }' Err codemadness.org 70 i 107 Err codemadness.org 70 i 108 To not have to unescape the data a different field separator and record Err codemadness.org 70 i 109 separator can be set using the -F and -R option. Err codemadness.org 70 i 110 Err codemadness.org 70 i 111 Err codemadness.org 70 i 112 License Err codemadness.org 70 i 113 ------- Err codemadness.org 70 i 114 Err codemadness.org 70 i 115 ISC, see LICENSE file. Err codemadness.org 70 i 116 Err codemadness.org 70 i 117 Err codemadness.org 70 i 118 Author Err codemadness.org 70 i 119 ------ Err codemadness.org 70 i 120 Err codemadness.org 70 i 121 Hiltjo Posthuma Err codemadness.org 70 .