Skip to content

Commit

Permalink
Merge pull request #42 from kellyjonbrazil/dev
Browse files Browse the repository at this point in the history 
Dev v1.4.6
  • Loading branch information
@kellyjonbrazil
kellyjonbrazil committed Nov 29, 2021
2 parents 9115ec4 + 2eb0d17 commit 3ed41ab
Show file tree
Hide file tree
Showing 11 changed files with 2,209 additions and 2,234 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
jello changelog

20211129 v1.4.6
- Note to Package Maintainers:
TLDR: `/jello/man/jello.1` is deprecated and only `/man/jello.1` should be used.
See the note at 20211126 v1.4.5
- Change schema output to include the base name of _
- Allow arrays to be printed as JSON Lines

20211126 v1.4.5
- Note to Package Maintainers:
TLDR: `/jello/man/jello.1` is deprecated and only `/man/jello.1` should be used.
Expand Down
152 changes: 77 additions & 75 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ See the [Jello Packaging](https://kellyjonbrazil.github.io/jello-packaging/) sit
### Usage
```
cat data.json | jello [OPTIONS] [QUERY]
```
```
`QUERY` is optional and can be most any valid python code. `_` is the sanitized JSON from STDIN presented as a python dict or list of dicts. If `QUERY` is omitted then the original JSON input will simply be pretty printed. You can use dot notation or traditional python bracket notation to access key names.

> Note: Reserved key names that cannot be accessed using dot notation can be accessed via standard python dictionary notation. (e.g. `_.foo["get"]` instead of `_.foo.get`)
Expand Down Expand Up @@ -110,10 +110,12 @@ You can print a grep-able schema by using the `-s` option:
```bash
echo '{"foo":"bar","baz":[1,2,3]}' | jello -s

.foo = "bar";
.baz[0] = 1;
.baz[1] = 2;
.baz[2] = 3;
_ = {};
_.foo = "bar";
_.baz = [];
_.baz[0] = 1;
_.baz[1] = 2;
_.baz[2] = 3;
```

#### Assigning Results to a Bash Array
Expand Down Expand Up @@ -160,87 +162,87 @@ Here is more [Advanced Usage](https://github.com/kellyjonbrazil/jello/blob/maste
```bash
$ jc -a | jello -s

. = {};
.name = "jc";
.version = "1.17.2";
.description = "JSON CLI output utility";
.author = "Kelly Brazil";
.author_email = "kellyjonbrazil@gmail.com";
.website = "https://github.com/kellyjonbrazil/jc";
.copyright = "© 2019-2021 Kelly Brazil";
.license = "MIT License";
.parser_count = 80;
.parsers = [];
.parsers[0] = {};
.parsers[0].name = "acpi";
.parsers[0].argument = "--acpi";
.parsers[0].version = "1.2";
.parsers[0].description = "`acpi` command parser";
.parsers[0].author = "Kelly Brazil";
.parsers[0].author_email = "kellyjonbrazil@gmail.com";
.parsers[0].compatible = [];
.parsers[0].compatible[0] = "linux";
.parsers[0].magic_commands = [];
.parsers[0].magic_commands[0] = "acpi";
.parsers[1] = {};
.parsers[1].name = "airport";
.parsers[1].argument = "--airport";
.parsers[1].version = "1.3";
_ = {};
_.name = "jc";
_.version = "1.17.2";
_.description = "JSON CLI output utility";
_.author = "Kelly Brazil";
_.author_email = "kellyjonbrazil@gmail.com";
_.website = "https://github.com/kellyjonbrazil/jc";
_.copyright = "© 2019-2021 Kelly Brazil";
_.license = "MIT License";
_.parser_count = 80;
_.parsers = [];
_.parsers[0] = {};
_.parsers[0].name = "acpi";
_.parsers[0].argument = "--acpi";
_.parsers[0].version = "1.2";
_.parsers[0].description = "`acpi` command parser";
_.parsers[0].author = "Kelly Brazil";
_.parsers[0].author_email = "kellyjonbrazil@gmail.com";
_.parsers[0].compatible = [];
_.parsers[0].compatible[0] = "linux";
_.parsers[0].magic_commands = [];
_.parsers[0].magic_commands[0] = "acpi";
_.parsers[1] = {};
_.parsers[1].name = "airport";
_.parsers[1].argument = "--airport";
_.parsers[1].version = "1.3";
...
```
### Printing the Grep-able Schema with type annotations (useful for grepping types)
```bash
jc dig example.com | jello -st

. = []; // (array)
.[0] = {}; // (object)
.[0].id = 55316; // (number)
.[0].opcode = "QUERY"; // (string)
.[0].status = "NOERROR"; // (string)
.[0].flags = []; // (array)
.[0].flags[0] = "qr"; // (string)
.[0].flags[1] = "rd"; // (string)
.[0].flags[2] = "ra"; // (string)
.[0].query_num = 1; // (number)
.[0].answer_num = 1; // (number)
.[0].authority_num = 0; // (number)
.[0].additional_num = 1; // (number)
.[0].opt_pseudosection = {}; // (object)
.[0].opt_pseudosection.edns = {}; // (object)
.[0].opt_pseudosection.edns.version = 0; // (number)
.[0].opt_pseudosection.edns.flags = []; // (array)
.[0].opt_pseudosection.edns.udp = 4096; // (number)
.[0].question = {}; // (object)
.[0].question.name = "example.com."; // (string)
.[0].question.class = "IN"; // (string)
.[0].question.type = "A"; // (string)
.[0].answer = []; // (array)
.[0].answer[0] = {}; // (object)
.[0].answer[0].name = "example.com."; // (string)
.[0].answer[0].class = "IN"; // (string)
.[0].answer[0].type = "A"; // (string)
.[0].answer[0].ttl = 46950; // (number)
.[0].answer[0].data = "93.184.216.34"; // (string)
.[0].query_time = 33; // (number)
.[0].server = "192.168.1.254#53(192.168.1.254)"; // (string)
.[0].when = "Fri Nov 26 13:17:38 PST 2021"; // (string)
.[0].rcvd = 56; // (number)
.[0].when_epoch = 1637961458; // (number)
.[0].when_epoch_utc = null; // (null)
_ = []; // (array)
_[0] = {}; // (object)
_[0].id = 23819; // (number)
_[0].opcode = "QUERY"; // (string)
_[0].status = "NOERROR"; // (string)
_[0].flags = []; // (array)
_[0].flags[0] = "qr"; // (string)
_[0].flags[1] = "rd"; // (string)
_[0].flags[2] = "ra"; // (string)
_[0].query_num = 1; // (number)
_[0].answer_num = 1; // (number)
_[0].authority_num = 0; // (number)
_[0].additional_num = 1; // (number)
_[0].opt_pseudosection = {}; // (object)
_[0].opt_pseudosection.edns = {}; // (object)
_[0].opt_pseudosection.edns.version = 0; // (number)
_[0].opt_pseudosection.edns.flags = []; // (array)
_[0].opt_pseudosection.edns.udp = 4096; // (number)
_[0].question = {}; // (object)
_[0].question.name = "example.com."; // (string)
_[0].question.class = "IN"; // (string)
_[0].question.type = "A"; // (string)
_[0].answer = []; // (array)
_[0].answer[0] = {}; // (object)
_[0].answer[0].name = "example.com."; // (string)
_[0].answer[0].class = "IN"; // (string)
_[0].answer[0].type = "A"; // (string)
_[0].answer[0].ttl = 48358; // (number)
_[0].answer[0].data = "93.184.216.34"; // (string)
_[0].query_time = 46; // (number)
_[0].server = "2600:1700:bab0:d40::1#53(2600:1700:bab0:d40::1)"; // (string)
_[0].when = "Mon Nov 29 09:41:11 PST 2021"; // (string)
_[0].rcvd = 56; // (number)
_[0].when_epoch = 1638207671; // (number)
_[0].when_epoch_utc = null; // (null)
```
### Printing the Structure of the JSON
```bash
jc dig example.com | jello -st | grep '(object)\|(array)'

. = []; // (array)
.[0] = {}; // (object)
.[0].flags = []; // (array)
.[0].opt_pseudosection = {}; // (object)
.[0].opt_pseudosection.edns = {}; // (object)
.[0].opt_pseudosection.edns.flags = []; // (array)
.[0].question = {}; // (object)
.[0].answer = []; // (array)
.[0].answer[0] = {}; // (object)
_ = []; // (array)
_[0] = {}; // (object)
_[0].flags = []; // (array)
_[0].opt_pseudosection = {}; // (object)
_[0].opt_pseudosection.edns = {}; // (object)
_[0].opt_pseudosection.edns.flags = []; // (array)
_[0].question = {}; // (object)
_[0].answer = []; // (array)
_[0].answer[0] = {}; // (object)
```
### Lambda Functions and Math
```bash
Expand Down
2 changes: 1 addition & 1 deletion jello/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""jello - query JSON at the command line with python syntax"""


__version__ = '1.4.5'
__version__ = '1.4.6'
AUTHOR = 'Kelly Brazil'
WEBSITE = 'https://github.com/kellyjonbrazil/jello'
COPYRIGHT = '© 2020-2021 Kelly Brazil'
Expand Down
38 changes: 4 additions & 34 deletions jello/lib.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -139,28 +139,12 @@ def create_schema(self, data):
self._schema_list *= 0
return myschema

def _schema_gen(self, src, path=''):
def _schema_gen(self, src, path='_'):
"""
Creates a grep-able schema representation of the JSON.
This method is recursive, and output is stored within self._schema_list (list).
"""
if isinstance(src, list) and path == '':
# print empty brackets as first list definition
val = '[]'
val_type = ''
padding = ''
if opts.types:
val_type = '// (array)'
padding = ' '
if len(path) + len(val) + len(val_type) < 76:
padding = ' ' * (75 - (len(path) + len(val) + len(val_type)))

self._schema_list.append(f'. = {val};{padding}{val_type}')

for i, item in enumerate(src):
self._schema_gen(item, path=f'.[{i}]')

elif isinstance(src, list):
if isinstance(src, list):
# print empty brackets as first list definition
val = '[]'
val_type = ''
Expand All @@ -178,7 +162,6 @@ def _schema_gen(self, src, path=''):

elif isinstance(src, dict):
# print empty curly brackets as first object definition
path = path or '.'
val = '{}'
val_type = ''
padding = ''
Expand All @@ -189,12 +172,9 @@ def _schema_gen(self, src, path=''):
padding = ' ' * (76 - (len(path) + len(val) + len(val_type)))

self._schema_list.append(f'{path} = {val};{padding}{val_type}')
if path == '.':
path = ''

for k, v in src.items():
if isinstance(v, list):

# print empty brackets as first list definition
val = '[]'
val_type = ''
Expand Down Expand Up @@ -296,17 +276,7 @@ def create_json(self, data):
if not opts.lines:
return json.dumps(data, separators=separators, indent=indent, ensure_ascii=False)

# check if this list includes lists
list_includes_list = False
for item in data:
if isinstance(item, list):
list_includes_list = True
break

if opts.lines and list_includes_list:
raise ValueError('Cannot print list of lists as lines. Try normal JSON output.')

# print lines for a flat list
# print lines
else:
flat_list = ''
for entry in data:
Expand All @@ -316,7 +286,7 @@ def create_json(self, data):
else:
flat_list += '\n'

elif isinstance(entry, (dict, bool, int, float)):
elif isinstance(entry, (dict, list, bool, int, float)):
flat_list += json.dumps(entry, separators=separators, ensure_ascii=False) + '\n'

elif isinstance(entry, str):
Expand Down
80 changes: 38 additions & 42 deletions jello/man/jello.1
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH jello 1 2021-11-26 1.4.5 "Jello JSON Filter"
.TH jello 1 2021-11-29 1.4.6 "Jello JSON Filter"
.SH NAME
Jello \- Filter JSON and JSON Lines data with Python syntax
.SH SYNOPSIS
Expand Down Expand Up @@ -126,10 +126,12 @@ You can print a grep-able schema by using the \fB-s\fP option:
.nf

$ echo \[aq]{\[dq]foo\[dq]:\[dq]bar\[dq],\[dq]baz\[dq]:[1,2,3]}\[aq] | jello -s
\&.foo = \[dq]bar\[dq];
\&.baz[0] = 1;
\&.baz[1] = 2;
\&.baz[2] = 3;
\&_ = {};
\&_.foo = \[dq]bar\[dq];
\&_.baz = [];
\&_.baz[0] = 1;
\&_.baz[1] = 2;
\&_.baz[2] = 3;

.fi
.SS Assigning Results to a Bash Array
Expand Down Expand Up @@ -162,23 +164,17 @@ done < <(cat data.json | jello -rl _.foo)
.nf

$ jc -a | jello -s
\&.name = \[dq]jc\[dq];
\&.version = \[dq]1.10.2\[dq];
\&.description = \[dq]jc cli output JSON conversion tool\[dq];
\&.author = \[dq]Kelly Brazil\[dq];
\&.author_email = \[dq]kellyjonbrazil\[at]gmail.com\[dq];
\&.parser_count = 50;
\&.parsers[0].name = \[dq]airport\[dq];
\&.parsers[0].argument = \[dq]--airport\[dq];
\&.parsers[0].version = \[dq]1.0\[dq];
\&.parsers[0].description = \[dq]airport -I command parser\[dq];
\&.parsers[0].author = \[dq]Kelly Brazil\[dq];
\&.parsers[0].author_email = \[dq]kellyjonbrazil\[at]gmail.com\[dq];
\&.parsers[0].compatible[0] = \[dq]darwin\[dq];
\&.parsers[0].magic_commands[0] = \[dq]airport -I\[dq];
\&.parsers[1].name = \[dq]airport_s\[dq];
\&.parsers[1].argument = \[dq]--airport-s\[dq];
\&.parsers[1].version = \[dq]1.0\[dq];
\&_ = {};
\&_.name = "jc";
\&_.version = "1.17.2";
\&_.description = "JSON CLI output utility";
\&_.author = "Kelly Brazil";
\&_.author_email = "kellyjonbrazil@gmail.com";
\&_.website = "https://github.com/kellyjonbrazil/jc";
\&_.copyright = "(C) 2019-2021 Kelly Brazil";
\&_.license = "MIT License";
\&_.parser_count = 80;
\&_.parsers = [];
\&...

.fi
Expand All @@ -187,17 +183,17 @@ $ jc -a | jello -s
.nf

$ jc -a | jello -st
\&. = {}; // (object)
\&.name = "jc"; // (string)
\&.version = "1.17.2"; // (string)
\&.description = "JSON CLI output utility"; // (string)
\&.author = "Kelly Brazil"; // (string)
\&.author_email = "kellyjonbrazil@gmail.com"; // (string)
\&.website = "https://github.com/kellyjonbrazil/jc"; // (string)
\&.copyright = "(C) 2019-2021 Kelly Brazil"; // (string)
\&.license = "MIT License"; // (string)
\&.parser_count = 80; // (number)
\&.parsers = []; // (array)
\&_ = {}; // (object)
\&_.name = "jc"; // (string)
\&_.version = "1.17.2"; // (string)
\&_.description = "JSON CLI output utility"; // (string)
\&_.author = "Kelly Brazil"; // (string)
\&_.author_email = "kellyjonbrazil@gmail.com"; // (string)
\&_.website = "https://github.com/kellyjonbrazil/jc"; // (string)
\&_.copyright = "(C) 2019-2021 Kelly Brazil"; // (string)
\&_.license = "MIT License"; // (string)
\&_.parser_count = 80; // (number)
\&_.parsers = []; // (array)
\&...

.fi
Expand All @@ -206,15 +202,15 @@ $ jc -a | jello -st
.nf

$ jc dig example.com | jello -st | grep '(object)\e|(array)'
\&. = []; // (array)
\&.[0] = {}; // (object)
\&.[0].flags = []; // (array)
\&.[0].opt_pseudosection = {}; // (object)
\&.[0].opt_pseudosection.edns = {}; // (object)
\&.[0].opt_pseudosection.edns.flags = []; // (array)
\&.[0].question = {}; // (object)
\&.[0].answer = []; // (array)
\&.[0].answer[0] = {}; // (object)
\&_ = []; // (array)
\&_[0] = {}; // (object)
\&_[0].flags = []; // (array)
\&_[0].opt_pseudosection = {}; // (object)
\&_[0].opt_pseudosection.edns = {}; // (object)
\&_[0].opt_pseudosection.edns.flags = []; // (array)
\&_[0].question = {}; // (object)
\&_[0].answer = []; // (array)
\&_[0].answer[0] = {}; // (object)
\&...

.fi
Expand Down

0 comments on commit 3ed41ab

Please sign in to comment.