NestedTextTo, CLI conversions with type casting

[AndydeCleyre]

Hello!

Thanks so much for NestedText, I'm a big fan.

So I made a convenient CLI for converting between it and JSON, YAML, and TOML, with concise type casting (thanks to yamlpath and cattrs especially).

I'm wondering if any Kunderts or other NestedText users would provide any feedback, and if it might get listed as a related project.

The project is on PyPI as nt2, and for TOML support should be installed as nt2[toml].

It may be invoked as nt2json, nt2yaml, nt2toml, json2nt, yaml2nt, or toml2nt. More details can be found at the repo itself.

Thanks for any response!

[kalekundert]

That looks like a really nice tool to me! I tried it out on some of my files and it worked flawlessly. I'll leave it to @KenKundert to update the docs, but thanks for sharing!

[LewisGaul]

Related: my zig-nestedtext project provides a nt-cli tool for converting to/from JSON and should be extremely performant. Static binaries can be downloaded from the releases to try it out, in case you're interested in comparing. https://github.com/LewisGaul/zig-nestedtext/tree/zig-master#cli-tool

[KenKundert]

Looks like a nice package. I have added it as a related project in the documentation. I also added a comment about Lewis's nt-cli to the documentation.
As for some feedback: While I am not a fan of stack traces for normal user errors, I must say, yours are the most attractive stack traces I have ever seen.

[AndydeCleyre]

Ah, the stacktraces are courtesy of rich, and only take that form if it happens to be installed in the environment (it's only a development requirement). Otherwise they're ugly/classic. Thanks! I'll try to reduce error output.

[KenKundert]

Which version of NestedText are you supporting?

[AndydeCleyre]

The reference Python NestedText implementation is used, currently tested and developed with package release 3.4.0, but installing with pip does not currently limit which package version is pulled in (or chosen by the user).

[kalekundert]

One other thought I just had, which you should feel free to take or leave: the name nt2 might give the impression that this is some "second version" of nested text, as opposed to a tool to convert between nestedtext and other formats. Maybe a name like nt2any, nt2xxx, nt2___, nt2other, or something like that would be more clear.

[AndydeCleyre]

I get this, and it does bother me a little. I just don't like any alternatives enough to change it.

[KenKundert]

Perhaps the descriptive nt-converters or nestedtext-converters.

[KenKundert]

I have some suggestions as well.

I wrote some programs that are similar to yours and put them in the examples directory:

nestedtext-to-json   json-to-nestedtext
nestedtext-to-yaml   yaml-to-nestedtext
                     toml-to-nestedtext
                     csv-to-nestedtext

I think the JSON translators are pretty good, but the others are after thoughts I put little effort in. You might want to take a look at the error handling in the JSON translators for guidance on how to improve your error handling.

One translator I had in the back of my mind to write but never did was a Python translator. It is not uncommon to use Python to hold configuration information and data. Including a Python translator might fit in well with the translators you have already implemented.

[AndydeCleyre]

I'm now pretty happy with the error handling, but please do let me know if there's a particular error case that presents in an annoyingly unclear way, still.

The python format idea is interesting, but I don't know what types ought to be supported (I'd start with the combination of all already supported types), or if it should be supported as an input format as well as output.

[KenKundert]

Andyde, I would appreciate it if you would consider doing at least a minor release. I feel a little uncomfortable recommending a package with a version number htat starts with 0.0. Of course you should only do that if your feel its ready to be used by others. If you feel that is not the case, let me know and I will remove the pointer for the time being.

[AndydeCleyre]

Sure, if no one here notices any glaring problems, I'll just try to put better usage examples in the readme and do a minor release later today.

[KenKundert]

No hurry.

[AndydeCleyre]

0.1.0 is out and ready for bug reports. As for the name, the project is now more definitely "NestedTextTo" not "nt2", but will still use "nt2" as the PyPI package name. None of the URLs change, but I ask/suggest the (nt2) bit be dropped from the name listing in the related_projects doc. Thanks!

[KenKundert]

Done. Thanks!

[AndydeCleyre]

NestedTextTo now provides automatic generation of the simple "schema" types files, for easier type-preserving round-tripping.

sample.toml:

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00

[database]
enabled = true
ports = [ 8000, 8001, 8002 ]
data = [ ["delta", "phi"], [3.14] ]
temp_targets = { cpu = 79.5, case = 72.0 }

[servers]

[servers.alpha]
ip = "10.0.0.1"
role = "frontend"

[servers.beta]
ip = "10.0.0.2"
role = "backend"

$ toml2nt sample.toml >sample.nt
$ toml2nt sample.toml --to-schema >sample.types.nt
$ nt2toml sample.nt --schema sample.types.nt

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27 07:32:00-08:00

[database]
enabled = true
ports = [
    8000,
    8001,
    8002,
]
data = [
    [
        "delta",
        "phi",
    ],
    [
        3.14,
    ],
]

[database.temp_targets]
cpu = 79.5
case = 72

[servers.alpha]
ip = "10.0.0.1"
role = "frontend"

[servers.beta]
ip = "10.0.0.2"
role = "backend"

$ cat sample.types.nt

date:
  - owner.dob
boolean:
  - database.enabled
number:
  - database.ports[0]
  - database.ports[1]
  - database.ports[2]
  - database.data[1][0]
  - database.temp_targets.cpu
  - database.temp_targets.case

# Above is a schema that literally matches the current data.
# Below, for your review, is a guess at a better schema.

# date:
#   - owner.dob
# boolean:
#   - database.enabled
# number:
#   - database.data
#   - database.temp_targets.case
#   - database.ports
#   - database.temp_targets.cpu

I still need to improve the commented suggested "better" schema; in this example, that one fails to properly match the database ports and the 3.14 value.

[KenKundert]

Nice!