Auto generate from grafana schema?

[fzyzcjy]

Hi thanks for the lib! I wonder whether it is reasonable to be auto-generated from the grafana schema https://github.com/grafana/grafana/blob/main/kinds/dashboard/dashboard_kind.cue. By doing that, we can, for example:

1. Always be up-to-date, no missing fields
2. Allow importing a dashboard JSON into grafanalib (for further customization)? #608 will be much easier

[alkuzad]

Grafana CUE/Thema is cute but highly experimental and not working. What you linked does not even have panel schemas, just general dashboard one Correction, it has them. The whole reason (I believe) grafanalib exists is that jsonnet is super hard to write and almost noone did. Replacements are not yet ready even if hints were given like 1,5year ago that this is coming.

[fzyzcjy]

@alkuzad Thank you for the information. I have implemented (since hours ago) using auto-generation and minor manual modifications: https://github.com/fzyzcjy/grafana-dashboard-python

The whole reason (I believe) grafanalib exists is that jsonnet is super hard to write and almost noone did. Replacements are not yet ready even if hints were given like 1,5year ago that this is coming.

I also do not use jsonnet, and love to use existing toolkits around python (huge ecosystem!)

[alkuzad]

@fzyzcjy It is a nice experiment but I do not believe in the stage for use yet, given the amount of manual changes needed. I saw also one additional problem - I use InfluxDB but Grafana/Grok does not have it and Grafanalib does.

[fzyzcjy]

I do not believe in the stage for use yet, given the amount of manual changes needed

Yes I think so.

I use InfluxDB but Grafana/Grok does not have it and Grafanalib does.

Hmm, where is that schema from? Is it by manual reverse engineering, or the JSON schema is published outside grok?

[alkuzad]

Hmm, where is that schema from? Is it by manual reverse engineering, or the JSON schema is published outside grok?

I just think it was not ported from the older grafonnet-lib. https://github.com/grafana/grafonnet-lib/blob/master/grafonnet/influxdb.libsonnet

[fzyzcjy]

I see. But I guess it is very easy to solve, if all the code is in https://github.com/weaveworks/grafanalib/blob/main/grafanalib/influxdb.py.

We only need to write a Pydantic class Influxdb mimicking that - which can be done in 5 minutes :)

[fzyzcjy]

but I do not believe in the stage for use yet, given the amount of manual changes needed

Btw what manual changes do you have in mind? I have already changed many things, but most are just adding default values and making type constraints looser, so they are quite straightforward.

[alkuzad]

but I do not believe in the stage for use yet, given the amount of manual changes needed

Btw what manual changes do you have in mind? I have already changed many things, but most are just adding default values and making type constraints looser, so they are quite straightforward.

https://github.com/fzyzcjy/grafana-dashboard-python/blob/master/misc/model_generation/manual_patch.patch this, it looked to me like a lot of tuning applied over auto-generated schema.

[fzyzcjy]

Yes and no. It is mostly just

1. adding default values, s.t. no need to specify a ton of fields when using it. (but it is fairly easy to PR and fix if you see a field that is mandatory but is reasonable to have a default value)
2. loosing types, s.t. it allows more types than defined in schema. (also trivial to PR, since if you see a type gets rejected but you think it should be good, just change it from SomeType to Union[SomeType, SomeMoreType] and PR)

[fzyzcjy]

@alkuzad By the way, did you find Grafanalib missing some fields once in a while? I did have that problem sometimes, and has to modify its source code.

[alkuzad]

@fzyzcjy I am not an extensive user yet, I am experimenting with dashboard.json generation and especially converting it from the raw export. trimDefaults seems to not work and I want to figure out how to convert dashboard.json raw form into more easy to read one :)

[K-Phoen]

As a quick side-note: a library fully-generated from schemas is being worked on by Grafana and published here https://github.com/grafana/grafana-foundation-sdk/ :)