Code generation via text templates

[magick93]

Hello

I'm currently relying heavily on CUE and Hof (https://github.com/hofstadter-io/hof) for generating source code.

I've recently discovered kcl-lang, and am very impressed by the project for many reasons.

I'd like to suggest that, while there is obvious value in applying kcl to configuration, it can also be used to generate source code. Consider it like uml - it can describe ever aspect of source code - and then, when combined with a text generator, be used to generate source code and artifacts.

For example, generating:

• database ddl
• svelte / react / vue web components
• finite state machines
• etc etc

To achieve this it would be good if there was ability to generate kcl models from existing models, for example, import json, jsonschema, yaml, and ideally cue.
Similarly, export structs in Rust, Go, Java etc, for use in existing templating.
But also a native templating system would also be good.

[Peefy]

Thank you for your attention! ❤️

In fact, we are working hard to complete the relevant KCL import tool so that KCL can communicate with other schema ecosystems. We have not officially released it to the public yet, but we will officially launch it by the end of this month. The schemas we have or are currently supporting include Go, Protobuf and Jsonschema. Ref: https://github.com/kcl-lang/kcl-go/tree/main/pkg/tools/gen

We will further support Json, YAML, Terraform Provider Schema, etc., and we will abstract them out a native templating system. Stay tuned.

[magick93]

Thanks @Peefy - looks very exciting!

[MirKml]

Yes, some built-in templating for plain text generation, this is what I miss the most.

We have many configs which aren't yaml/JSON structured e.g. Apache httpd configs, lot of ini files, XML.
As I read documentation - nice and very useful - only string placeholders, format function, are possible now.

[Peefy]

Yes, some built-in templating for plain text generation, this is what I miss the most.

We have many configs which aren't yaml/JSON structured e.g. Apache httpd configs, lot of ini files, XML. As I read documentation - nice and very useful - only string placeholders, format function, are possible now.

Hi there @MirKml . Thank you for your attention. If you have any needs, please feel free to contact me. ❤️

I would like to further understand whether we can use KCL to render or template these non JSON/YAML formats, and what are your requirements?

[MirKml]

As I said, we have many non JSON/YAML formats, for example
apache httpd config it's like this https://gist.github.com/sterndata/f27fb487ea9442e02379a668e10cd08b for frontend microservices.
We have custom bash (with awk, sed) scripts, which makes the "templating" - these replaces some strings here and there in such configuration and generates the final web server configuration for particular customer, environment (dev, test, production ...).

We have also some "configuration" frontend typescript (not json) code, which is "templated" same way. This is similar case as @magick93 said.

We have also templated ".npmrc" https://docs.npmjs.com/cli/v9/configuring-npm/npmrc. which is "ini file".

Many software components has the configuration in non YAML formats. Other "traditional" example is common unix/linux DNS server "named" https://docs.oracle.com/cd/E19683-01/816-7511/6mdgu0h00/index.html. There "templating" is used for generating customer domains.

Most above configs are used later as configmaps. Not all, but many
We want get rid of these bash, sed, templating and use KCL with its benefits like schemas, validation ..., for all our configuration, not only YAML, JSON.

Btw I didn't see how it's possible generate JSON. Everywhere is YAML. I think Kcl binary has no option for generating JSON. It's necessary to use encode/decode func. But this is out of this discussion scope.

[Peefy]

As I said, we have many non JSON/YAML formats, for example apache httpd config it's like this https://gist.github.com/sterndata/f27fb487ea9442e02379a668e10cd08b for frontend microservices. We have custom bash (with awk, sed) scripts, which makes the "templating" - these replaces some strings here and there in such configuration and generates the final web server configuration for particular customer, environment (dev, test, production ...).

We have also some "configuration" frontend typescript (not json) code, which is "templated" same way. This is similar case as @magick93 said.

We have also templated ".npmrc" https://docs.npmjs.com/cli/v9/configuring-npm/npmrc. which is "ini file".

Many software components has the configuration in non YAML formats. Other "traditional" example is common unix/linux DNS server "named" https://docs.oracle.com/cd/E19683-01/816-7511/6mdgu0h00/index.html. There "templating" is used for generating customer domains.

Most above configs are used later as configmaps. Not all, but many We want get rid of these bash, sed, templating and use KCL with its benefits like schemas, validation ..., for all our configuration, not only YAML, JSON.

Btw I didn't see how it's possible generate JSON. Everywhere is YAML. I think Kcl binary has no option for generating JSON. It's necessary to use encode/decode func. But this is out of this discussion scope.

OK, I see. At present, KCL has specific structured representations implemented internally, and we can serialize it into JSON/YAML. We will support the format parameter in the future to configure more formats such as using KCL to output toml/ini, and the cost is not high. In addition, KCL supports Python plugins for interoperability with KCL, and we hope it can help you. Of course, we will continue to improve the template tool mentioned in this discussion.

[MirKml]

What will be the best for us - as I said - universal built-in templating like razor, go-templates, even PHP ... but with native KCL

sample.k

import math

_calc_result = math.log10(100) # 2

tmpl
this is the plain the output which will be printed out by default, it can use {{_calc_result}} from code behind etc.
{{ now() }} for printing any function result directly etc.
<<<templ
anotherFunction()
anotherKclCode()

tmpl
this is next template part
<<<templ

Output after compilation will be result of computed, printed templ parts

Currently it can be done with multiline string with parameters I think. But it's like workaround, not really native templates, with helpers etc.

[Peefy]

OK. I see. Perhaps the following content is the best form? We can use the kcl plugin or builtin module to impl this template render features?

import math
import kcl_plugin.go_template  # Can be other built-in/plugin template libraries, such as Jinja2, Go template, Vercel, PHP, etc etc.

_calc_result = math.log10(100) # 2

templ = go_template.exec("/path/to/go_template", {"_calc_result": _calc_result})

[MirKml]

Yes, your proposal with kcl/built-in plugin for templating is fine (cc @magick93)
One templating plugin should be built-in - it means without necessary to install other external sdk or language runtimes, just kcl binary for compilation.
Others can require go/python/php whatever - but at least one should be default, built-in and always accessible. Maybe some preferred/liked most by rust community 😄. It's up to you.

[Peefy]

Okay, your suggestion for the built-in template package is very good. I will consider it and provide it in the next major version. ❤️ The issue is here: #313