Rules serialization to/from JSON

[snikolayev]

Started working on a spike for ability to serialize rules to/read from JSON. I wanted to use this discussion to gauge interest around this feature.
https://github.com/NRules/NRules/tree/feature/serialization-json

[a046]

A challenge with JSON serialization is that many users that want to store rules in a datastore are probably also wanting to write them outside of Visual Studio in some way. I think the actual JSON format would be heavily influenced by how they want to do that. A limited front end would work well with any format, but anything more fully featured might demand a bespoke serialization format. But I think out of the box support would still be nice as it would be less overwhelming for people to make their own front ends on top of the standard JSON format.

Having first class support for JSON serialization in NRules would also be a first step towards allowing serialization and reuse of compiled rete networks I think too which if fast enough would be great for large rule sets, irrespective of how those rules are written.

[snikolayev]

@a046 thanks for your thoughts on this. Very helpful.

I think ultimately, rules need to map to CLR. And I think at the moment, expression trees are the best mechanism to represent arbitrary logic as data, as long as it operates within the .NET type system. Then the rules build on top of those expressions, by adding a higher-level structure of patterns, aggregations, action, etc. So, with the JSON serialization my intention is to preserve those constructs, such that it can more or less serialize a rule definition with all its lambda expressions verbatim. Then I can see two extremes in terms of using this format. One extreme is where you utilize the rule elements, but mostly just limit your expression trees to method/property calls on custom CLR types that represent facts. This can easily lend itself to rule writing UIs that inspect custom types via reflection and allow rule composition by using members of those types. The other extreme is utilizing a full-fledged programming model, operating on custom fact types and primitive types alike, and fully defining the rule logic within the rule expressions. As long as that logic can be mapped to expression trees, it should still be possible (though definitely a lot more work) to write a UI for it. My thinking is that as long as JSON serialization can fully support expression trees serialization in the .NET framework, it should be possible to adopt to most use cases.
And TBC, rule writing UIs is definitely a use case this will open up, but I think there are other interop scenarios this could enable (e.g. converting non-code DSLs to rules via JSON intermediary, such as Excel decision tables, etc.).

The other thing you mention about compiled rete serialization - there is another thread on this forum that made me think about this. You may want to take a look. It's in the context of AOT compilation, for runtimes that don't allow on-the-fly IL generation. I think it's less about interoperability and extensibility (as is the case with rules serialization), but more about efficiency and performance - pre-compiling rules to target such runtimes or even to optimize code for regular .NET runtimes. Or did you have some other use cases in mind?

[a046]

Yeah I definitely agree that a serialization format close to the in memory setup is the best approach.

Regarding compiled rete serialization, I was definitely thinking of efficiency and performance within the same runtime - I think being able to serialize expressions is a large part of that. The iOS AOT case in that other thread is a bit interesting, but it sounds more complicated.

[jacbar]

Having ability to dump/load compiled rete would be great achievement. It would allow us to migrate from aws ecs to aws lambda and load compiled rules i.e. from ElastiCache. Currently our set of rules is quite big and loading it per lambda is time consuming.

[MadSciencist]

This would be game changer for me. Any ETA?

[snikolayev]

I’m shooting to get it out in June. It’s already functional on a feature branch, if you want to check it out: https://github.com/NRules/NRules/tree/feature/serialization-json

And you can see usage examples here: https://github.com/NRules/NRules/blob/feature/serialization-json/src/NRules/Tests/NRules.Json.Tests/RuleSerializerTest.cs

[MadSciencist]

I was actually thinking of serializing the compiled session/session factory. I assume this is not in scope of this discussion and it's not easily achievable?

[snikolayev]

@MadSciencist I think what you are describing is closer to #264.
I think it’s achievable, but definitely way outside the scope of this feature

[rlightner]

@snikolayev Should we look to use that branch still or is it close to being merged in?

[snikolayev]

@rlightner it’s taking a bit longer than I originally anticipated. Will merge it in the nearest future, but it’s not merged yet. So you can indeed use the feature branch in the meantime.

[snikolayev]

Merged JSON serialization feature to develop branch: #284

[snikolayev]

Released in https://github.com/NRules/NRules/releases/v0.9.3