Defining Media Types for the textual representations

[ghys]

Hello @openhab/architecture-council!

With this I'm seeking a discussion and eventual decision on the subject of defining official (note: not "really" official as in IANA-registered... for now! rather "generally accepted") Media Types - formerly MIME types - for the textual representations of openHAB objects and entities.

The reason for this is mainly because my pet project of late (today, really) has been to explore adding some "hint" mechanism, aka. Intellisense or autocomplete, to some of the CodeMirror editors in the UI to aid in writing structures and code in these.

Example:

For now I'm focused on the pages/UI components YAML structures, but I'm eager to expand this - not necessarily myself btw, perhaps others could contribute...? - to all types of "code" we can edit in the UI: rule YAML representations, things, items... but to do this we need to identify exactly what we're talking about, hence the expansion of media types.

Following discussions with @kaikreuzer we have already informally defined the application/vnd.openhab.dsl.rule media type for the rule DSL - it's recognized by the rule engine as a language for the "run a script" actions and conditions.

I'd like to propose expanding this to for instance (here's where I seek your approval):

• application/vnd.openhab.items-definition - for textually defined items (multiple, as in a text file with several of them)
• application/vnd.openhab.things-definition - for textually defined things (ditto)
• application/vnd.openhab.sitemap-definition - for a (singular) textually defined sitemap

Now as you may have noticed, the OH3 UI offers a YAML representation of a rule in the new rule engine:

For these I think a suffix is necessary because it's in essence an object representation which could be serialized using any syntax. For instance these rules could be stored in JSON format in the JSON DB or represented in YAML in the UI. While +json is officially recognized in the IANA's Structured Syntax Suffix Registry (and RFC 6839), +yaml is not, although it seems to be commonly adopted. I propose to adopt it anyway.

If we were to define the application/vnd.openhab.rule-definition media type for the structured definition of a rule, the YAML representation of it would therefore be:

application/vnd.openhab.rule-definition+yaml

In the same manner, "UI components" (simple hierarchical structures mostly comprising a type, configuration and "slots") are stored as JSON and represented as YAML, however the nature of their "root components" could help identifying some context, for that I suggest we use parameters to the media type to give these indications. I don't want to overwhelm you with specifics, but we could have different types of pages as root components (layout-page, chart-page, widget), their media types might therefore look like this:

application/vnd.openhab.uicomponent-definition+yaml?type=widget

What do you think? Looking forward to your feedback!

[rkoshak]

As and end user I don't really have an opinion on this one. The proposal seems reasonable but I can't comment on the implications and it largely looks like it would not be something end users have to worry about. But it does sound like it would make new features possible in the UI which would be good, right?

[ghys]

Yeah, I wasn't sure where to file this since it seemed to be a project-wide decision. Nobody seems interested though...

Still I used the above anyway until it's challenged and worked a bit on expanding the autocompletion to rules in openhab/openhab-webui#364 so it's easier to add modules to these UI-driven rules in code.

[kaikreuzer]

Nobody seems interested though...

Hey, don't be disappointed so easily 😆 . As I had previously discussed the rule dsl media type, you know that I care!
I think your proposal make sense and I guess you are the best expert from all of us to judge on the best naming conventions. What I was wondering is whether the "-definition" part is really necessary? Are others doing that as well in their names? I understand that you want to differentiate between the dsl and the object-representation, but wouldn't the application/vnd.openhab.dsl.rule vs application/vnd.openhab.rule+yaml be enough for that already?

[ghys]

What I was wondering is whether the "-definition" part is really necessary

I asked myself this very question, I figured it made sense to distinguish the objects themselves (a rule, an item, a thing...) with the syntax used to define them, but now I'm not so sure... and the fact that you brought it up too probably is a good hint that it doesn't.

So to recap we'd have (after fixing the parameter syntax and adding the persistence which I forgot):

Media type & Description	Example
application/vnd.openhab.dsl.rule a domain-specific scripting language used for rules actions (the "then" part)	val newColor = new Color(red, blue, green) MyColorItem.sendCommand(new HSBType(newColor))
application/vnd.openhab.rules File extension: .rules several rules, complete with triggers and scripts in the domain-specific language	when   Item MyItem received update then   MySwitch.sendCommand(ON) ... end when   ... then   ... end ...
application/vnd.openhab.things File extension: .things several things described in their domain-specific syntax	Thing network:device:webcam "Webcam" @ "Living Room" [ hostname="..." ] Thing astro:moon:home [ geolocation="50.12345,10.98765", interval=300 ] Thing ntp:ntp:local [ hostname="de.pool.ntp.org" ] ...
application/vnd.openhab.sitemap File extension: .sitemap a sitemap, described in its domain-specific language	sitemap demo label="My home automation" {   Frame label="Date" {     Text item=Date   }   ... }
application/vnd.openhab.persistence File extension: .persist persistence configuration for the service identified by the file name, described in their domain-specific language	Strategies {   everyHour : "0 0 * * * ?"   everyDay : "0 0 0 * * ?" } Items {   Temperature*, Weather* : strategy = everyHour   ... }
application/vnd.openhab.rule+yaml a rule represented in YAML	uid: myrule name: My rule tags: [] triggers:   - inputs: {}     id: "1"     configuration:       key: value       ...     type: core.ItemCommandTrigger   - ... conditions: [] actions:   - ...
application/vnd.openhab.rule+json a rule represented in JSON may or may not be useful...! Just to illustrate the suffix vs. +yaml	{   "uid": "myrule",   "name": "My rule",   "tags": [],   "triggers": [     {       "inputs": {},       "id": "1",       "configuration": [ ... ],       "type": "core.ItemCommandTrigger"     },     ...   ],   "conditions": [],   "actions": [     ...   ] }
application/vnd.openhab.thing+yaml the editable subset of a Thing represented in YAML (note the channels under the channels array are only those which are extensible i.e. user-defined)	uid: mqtt:topic:thing1 label: My MQTT Thing thingTypeUID: mqtt:topic configuration:   availabilityTopic: thing1/lwt   ... bridgeUID: mqtt:broker:broker1 channels:   - id: temperature     channelTypeUID: mqtt:number     label: Temperature     configuration:       stateTopic: thing1/temperature       ...   - id: ...
application/vnd.openhab.uicomponent+yaml; type=componentType a UI component represented in YAML Type may be: widget, page, layout, chart...	uid: mycomponent props:   parameters:   - description: A text prop     label: Prop 1     name: prop1     required: true     type: TEXT   - ...   parameterGroups: [] timestamp: ... tags: [] component: oh-card config:   key: value   ... slots:   default:     - component: oh-slider       config:         key2: value2         ...       slots:         ...
application/vnd.openhab.uicomponent+json; type=componentType a UI component represented in JSON, may or may not be useful...	cf. above but in JSON :)

At some point I think it would also be useful to provide a way to or modify Things with a text editor in the UI. That would be handy to be able to copy-paste to create new similar things or duplicate channels, with autocompletion etc... The best of course would be to have a parser/stringifier for the .things syntax but I was wondering if YAML (and therefore application/vnd.openhab.thing+yaml) could also be an alternative... ? It would certainly be easier!

[rkoshak]

Definitely +1 for copy/paste/modify for Things. Especially with HTTP and MQTT binding having that capability would be really appreciated. From what I've been told, the modbus binding almost would require this.

For now I recommend people to pull the JSON from the REST API, modify the JSON and resubmit the modified JSON using the REST API to duplicate similar Things and Rules. Alternatively modifying the JSONDB itself (with OH stopped of course).

This is probably the biggest pain point causing users to use .things files over automatic discovery and manually created Things in the JSONDB. Personally I would put this capability relatively high on the list of priorities actually.

[ghys]

I saw your posts and that's part of the reason why I think it would be good to have something that could handle this better. With YAML as the thing description syntax it would be relatively easy to implement and provide a good added value, even if as a principle I'd prefer using the established description languages whenever possible - to try to keep the docs current with no changes if we can.

The thing details page in the new UI was one of the first I designed for the first prototype almost 2 years ago and I'm already thinking about revisiting it, that is, merging the Info & Config tabs together and adding a new Code tab, similarly to many other pages throughout the UI where the last tab is always Code.

[digitaldan]

Hi @ghys , i have a probably silly question, but what endpoints would expose the available media types? Is this the /module-types endpoint? When specifying application/vnd.openhab.uicomponent+yaml for example, is this returned in the response document, indicating it's content is stored and returned as YAML, or is this in the request parameter indicating it should be transformed and outputted as YAML? This is a part of OH core I am not familiar with at all !

[ghys]

No silly question, currently the REST API mostly sets the "generic" JSON media type aka. application/json as the Content-Type header but we might think about changing that if it clearly outputs one of our vendor media types like application/vnd.openhab.rule+json. The practical purpose of coming up with these new media types now is to formally identify those specific outputs, so I can feed them to the CodeMirror editor control when appropriate and have special behavior based on it, like autocompletion or (eventually later) syntax highlighting :)

[digitaldan]

So i ask for ?type=json and the response content type in the HTTP header is application/vnd.openhab.rule+json and the content body is outputted as JSON ? Make sense to me. If this is the case, is there anything that might cause grief with the browser and not using known mime types ? And not to overcomplicate this, but should we just have a yaml output option for all REST requests ? I don't know if that even makes sense :-)

[ghys]

This is not about what the REST API can do, really, it's just a matter of identifying MIME types - now called media types - of the documents (as in sequences of bytes) that we can encounter and are related to openHAB, so when you see such a document, no matter what its source is (a file, or the output of an API) with "when... then... end" structures you can affix the application/vnd.openhab.rules MIME type to it like image/jpeg is the MIME type of a JPEG image.

[ghys]

Now to actually answer your question :) theoretically if you would specify the Accept: application/vnd.openhab.rule+yaml HTTP header to a request to /rest/rules/rule1 then in theory, yes, the API should respond with YAML instead of JSON.

[ghys]

Definitely +1 for copy/paste/modify for Things. Especially with HTTP and MQTT binding having that capability would be really appreciated. From what I've been told, the modbus binding almost would require this.

In case you missed it, I have squeezed that feature in time to have it the milestone release (openhab/openhab-webui#368) so people who start using OH3 with the upcoming release can test drive it. I didn't manage to add the ability to create new things from YAML, but you can edit existing things, their configuration and the (user-defined) channels in the new Code tab; the thing details page now behave quite similarly to the rule details page - you can save with Ctrl+S, disable/re-enable the thing with Ctrl+D, and so on. It doesn't seem to work too badly and hopefully should make the process of building generic MQTT or HTTP things a little less tedious.

I have updated the table above with the new media type application/vnd.openhab.thing+yaml.