Demo: structured data over CA

[tacaswell]

I finally did the work to do a proof of principle of what I proposed at the EPICS meeting in Australia in 2018 to get structured data over CA.

This is making use of a bytes channel, but in principle we could go as far as getting a new type(s) for various schemes of encoding.

The three examples:

• using msgspec to build an IOC that publishes a jsonspec + json as a bytestring. This is good because it gets easy interop with the web world and it should be an easy but the downside that it is going through a text encoding so that is not great for (big) arrays.
• using msgpack + fancy encoders to get numpy arrays in. This is used to mimic something that looks like a table (e.g. flyscan trajectories + trigger conditions). The underlying structure of the numpy array encoding is dtype (as str), shape, and the array buffer so it should in principle be possible to interpret it in non-Python schemes. An alternate here that may be better (and more accessible to other languages) is to use arrow chunks, but msgpack + numpy is what I was playing with.
• using protobuf to encode frame number + image. Again it is using a very simple encoding of the exact numpy dtype, shape, and array buffer. There may be better ways to encode imegas (and more cross-language ways!), but this was enough to get it to work. Using protobuf gets us way more cross-language support out of the box (at the cost of having to pre-declare the message models

[tacaswell]

An obvious improvement is to call the encoding channel "mimetype" and make the values the correct mimetypes.