[WIP] Add ability to write meta data

[matze]

PR and discussion about writing experiment meta data. Still not ready, need to decide where this should live (in an experiment itself?). Here some open issues

• Decide who is actually responsible for writing (experiment class, the user, ...)
• Generalize for all kinds of experiments.
• Don't write into current directory but use the walker infrastructure.
• What about HDF5. can we write JSON or should we serialize the data differently?
• Storage of external meta data, such as sample description and the like.

[tfarago]

Finally someone has started 😄 We can put it to the experiment in a very loose way, why not.

[MarcusZuber]

I also was thinking about somthing simmilar after I saw how many scans we already had produced...
Maybe the write_metadata function could be more general. It could be possible to pass a list of devices and it writes all parameters. Something simmilar is already implemented in concert.session.utils.ddoc.

Also the possibility to add additional data, that is not created by concert, like sample discription, external filtration etc., would be helpfull.

The folder for the file could be determined by the walker of the experiment.

[matze]

I might add that this is just a very rough draft concerning data @Threadmonkey is most interested in. I will update the PR comment with the most dire requirements.

@MarcusZuber I did something along the lines some time ago in a separate branch (see 8a9aa2d) but to fully generalize that, we would serialize the state of the devices available in a session. Unfortunately, it is not very easy without relying on clutches like this.

[matze]

Some new insight:

• Use setup of an experiment to define coarse selection of devices that are dumped by default.
• Allow opt-out for specific devices and attributes.

[tfarago]

This is pretty old but actually the idea was good and we should implement this, but after python 3.

[MarcusZuber]

Most of this is already paritlally in the Experiments in a way that most of the experiment properties can be read by the users.

Short summary how it works currently:
The Experiment inherits from Parameterizable and all relevant settings should be added as Parameters to the experiment.
The Experiment creates a txt-log-file in the walker.current directory when the run() is called and writes the __repr__ of the experiment there.

So I would suggest to add a second, more machine-readable output to the experiment, that puts all the properties into a json, xml or something similar.

The the possibility to add arbitrary devices/parameters the output would also be nice.
I imagine this like I would pass a list of devices to the experiment (in my session file) like slits, filters etc. and then in each run() all parameters are also put in (both) logs.

[MarcusZuber]

I just tried an early implementation in https://github.com/ufo-kit/concert/tree/new-metadata (I didn't want to compeltely overwrite this old branch).

[MarcusZuber]

I think this #483 should solve this almost completely.

Should we close this PR or do we need something in addition?

[tfarago]

What about the hdf5 point? Are we ever going to support that?

[MarcusZuber]

Maybe. But then we should start first with a proper hdf5 (+nexus?) walker and ImageWriter. Putting there the data, that is currently put in the json, should then not be a major issue.