Configuration management in Github

[mstormi]

[contribute your ideas here, please]

Let's store the whole openHAB configuration in a Github repo.
(note: this is about storing openHAB config, not any other data, that's what the Backup/Restore tools in menu 50 are for.)

Rough idea is to add a store_to_github routine (=git add+push) and restore_from_github (= git pull+checkout) routines in openHABian to run unattended to store/retrieve the current config so they can be automatically triggered e.g. via systemd timer, too.

Let's take care we properly handle multiple OH instances:
use case can be one "operator" [or "installer"] operating OH for multiple customers. In this case every customer (= end user) needs his own Github repo. He can assign maintainer rights to the operator's Github user.
AFAIK private Github repos are free now and can have up to 3 collaborators.

We should use one branch per location.

TBD: also store 3rd party tool config (mosquitto? what else?)

Related: #1214

[thefechner]

That sounds like a cool feature. With OH 2.5 I also stored my config files under /etc/openhab2/ (e.g. items, things, rules & Co.) in a free gitlab repository. With OH 3.0 I learned there are more files in different folders to be considered.

Not sure but as far as I understood, all the new cool stuff like equipments points, properties, models, pages with all their details are stored in the jsondb files under /var/lib/openhab/. (sorry there was a comment from rlkoshak in one of his oh3-wiki-preparation threads but I can't find it anymore). As far as I remember, all files under /var/lib/openhab/ except tmp, backup and persistance might be relevant.

I was also searching for a possiblity to replace/remove/hide/surpress passwords and other non-public data before syncing to git but it seems, that it is not so easy to solve. Might be considered as well or at least a big warning to the user should appear.

btw: Thank you very much for the efforts you have spend so far. Great job. Cheers thefechner

[JAMESBOWLER]

I have been playing with this concept and love the idea

My take on it from the users input perspective.

• The user would need to create a github repo that is private so no sensitive data shared.
• Config would be username reponame token in openhabian.conf

As per the idea of #1214 the openhabian.conf could be downloaded from this repo before install like adminkeyurl

Then after openHAB is installed pull the config files before starting.

I went looking for a project with similar goals and found etckeeper

[mstormi]

As per the idea of #1214 the openhabian.conf could be downloaded from this repo before install like adminkeyurl

Hmm, for openhabian.conf itself that's zero-conf / bootstrapping. VERY tricky to get it right to overcome the chicken'n'egg problems it comes with.
How do you get the data onto the system needed for initial access such as reponame und credentials to access it ?
In any operator-like setup the person on site will have little to no knowledge about the system and teaching our moms vi isn't a feasible option I guess.
For now I'd think this should be a menu option in openhabian-config but if anyone has a more clever idea let me know.

etckeeper I haven't looked at so far but it has been part of openHABian in a long time, see packages.bash.
Would you be willing to find out if it is adaptable/applicable and flexible enough for CM ?

[JAMESBOWLER]

Yes I will take this onboard.

I did install etckeeper and the parent git is in the /etc/ folder which will need to be moved to the root directory to be able to keep all the configs.

[mstormi]

Yes I will take this onboard.

Did you mean to say you work on this ? Can we hope for a PR ?

[JAMESBOWLER]

@mstormi Yes I have taken some time to learn how to get this all up and running.

I taking the time to do it properly and would like to use the gh cli. ITs been a crazy week here we had to go into lockdown for 3 whole days because 1 person 1.853 million km² had the UK strain of covid and work that pays the bills got crazy.

[mstormi]

@JAMESBOWLER Is there any reason why you retracted your PR that I should be aware of ?

[mstormi]

FWIW about every 2nd reply in https://community.openhab.org/t/benefit-of-your-experience-what-have-you-learned-that-you-wish-you-had-known-at-the-start/121422/36 says they store their config in Github so I guess this would be a highly welcome feature.

[mstormi]

@JAMESBOWLER Is there any reason why you retracted your PR that I should be aware of ?

@JAMESBOWLER could you please provide me with your failed coding attempt ?

[mstormi]

@JAMESBOWLER would you mind to provide me with your failed coding attempt ?

[mstormi]

@JAMESBOWLER any input from your side ?

[mstormi]

I was recently thinking how as an operator to centrally manage an items+rules subset for users, similar to the rules and widgets distributed via OH marketplace.
If a user's config is stored in the user's own GH repo then I as the rule developer cannot update it so we would rather need some central repo. Peeking at commercial applicability, it cannot be a public repo that's freely accessible. We would need at least some sort of password protection.

any thoughts anyone?

@ecdye @JAMESBOWLER anyone willing to take the ball on this ?

[ecdye]

I don't think that we could safely implement a proper central storage solution like that. I think that if we were going to do it we would need to start small and scale up. So do what the original plan was with the GitHub repo and then if that works implement tools or API keys for a bot to update it or something.