Support other deployment models

[doronbehar]

Is your feature request related to a problem? Please describe.

@fsagbuya and I, and possibly others too, would like to deploy Linien's server in a declarative manner - for example on boards other then RedPitaya, and with other build systems such as Yocto/Petalinux (for Xilinx boards) and Nix (see not-os). Even in the simple case where the RedPitaya is not connected to the internet, and you want the SD card to include it prior to deployment in the lab, it seems currently non-trivial to do so.

I discussed this topic with you at #295 (comment) . I also still hold my nixFlake branch in which I made some small but significant changes to the client packages, that allow one to disallow imperative installation. Unfortunately, rebasing those changes now to the 1.0.1 branch is not trivial...

Describe the solution you'd like

I'd like the server and the client to support several deployment models:

1. Declarative, with Systemd.
2. Declarative, with runit.
3. Declarative as part of a build recipe, with SysVInit (for yocto based distributions).
4. Imperative - like it is now.

In both declarative deployment models, I want the clients not to try to install the server if it isn't installed - to keep the embedded Linux image pure. Non of the above mentioned common init systems use screen as a process manager...

As a bonus, it would be helpful for us Nix users if the project would have tracked a flake.nix file :).

Describe alternatives you've considered

Keep maintaining my fork - too much work.

Additional context

#295 as mentioned, and #293 .

See also the behavior I programmed the GUI in my outdated nixFlake branch:

https://github.com/doronbehar/linien/blob/a3e80dc41fb62bb00163944629afe647939d9a6c/linien-gui/linien_gui/ui/device_manager.py#L90-L110

[bleykauf]

Thank you for contributing this issue.

My first question to regarding the proposed changes would be why? I do not really see how having a multitude of different deploy methods would help people locking lasers.

I am well aware that running linien-server in a screen session is not good practice. I overhauled the startup process for the upcoming release of linien 2.0, see the develop branch. I plan to use systemd (#293). I simply did not get around to do that yet. Main benefits I do see here is autostart of the linien server and that it is a less hacky way of running the server.

As you yourself mention supporting Nix and/or several more deployment methods comes with significant additional workload and additional sources of bugs. So without clear benefits outweighing this additional workload I do not see why this should be implemented. Genuinely interested, what does your setup look like that you would have use for this?

[doronbehar]

It is a bit hard to split the deployment topic to installation and setting up autostart mechanism, but I'll try to do my best.

First of all, one type of target machines that we wish to deploy Linien onto, are machines with a read only file system, that boots the Linux file system into memory - without the ability to change permanently the file system - like a Linux installation Live CD. I encountered this kind of system installed on an SD card on a Xilinx development board, and if I understood correctly, @fsagbuya works now on deploying Linien on such a Xilinx' FPGA as well.

So installing Linien server on this kind of system would require us to do it ourself, and we wish that it would have been possible to disallow the clients to even try to install the server - this is the behavior I implemented in my nixFlake branch - to make sure that we work with the Linien server that we installed and setup to autostart by ourselves, and not with the one installed by the ssh script the client runs.

Another modification we would like to see from the client is that it'd be able to handle an error such as "systemctl command not found on server" with a message such as: "Couldn't start the server via ssh systemctl, try to start it manually". This way, advanced users like us can install the clients on our lab, and less technical users of the GUI, won't get error messages that are inappropriate for setups like ours.

I think that now I am able to summarize our hopefully modest feature request into 3 parts:

1. Support (perhaps via environment variables) disallowing the clients to try to install the server on the target machine (see this implementation).
2. Again in the clients, handle gracefully failing attempts to autostart the server, in case the server uses a different init system other then Systemd / screen (whatever you'd settle upon in 2.0).
3. BONUS (would be nice): Add Nix files such as flake.nix to allow us to use your repository and to help us help you contribute changes :).

[bleykauf]

So your plan is to port Linien to a different hardware platform? This is generally encouraged, see e.g. #275. However, I will not really be able to work on this myself.

In principle, it does not matter how the server is started for the behavior of the client. If the server is running the client will be able to establish a connection via rpyc. If the autostart_server flag is set to False, the client will also not try to start the server but it will simply fail with an exception being raised. When running from the GUI, this flag is set to True. This could be easily made user configurable and error handling when the server could not be autostarted as you mention could certainly be improved . The client will never try to install the server without the user explicitly confirming it, even when running the GUI.

As for installation without internet connection, there are ways to do that using pip, see https://stackoverflow.com/questions/36725843/installing-python-packages-without-internet-and-using-source-code-as-tar-gz-and. I actually think this should be documented since some institutions do not allow RedPitayas to connect to the internet. Tracked in #378.

I think it is virtually impossible to implement all possible startup methods into Linien and even implementing a subset is error-prone, needs additional maintenance and development work rather spent elsewhere. Specifically supporting nix and maintaining this is non-trivial as you said. I think this should rather be relegated to the individual users. Non-standard startup of the server could be handled by a simple script being run before starting up the client/GUI.

[doronbehar]

So your plan is to port Linien to a different hardware platform? This is generally encouraged, see e.g. #275. However, I will not really be able to work on this myself.

Of course, we don't want to ask for your efforts for this task, I am just pointing it out as a use case for supporting different installation methods, that won't require screen for example, and that would not necessarily rely on the GUI trying to run systemctl, and fail in an ugly manner.

As for installation without internet connection, there are ways to do that using pip, see stackoverflow.com/questions/36725843/installing-python-packages-without-internet-and-using-source-code-as-tar-gz-and. I actually think this should be documented since some institutions do not allow RedPitayas to connect to the internet. Tracked in #378.

Sure that can be tracked and discussed separately. If for example we'll use Petalinux for our boards, we might contribute eventually a bitbake recipe that this repository can include / link to.

I think it is virtually impossible to implement all possible startup methods into Linien and even implementing a subset is error-prone, needs additional maintenance and development work rather spent elsewhere.

I agree! I didn't mean to suggest this. In a similar manner to the 2nd request from my previous comment, I am only hoping that there was a way to disable autostart_server via an environment variable, and disallow suggesting to install the server by the GUI. It'd be rather easy to implement this feature, and I'm ready to do it myself, I just want to get your approval first 🙏.

Specifically supporting nix and maintaining this is non-trivial as you said.

Maintaining a flake.nix file is easy :), and I'm ready to be responsible for this. Not sure what I said about non-trivial maintenance.