Provide standard system packages as an alternative to Docker

[proffalken]

Summary:
All major linux distributions rely on package managers to install software in a controlled, secure manner.

This proposal is that packages are created in RPM, deb, and opkg formats as a starting point, with additional package formats being added at a later date.

Why do we need this?
Many enterprise organisations prevent git from connecting to any servers other than their own, and may not allow wget commands either.

Docker is frequently also not available within these organisations, however access to package repositories is a "known quantity" and therefore easier to sell to the security teams, especially as packages tend to be fully signed as part of the creation process.

There are also situations for which Docker is just not a good fit. This could include when Systems Integrators/Managed Service Providers want to scale using something AWS Auto Scaling Groups based on instances rather than containers, or for bare-metal installations in secure environments.

Installation via configuration management tools such as Ansible, Chef, or Puppet is also incredible difficult when you don't have packages available.

Finally, installing from a git repository as opposed to a package is difficult without compilers, which opens up a security risk by installing compilers on production systems.

What is already there? What do you see now?

At present, we have a choice between two docker containers (ttn-lw-stack and ttn-lw-cli) or building binaries of the same name from git.

Deployment is either via docker pull or git clone, both of which may not work behind corporate/educational firewalls.

What is missing? What do you want to see?

The ability to install via yum, apt, or opkg

How do you propose to implement this?

This should be relatively easy to implement using GO Releaser, however GO is not a language I'm particularly familiar with (yet!)

Environment:

Linux - CentOS/RHEL-based, Debian/Ubuntu-based, OpenWRT-based

What can you do yourself and what do you need help with?

I'd need confirmation that this is the appropriate way to package a go application for deployment, and then some help with the implementation from someone who understands GO!

[rvolosatovs]

I will start with adding packages to APT, AUR and https://github.com/nixos/nixpkgs, since I'm already familiar with the format of these. Other ones will be added later.
Contributions are, of course, always welcome!

[proffalken]

@rvolosatovs are you going to be using go-releaser, or another way of creating the packages?

I'd be happy to work on the RPM side of things as long as I know what you're using to create them! :)

[johanstokking]

We’ll make this part of CD as well.

Also adding brew to the wishlist.

[mrpackethead]

Just adding my support to this. I've never used docker before, and while i do appreciate that there are lots of programming languages, this was a huge jump for someone whos never done it before. While isntalling from a repo is how developers like to do things, its not very friendly for end-users.

[proffalken]

Thanks for all your work on this, I'm hoping to find time in the next few days to add RPM support and test it.

[rvolosatovs]

@proffalken We will build the RPM package as part of CI - you will be able to find it on our releases page once we do a release. (FYI, github also supports Atom feeds for releases: https://github.com/thethingsnetwork/lorawan-stack/releases.atom)

I followed your suggestion of using goreleaser.
Regarding RPM package itself - it is (for now) just

lorawan-stack/.goreleaser.yml

Lines 79 to 92 in b118396

	nfpm:
	vendor: The Things Network
	homepage: https://www.thethingsnetwork.org
	maintainer: The Things Network Foundation <stack@thethingsnetwork.org>
	description: The Things Network Stack for LoRaWAN
	license: Apache 2.0
	formats:
	- deb
	- rpm
	recommends:
	- redis
	- cockroach
	files:
	"public/**/*": "/var/lib/ttn/public"

I suspect the location of frontend, for example, is non-standard to RPM packages and, in fact, is not default for stack either(we expect it to be at /srv/ttn-lorawan/public).
So your input on how this should be done would be very helpful.

Note, that for brew, for example, we wrap the stack binary to set the related environment variable to point to the frontend:

lorawan-stack/.goreleaser.yml

Lines 116 to 119 in b118396

	env = {
	:TTN_LW_HTTP_STATIC_SEARCH_PATH => libexec/"public"
	}
	(bin/"ttn-lw-stack").write_env_script libexec/"ttn-lw-stack", env

I'm sure the expected location of this in RPM package is different, but we can hopefully use the same approach there.

[proffalken]

@rvolosatovs makes perfect sense to me.

The setting of the env vars is probably best done in a systemd service script, I'll try and put one together as soon as I can.

Sticking things into /srv/ttn-lorawan/public is also perfectly acceptable :)

[johanstokking]

@rvolosatovs can you update the labels and/or original comment to indicate that we don't need help anymore, or help with a specific distribution? Before community members start eagerly implementing this while most of it is in progress.

[rvolosatovs]

Blocked by lack of v3.0.0 release.
It does not make sense to publish pre-release versions to repositories.

[johanstokking]

Closed because of release. We have Snappy and Brew support, as well as deb and rpm files