APT: Use https instead of insecure http

[ColorfulRhino]

Which feature would you like to have?

The Armbian builder is using HTTP mirrors with apt. The same is true in the mirrors from sources.list in the built Armbian images (tested with self-built bookworm image for NanoPi R5C, see below).
Using HTTPS transport for APT as a safer default would enhance security and privacy.

But APT already has its own signing and verification mechanisms!

Yes, this is true. APT packages are signed and verified to make sure the package you install is actually the correct package and not some malicious package. And this is a great thing!
However, many bugs, weaknesses and flaws in these mechanisms have been discovered in the past (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16 to name some), which allowed exploits and Man-in-the-Middle attacks, many of which could have been prevented through the use of secure transport via HTTPS.

But all those bugs have long been fixed by now! - Yes of course, that's a good thing and normal practice to patch vulnerabilities found in software. Unfortunately, as we know, there will always be bugs. It is only a matter of time when they get discovered. If we're lucky, they get discovered by a good guy. If we aren't, the vulnerability will likely be exploited by not-so-nice people.

This article describes how Max Justicz was able to exploit a bug in APT by planting a malicious package, which he could no have exploited with HTTPS.
He also discusses the use of HTTP vs HTTPS and comes to the conclusion that

Supporting http is fine. I just think it’s worth making https repositories the default – the safer default – and allowing users to downgrade their security at a later time if they choose to do so. I wouldn’t have been able to exploit the Dockerfile at the top of this post if the default package servers had been using https.

This discussion is also an interesting read and goes through some pros and cons, also you can see how things have changed over the years as HTTPS support got wider and better.

If you're up for a longer and more in-depth read about package manager security and weaknesses, the paper A Look In the Mirror: Attacks on Package Managers, even if a bit older, will have you covered.

As a little bonus, users's privacy will benefit as well with using HTTPS: When using HTTP, a potential MITM attacker or snooper can see exactly which packages you download, since the traffic is plain. Using HTTPS, the attacker could merely roughly estimate on the file size of the download.

Okay I get it, HTTPS is easily implemented and will give me enhanced security and stuffs, but....

But the apt caching software I have installed on all my Armbian machines will break!

Since APT caching has to be enabled and configured manually, the switch to HTTPS will likely be invisible for the majority of users installing Armbian. However, there likely are some people who have apt caching enabled and configured, e.g. to share packages in their local network. Good news is, as easy as it is to enable secure transport via HTTPS, as easy it is to disable it again.

For this, I propose the following solutions (only one needs to be implemented):

1. 100% user side: Users can change their apt sources to HTTP by adding one or two lines to their customize-image.sh (using sed for example)
2. 50/50 user/Armbian side: A simple build option can be added to disable HTTPS sources at build time
3. 100% Armbian (automatic): Whereever MANAGE_ACNG does its stuff, add a few lines to make sure the mirrors are configured with HTTP

In addition to that, it is possible to use apt-cacher-ng together with HTTPS sources. Instructions here: https://www.unix-ag.uni-kl.de/~bloch/acng/html/howtos.html#ssluse

Conclusion

All in all, I believe that, to cite Max Justicz, "it’s worth making https repositories the default – the safer default – and allowing users to downgrade their security at a later time if they choose to do so."
But please, make up your own mind on this topic and feel free to comment. Open for discussion😄

[github-actions]

Jira ticket: AR-2067

[adeepn]

See https://archive.ph/y65aN

APT verifies the signature of packages. So you do not need to have a form of transportation that provides data authentication. If an attacker modifies the files you're downloading, this will be noticed. Using a signature verification is better than using an HTTPS connection, because it'll detect an attack on the server you're downloading from, not just an attack in transit.

And HTTPS hinders the functioning of proxy caching.

[ColorfulRhino]

Using a signature verification is better than using an HTTPS connection, because it'll detect an attack on the server you're downloading from, not just an attack in transit.

HTTPS can be used in addition to signature verification though.
When using insecure HTTP, imagine the following attack scenario: A severe vulnerability is found in a package, for example ssh. An attacker could impersonate the apt server and distribute a legit, but old version of that package to you to keep your system in a vulnerable state where the attacker can use those vulnerabilities to exploit your system.

Your link states about these replay attacks that To mitigate this problem, APT archives includes a timestamp after which all the files are considered stale. With the Debian wiki for Release format says The Valid-Until field may specify at which time the Release file should be considered expired by the client. Client behaviour on expired Release files is unspecified.
However, when you look at the latest Debain Release file, you can see that there is no Valid-Until field anymore.
So, as far as I can see, this mitigation for replay attacks is, for whatever reason, not in use anymore (keep in mind your article was written 2017/2018). Maybe it was causing issues.

Another argument is privacy and security combined: When using http, a potential MITM attacker or snooper can see exactly which packages you download. Using https, the attacker could merely roughly estimate on the file size of the download.

[adeepn]

Ubuntu and Debian both use http. Ask them why)

[ColorfulRhino]

Citing another argument from thomasrutter:

Another reason you may consider using HTTPS is to guard against flaws in APT's own signing and verification. Flaws in this have been discovered, and fixed, in the past.

[adeepn]

and fixed, in the past

[ColorfulRhino]

Ubuntu and Debian both use http. Ask them why)

Ubuntu also uses and holds on to Snap. Not all desicions are necessarily the best and many can probably be improved upon, which is why we are discussing this :)

and fixed, in the past

Obviously flaws in APT will get fixed, if discovered. As you know, no software is without bugs and flaws. So of course there will be new discoveries of flaws in APT in the future.
A replay attacker can hold you off from updating APT itself, so you won't get a fixed version of APT.

[adeepn]

Ubuntu also uses and holds on to Snap. Not all desicions are necessarily the best and many can probably be improved upon, which is why we are discussing this :)

Obviously flaws in APT will get fixed, if discovered. As you know, no software is without bugs and flaws. So of course there will be new discoveries of flaws in APT in the future.

It's demagoguery. There are two upstream distributions that deliberately chose the upgrade policy, with their own arguments. You propose to break this policy and replace it with a new one without proper arguments.
In doing so, you will clearly make things worse by the fact that:

• in the case of a site certificate expiration or revocation, the update will break (if you don't update for a long time, you won't be able to put a new update in)
• if time is not synchronized, you will not be able to update
• the system of caching packets through proxies breaks down

[ColorfulRhino]

You propose [...] without proper arguments.

Please see all my comments above for proper arguments and explanaitions.

* in the case of a site certificate expiration or revocation, the update will break (if you don't update for a long time, you won't be able to put a new update in)

When was the last time you had problems with invalid HTTPS certs on package repositories? Expirations on higher traffic sites get noticed and resolved quickly. If there is ever a cert problem, do your update later or tomorrow and it will most likely have resolved itself by then.
I'm using HTTPS mirrors on various distributions with various package managers for 7+ years now and never once had a problem because of invalid HTTPS certs. The only problems I had when not updating for a long time were with expired keyrings (local package verification). The upgrade will not be able to go through. You'll need to update the keyring-package and then the upgrade will go through normally. However, this has nothing to do with HTTP or HTTPS and can happen with either, since it has to do with the package verification mechanism.

* if time is not synchronized, you will not be able to update

If time is unsynchronized, you might get problems with both HTTP and HTTPS, see the following example:

* the system of caching packets through proxies breaks down

Armbian does not have any apt-cachers like apt-cacher or apt-proxy installed. Local package caching does only make sense if you have multiple machines with the same distro. People explicitly setting up apt caching in their network are likely experts. To set this up, they will have to fiddle with sources.list and a caching server.

In contrast, every user of Armbian, no matter if expert or newbie, is using apt. Improving privacy and security by using https mirrors will benefit everyone overall.

[adeepn]

Please see all my comments above for proper arguments and explanaitions.

You are substituting general experience for your personal experience. The fact that everything works for you is great, but it doesn't mean it works for everyone.

When was the last time you had problems with invalid HTTPS certs on package repositories?

2024. https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/ And we do have difficulties with a large number of devices due to this event.

If time is unsynchronized, you might get problems with both HTTP and HTTPS, see the following example:

I seems to be wrong about the timing, a roughly set time is sufficient.

Armbian does not have any apt-cachers like apt-cacher or apt-proxy installed

Have you seen the MANAGE_ACNG option in the armbian for example?

[ColorfulRhino]

Please see all my comments above for proper arguments and explanaitions.

You are substituting general experience for your personal experience. The fact that everything works for you is great, but it doesn't mean it works for everyone.

Where above this original comment have I talked about personal experience instead of talking about vulnerabilities in APT's signing and verification steps, privacy issues or MITM attacks?

To add to that, I can recommend these two reads:

1. Remote Code Execution in apt/apt-get: This articles dives into how apt was vulnerable and could be exploited due to this.
2. A Look In the Mirror: Attacks on Package Managers: This is a paper discussing weaknesses of packet managers. A bit older, but still a quite interesting read!

When was the last time you had problems with invalid HTTPS certs on package repositories?

2024. https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/ And we do have difficulties with a large number of devices due to this event.

Would you mind explaining this further please? I don't think I'm getting your argument here. Debian ships up-to-date root CA certificates. Which devices running current Armbian are running with ancient out-of-date root CA certificates and why?

Armbian does not have any apt-cachers like apt-cacher or apt-proxy installed

Have you seen the MANAGE_ACNG option in the armbian for example?

No I have not. Why? Because there is no documentation about this to be found whatsoever:

Which means that such an option is very unlikely to be used by the vast majority of Armbian users.
It's not that HTTPS sources are set in stone once changed by default 😊

I propose the following solutions for everyone who needs HTTPS disabled:

1. Change sources to http via their customize-image.sh (a one- or twoliner using sed)
2. A simple build option can be added to disable HTTPS sources at build time
3. Whereever MANAGE_ACNG does its stuff, add a few lines to include HTTPS --> HTTP

In addition to that, it is possible to use apt-cacher-ng together with HTTPS sources. Instructions here: https://www.unix-ag.uni-kl.de/~bloch/acng/html/howtos.html#ssluse