Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump the smtp-tls patch branch to master #226

Open
wants to merge 135 commits into
base: patches/notqmail/smtp-tls
Choose a base branch
from

Conversation

josuah
Copy link

@josuah josuah commented Oct 4, 2021

As discussed in #210.

Another rebase was planned by @DerDakon for 1.09, this is an intermediate update to keep the patch abit more up to date. Or should we target the latest release rather than master?

Not for merging direcly: rather to push -f to the target branch for updating it. I am doing a merge request for review purpose.

DerDakon and others added 30 commits May 20, 2020 20:51
These 3 scripts call maildir2mbox before calling one of 3 mail programs. Both
elq and pine are obsolete and are not maintained anymore, and I have no idea
what "Mail" was. Noone uses any of these today anymore, and most people don't
do on local mail stores either.
Both of these functions are only called from a single place, where both pass in
the alloc() function. The parameter was also named alloc, which can cause
compiler warnings when -Wshadow is enabled. Simply remove the parameter, which
makes everything work exactly as before.
 * it's as simple as possible, probably intentional. This makes things like
   alloc_re() (i.e. realloc()) inefficient as it does not look if the thing it
   reallocates is already the last thing in the batch, and simply extends that,
   or if alloc_free() has the last block and adds things back to the pool
   (ideally with zeroing for security reasons).
 * it makes it basically impossible to detect overruns on early small
   allocations with tools like valgrind, address sanitizer or friends as
   those allocations never hit malloc(), so are never traced by any custom
   tools. Any overrun will just go into the next variable, untracable.
 * allocators do their own buffering, alignment, and so on. If they are broken,
   your system is severely screwed. And if we need more money^H^H^H^Hemory we
   will hit that buggy allocator anyway, making things harder to detect as it
   will only happen on "heavy" memory usage. This is not TruOS, SunOS 1.x or
   something like that anymore. We expect a reasonably sane OS. Everything else
   will get us screwed in much more severe ways.
 * it has shown issues when coming close to 4GiB allocations (#37, #109, older
   reports). DJB correctly says that an allocation of that size in your mailer
   shows that you have already done something wrong, but still: 64 bit
   platforms are the default, so using a 32 bit type to pass allocation sizes
   is a bug of it's own.
 * when any additional patch accidentially mixes calls to this functions with
   normal realloc()/free() this can lead to random crashes
 * why bother at all with having a custom allocator?
Spotted by Qualsys during their audit.
Basically 2 things could happen here:

-the custom implementation is used, which is probably slower than the one
 provided by the compiler
-the compiler or linker notices that this is just strlen() and replaces it with
 the normal one

In both cases we are better of just removing it altogether.

It would in theory be possible that this implementation is faster than what your
compiler or libc offers. But than your system is already slow, this few calls
for your mailer daemon will not matter then anyway.
For the very same reasons as str_len() this can go away.
That's how that macro is defined, so use it.
This one is actually a bit special: byte_diff() was used only for implementing
the byte_equal() macro and had only one direct callsite. And this in turn should
have been calling byte_equal(), too. When touching it anyway change it to
memcmp() directly.
This changes the return type, but noone ever looked at it anyway.
All of these functions are gone now and replaced by libc ones.

This reverts commit 8654051.
Rationale, in some sort of order:

- Plain text is already valid Markdown (and we can improve it as we go)
- Markdown is already plain text, readable in all the same ways
- GitHub also renders Markdown nicely in web browsers
- Changing installed names for docs is a non-zero compatibility break,
  but we consider it small enough to tolerate:
    1. Programs are unaffected
    2. People will press <tab> and find what they're seeking

Supports goals: Being easily packaged by OS integrators
Broaches non-goals: Breaking compatibility (but tolerably so)
Risks: Users won't find needed info; patches to hier.c may get skewed
DerDakon and others added 21 commits January 24, 2021 11:32
Original: qmail-103.patch

Sometimes (rarely) the answer to a DNS query is larger than 512 bytes.
This is an important limit: it's the biggest response (unpatched) qmail
can understand. This threshold was originally based upon the UDP DNS
protocol definition (RFC 1025, section 4.2.1). It is still relatively
unusual for DNS responses to be bigger than that limit, even though they
are allowed to be. However, it does happen from time to time,
particularly on domains with a lot of alternatives (if each MX record is
approximately 50 bytes, you need 11 of them to cross the limit). This
patch allows DNS packets to be as big as the maximum EDNS response size:
64kb (though RFC 6891 strongly recommends capping responses at 4kb).

(Description from http://www.memoryhole.net/qmail/#oversize-dns)
When building with -Wextra there are a lot of false positives for fallthroughs,
which happen because the compiler does not detect that these functions in fact
never return. Add an attribute if building with gcc. According to the
documentation it is supported since version 2.5, which is way older than
anything one can reasonably expect this to work on.
The FreeBSD 10 image is no longer available, and others have been upgraded.
TravisCI has been removed, but we have a Solaris one instead now.
In Git 2.32.0 and newer, `git status` (for instance) says:

    warning: unable to access '.gitignore': Too many levels of symbolic links

https://github.com/git/git/blob/master/Documentation/RelNotes/2.32.0.txt
explains thus:

    * It does not make sense to make ".gitattributes", ".gitignore" and
      ".mailmap" symlinks, as they are supposed to be usable from the
      object store (think: bare repositories where HEAD:.mailmap etc.
      are used). When these files are symbolic links, we used to read
      the contents of the files pointed by them by mistake, which has
      been corrected.

To satisfy Git, turn .gitignore back into a regular file containing the
contents of TARGETS. To satisfy other uses of TARGETS (such as in
exported tar archives, which don't include .gitignore), keep it as a
regular in-tree file too, with reminders in each to sync with the other.
The comment would be taken word by word as filenames, resulting in .gitignore
and TARGETS being deleted on "make clean".

Fixes: 8ea74d6
This patch implements RFC 3207 in qmail.
This means you can get SSL or TLS encrypted and
authenticated SMTP between the MTAs and from MUA to MTA.
The code is considered experimental (but has worked for
many since its first release on 1999-03-21).

Usage: - install OpenSSL-1.1.0 http://www.openssl.org/ or later
         (any version since 0.9.8 is presumed to work)
       - apply patch to netqmail-1.06 http://www.usenix.org.uk/mirrors/qmail/netqmail
         The patches to qmail-remote.c and qmail-smtpd.c can be applied
         separately.
       - provide a server certificate in /var/qmail/control/servercert.pem.
         "make cert" makes a self-signed certificate.
         "make cert-req" makes a certificate request.
         Note: you can add the CA certificate and intermediate
         certs to the end of servercert.pem.
       - replace qmail-smtpd and/or qmail-remote binary
       - verify operation (header information should show
         something like
         "Received [..] with (DHE-RSA-AES256-SHA encrypted) SMTP;")

Optional: - when DEBUG is defined, some extra TLS info will be logged
          - qmail-remote will authenticate with the certificate in
            /var/qmail/control/clientcert.pem. By preference this is
            the same as servercert.pem, where nsCertType should be
            == server,client or be a generic certificate (no usage specified).
          - when a 2048 bit RSA key is provided in /var/qmail/control/rsa2048.pem,
            this key will be used instead of (slow) on-the-fly generation by
            qmail-smtpd. Idem for 2048 DH param in control/dh2048.pem.
            `make tmprsadh` does this.
            Periodical replacement can be done by crontab:
            01 01 * * * /var/qmail/bin/update_tmprsadh > /dev/null 2>&1
          - server authentication:
            qmail-remote requires authentication from servers for which
            /var/qmail/control/tlshosts/host.dom.ain.pem exists.
            The .pem file contains the validating CA certificates.
            One of the dNSName or the CommonName attributes have to match.
            WARNING: this option may cause mail to be delayed, bounced,
            doublebounced, and lost.
            If /var/qmail/control/tlshosts/exhaustivelist is present,
            the lists of hosts in /var/qmail/control/tlshosts is
            an exhaustive list of hosts TLS is tried on.
            If /var/qmail/control/notlshosts/host.dom.ain is present,
            no TLS is tried on this host.
          - client authentication:
            when relay rules would reject an incoming mail,
            qmail-smtpd can allow the mail based on a presented cert.
            Certs are verified against a CA list in
            /var/qmail/control/clientca.pem (eg. from
            http://curl.haxx.se/ca/cacert.pem)
            and the cert email-address has to match a line in
            /var/qmail/control/tlsclients. This email-address is logged
            in the headers. CRLs can be provided through
            /var/qmail/control/clientcrl.pem.
          - cipher selection:
            qmail-remote:
              openssl cipher string (`man ciphers`) read from
              /var/qmail/control/tlsclientciphers
            qmail-smtpd:
              openssl cipher string read from TLSCIPHERS environment variable
              (can vary based on client IP address e.g.)
              or if that is not available /var/qmail/control/tlsserverciphers
          - smtps (deprecated SMTP over TLS via port 465):
            qmail-remote: when connecting to port 465
            qmail-smtpd: when SMTPS environment variable is not empty

Caveats: - do a `make clean` after patching
         - binaries dynamically linked with current openssl versions need
           recompilation when the shared openssl libs are upgraded.
         - this patch could conflict with other patches (notably those
           replacing \n with \r\n, which is a bad idea on encrypted links).
         - needs working /dev/urandom (or EGD for openssl versions >0.9.7)
           for seeding random number generator.
         - packagers should make sure that installing without a valid
           servercert is impossible
         - when applied in combination with AUTH patch, AUTH patch
           should be applied first and first part of this patch
           will fail. This error can be ignored. Packagers should
           cut the first 12 lines of this patch to make a happy
           patch
         - `make tmprsadh` is recommended (or should I say required),
           otherwise DH generation can be unpredictably slow
         - some need "-I/usr/kerberos/include" to be added in conf-cc

Copyright: GPL
           Links with OpenSSL
           Inspiration and code from examples in SSLeay (E. Young
           <eay@cryptsoft.com> and T. Hudson <tjh@cryptsoft.com>),
           stunnel (M. Trojnara <mtrojnar@ddc.daewoo.com.pl>),
           Postfix/TLS (L. Jaenicke <Lutz.Jaenicke@aet.tu-cottbus.de>),
           modssl (R. Engelschall <rse@engelschall.com>),
           openssl examples of E. Rescorla <ekr@rtfm.com>.

Bug reports: mailto:<qmail-tls akrul inoa.net>
…[8c]

Modification to Frederik Vermeulen's qmail-smtp-tls 20190517 patch.
Changes to the library search path belong in conf-ld.

If the openssl/ libraries are not available in the default library
search path, the user will need to modify their own configuration
to match their platform.

Modification to Frederik Vermeulen's qmail-smtp-tls 20190517 patch.
If the openssl/ headers are not available in the default include
search path, the user will need to modify their own configuration
to match their platform.

Modification to Frederik Vermeulen's qmail-smtp-tls 20190517 patch.
Modification to Frederik Vermeulen's qmail-smtp-tls 20190517 patch.
@josuah josuah force-pushed the josuah-patches/notqmail/smtp-tls branch from 59367e5 to 7ea8bc4 Compare October 4, 2021 17:42
alanpost and others added 3 commits October 4, 2021 20:31
Modification to Frederik Vermeulen's qmail-smtp-tls 20190517 patch.
This reduces collisions with other patches.
This reduces collisions with other patches.
@josuah josuah force-pushed the josuah-patches/notqmail/smtp-tls branch 2 times, most recently from 5498636 to f34c856 Compare October 4, 2021 18:40
@josuah
Copy link
Author

josuah commented Oct 4, 2021

IF you also want to drag in LibreSSL support to the TLS patch, one could try the below on an OpenSSL-based system and check whether this LibreSSL support broke OpenSSL support :

git checkout  josuah-patches/notqmail/smtp-libressl
make clean it

5498636

@josuah josuah changed the title [WIP] Bump the smtp-tls patch branch to master Bump the smtp-tls patch branch to master Oct 4, 2021
@DerDakon
Copy link
Member

DerDakon commented Oct 5, 2021

I would do the rebase when 1.09 is out, and hopefully we can convince the patch author to do a new release around that timeframe, too, to reduce the diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants