New composite non-Linux UDP transport

[chrysn]

The udp6 transport, while working greatly on Linux, stands little chance in working for non-Linux OSes as long as they lack RECVERR (see recent summary). So, what to do?

The simple6 / simplesocketserver transports are a bit weird because they are separate server/client implementations, and because they don't allow sending from the regular socket address. So, let's assume we have a PKTINFO capable system (it's probably possible on Windows, hard to find someone who can do it, but either way it'd work for the rest of the world).

We'd use

• one socket that is bound (possibly to an any-address -- we have PKTINFO after all), on which we're mainly receiving.
• while we're sending (possibly LRU'ing up to the NOFILE rlimit), we create a socket, bind it (so we can influence the source address -- otherwise once it gets connected, one is picked at random), and connect it. We then send through this, and should get ICMP errors through this.
  A caveat is that we have to recvmsg also on those sockets, at least for some time -- while it is bound and not connected, the socket may queue up incoming packets that are not even from the peer it's connected to, even if the recv is done after the connect.

Cloudflare calls this established-over-unconnected.

This should be fully functional in that we are in full control of the addresses, and get errors. The downside compared to udp6 is that we need many sockets while we're sending there. A possible further downside is that when we close sockets, there might be packets lost in their queue between the last recvmsg and the close().

ToDo:

• Set up a demo that shows this can work.
• Verify that in this setup we even get ICMP errors forwarded. (The ones I've seen earlier might have just been from going through loopback, which has different rules). If that is not the case, this whole exercise is moot, but also there is no benefit in using simple6/simplesocketserver on non-MSGERR platforms.