Support cancellation of HTTP requests

[DerGuteMoritz]

Problem

At the moment it is not possible to cancel HTTP requests (be it in-flight or during connection establishment). The only option users have is to just drop the response deferred on the floor and have it GC'ed. However, this doesn't free up the underlying connection until the response is complete. Combined with the fact that there is no default request-timeout, this can lead to resource exhaustion.

Manifold's idiom for cancelling a deferred is to put it into an error state but that only affects chained deferreds downstream from the "cancelled" one (see also clj-commons/manifold#167). Since the response deferred returned by aleph.http/request sits at the very end of the deferred chain, putting it into an error state has no repercussion on any of the upstream deferreds nor on the underlying connection.

This also means that placing a timeout on a response deferred to limit the overall request duration (i.e. connection setup, request transmission and response reception) doesn't have the effect a user might expect:

@(-> (http/get "...") (d/timeout! 1000))

As explained above, this will simply put the response deferred into an error state when the timeout expires but the underlying request operation will still proceed, tying up any resources it has acquired until done.

Solutions

The most elegant solution would be to change Manifold to also cancel any upstream deferreds which feed into a "cancelled" deferred (if there are no others left). However, according to Zach, this cannot be implemented in the current architecture of Manifold and would require a "major reengineering".

A more tractable solution would be to explicitly cancel any upstream operations when the response deferred is set into an error state.

[DerGuteMoritz]

Related old issue where a user attempted to use d/timeout! as in the example above: #152

[DerGuteMoritz]

Cancellation of in-flight connection attempts is still missing. This can be implemented once Netty 4.1.108.Final is released which will include the patch for fixing netty/netty#13843.