Add support for HTTP range requests in the gateway

[yondonfu]

This was previously requested in ArweaveTeam/gateway#15 as well in the archived ArweaveTeam/gateway repo.

Problem

The arweave.net gateway currently does not seem to support HTTP range requests.

I tested this by running the following request which should return a HTTP/2 206 in the response header to indicate Partial Content returned if the server supports range requests:

curl --header "Range: bytes=100-107" -I https://7qblvuit2jxlwrjb55mo3rli7ven22kz75u2ma6n5curhjte3nha.arweave.net/_AK60RPSbrtFIe9Y7cVo_UjdaVn_aaYDzeipE6Zk204

However, the request returned a HTTP/2 200 in the response header instead:

HTTP/2 200
content-type: video/mp4
content-length: 1406563476

Here is an example of a request that does show that a server supports range requests:

curl --header "Range: bytes=100-107" -I https://bafybeievx532pbyokkoy3h23lgncoaqwvlthap3guozd2iyjowptpedeoe.ipfs.w3s.link/

We can see that the request returned a HTTP/2 206 in the response header:

HTTP/2 206
date: Tue, 15 Nov 2022 22:09:12 GMT
content-type: video/mp4
content-length: 8

Motivation

HTTP range requests enable video/audio streaming workflows as mentioned in ArweaveTeam/gateway#15. For example, if the gateway supports HTTP range requests, then another service can download the file in a streaming fashion and apply compute on the data (i.e. for a video this could include transmuxing into different file formats as well as transcoding into different qualities) as it is received. Without HTTP range requests, the service would need to download the entire file first before it can apply further operations.

[rosmcmahon]

the ar.io team manages the gateway. it would be best to make the request directly to them. their repo is currently private on github, you can join their Discord here though https://discord.gg/yxPdyEzvjq

[hlolli]

The bug here seems to just be related to how cloudfront caches status codes and headers.

curl -H "Range: bytes=0-1023" https://7qblvuit2jxlwrjb55mo3rli7ven22kz75u2ma6n5curhjte3nha.arweave.net/_AK60RPSbrtFIe9Y7cVo_UjdaVn_aaYDzeipE6Zk204 -o- > deleteme

> ll
.rw-r--r--  1.0k hlolli 16 Nov 10:52  deleteme

it's maybe not a cheap lift to cache every single range request header, but I'll have a look at this.

There are already multiple video streaming services on arweave utilising range-requests, so far they are working just fine.