Data compression for Discordbot when pulling in an external page (Accept-Encoding header)

[mboelen]

The Discordbot is not sending an Accept-Encoding header when pulling in external requests. So it would be great if it can do this, to save bandwidth on all sides (API, data center, publisher). Better for performance and climate-friendlier.

Good options to consider:

• gzip - well-known, implemented by most web servers
• brotli - better compression rates than gzip, implemented by more and more web server, almost all modern browsers support it
• zstd - even better compression, not widely available or implemented on client or web server side

Example log entry (406 means that the request was rejected as no type of encoding was offered):

2024-04-22T17:50:45+00:00 406 1.2.3.4 "GET /understand-and-configure-core-dumps-work-on-linux/ HTTP/1.1" 16 "-" "Mozilla/5.0 (compatible; Discordbot/2.0; +https://discordapp.com/)" TLSv1.3/TLS_AES_256_GCM_SHA384 0.000 .