No parsing of headers after malformed HTTP/1.1 header (e.g. space)

[bwbroersma]

No parsing of headers after malformed HTTP/1.1 header (e.g. space). It looks like this can only happen in HTTP/1.1?

See RFC 7230 page 23 and § 3.2.4 that field-name : value is not valid. Based on the related bugs, it seems at least possible to setup an invalid HTTP header in Microsoft IIS (2/3 cases are IIS).
@baknu noticed that 🦊 Firefox won't show these invalid headers in the Network tab in the Response Headers, even in 'Raw' view.

The problem is an upstream 🐛 bug in Python http.client which is used by Requests:

• python3 fail on parsing http header python/cpython#69725

Related Requests 🐛 bugs:

• Requests stops parsing of headers after illegal space in header name. psf/requests#6249
• requests cannot fetch this website (only in Python 3) psf/requests#3133

Related issues:

• HTTP/2-only websites can't be tested #1259

---

Example https-client.py (used with $ python https-client.py target.host):

import http.client
import ssl
import sys

context = ssl.SSLContext()
context.options |= ssl.OP_NO_TICKET
host = sys.argv[1]
conn = http.client.HTTPSConnection(host,context=context)
conn.request("GET", "/", headers={"Host": host,"User-Agent":"internetnl/1.0","Accept": "*/*","Accept-Encoding":"deflate, gzip, br"})
response = conn.getresponse()
[print(f"{k}: {v}") for k,v in response.getheaders()]

Example curl (with a similar TLS ClientHello):

$ curl -A 'internetnl/1.0' 'https://target.host/' -D- -o/dev/null -sSf --http1.1 --compressed --no-alpn --no-npn

Doing a diff (skipping the Response line with | tail -n+2) results in:

Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/10.0
X-Powered-By: PHP/8.2.4
X-Powered-By: ASP.NET
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
> X-Content-Type-Options : nosniff
> Strict-Transport-Security: max-age=63072000
> Referrer-Policy: same-origin
> Content-Security-Policy: default-scr 'self'
> Date: Fri, 29 Mar 2024 14:59:16 GMT

The last 5 lines will be shown in curl, but won't be available in Requests / http.client (of course the other issue here is in the CSP, default-scr should be default-src).