Multi-layer web fingerprint

[Mzack9999]

Please describe your feature request:

Modern web applications are generally composed of multiple layers of different software and technologies, and current tools (httpx included) fingerprints based on pattern matching (keywords, regexes, specific headers presence, etc) leading to ignore all the layers between the client and the final server, for example an app with a configuration like:

client => cloudflare/akamai => haproxy => nginx => http server

will lead to perform analysis only on the final layer and on response specific headers added by intermediate layers.

The purpose of this task is to introduce a new recon method based on triggering tech-specific parsing errors in intermediate layers, that will cause the request to be rejected and will provide a specific fingerprint for the specific technology. Intermediate layers implies correct parsing of the request (passthrough).
Errors can be detected heuristically from a baseline response of various frameworks via some metric distance like jaccard keeping into considerations features such as status code, meaninful headers/body parts.

Proposed approach

Given a generic RFC compliant http request in the canonical form:

Method URI Protocol/VersionMajor.VersionMinor
HeaderName: HeaderValue
\n
Body

it should be converted to a generic Bachus-Naur form with one level nesting where symbols are divided in terminal and non-terminal, as follows:

<request> := <method><uri><Version> # non-terminal
<uri> := / # terminal
<method>:=GET|POST|... # terminal

The symbols can follow two type of expansion:

• Literal: only terminal symbols by picking a random value among the available ones and applying potential string mutators
• Node: non-terminal symbols are expanded into other symbols (for example <method> => <uri>

Differently from classical fuzzing this approach might help to unveil parsing errors and identify specific technology errors that different from the baseline of the original request. In particular, when the expansion is applied to headers like content-length or chunking, it can unveil potential desync attacks that fit particularly well with identifying various layers.