Log HTTP referer and origin

[rzats]

Related: #1386.

Summary:

Logs the HTTP Referer (or referrer) header as req_referrer and Origin header as req_origin, helping us track where requests come from. The easiest way to test this is via Postman:

This generates the following log string:

{"method": "GET", "url": "http://localhost:10080/epidata/signal_dashboard_status/", "form_args": {}, "req_length": null, "remote_addr": "172.18.0.1", "real_remote_addr": "172.18.0.1", "user_agent": "PostmanRuntime/7.30.1", "api_key": null, "user_id": "None", "req_referrer": "https://test.com/test", "event": "Received API request", "logger": "server_api", "level": "info", "pid": 8, "timestamp": "2024-05-09T11:28:19.206454Z"}

{ ... "req_referrer": "https://test.com/test" ... }

An automated version of this test has also been included. If no referrer is present, the HTTP Origin is used instead.

As mentioned in #1386, we might also need to modify the referrer policy across our webapps in order to properly set the relevant fields. The default referrer policy is:

Send the origin, path, and querystring when performing a same-origin request. For cross-origin requests send the origin (only) when the protocol security level stays same (HTTPS→HTTPS). Don't send the [Referer](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer) header to less secure destinations (HTTPS→HTTP).

Since the API endpoint is HTTPS, for some webapps these headers are likely to be already set out of the box, but further testing will still be necessary.

Prerequisites:

• Unless it is a documentation hotfix it should be merged against the dev branch
• Branch is up-to-date with the branch to be merged with, i.e. dev
• Build is successful
• Code is cleaned up and formatted

[melange396]

Please also add to the log_info_with_request() method!

[melange396]

• Can you move this to right underneath user_agent=... (since they come from the same place (request headers) and because they both identify client context)?

• Why not just use request.origin? Presumably so you can default to the empty string, but why prefer that over None?

• We dont need to keep both values; if both are provided, refer[r]er should be a superstring of origin.

Suggested change

	req_referrer=request.referrer,
	req_origin=request.environ.get('HTTP_ORIGIN', '')
	refer_origin=request.referrer or request.origin,

[rzats]

Fixed!

[melange396]

just a couple quick changes and we should be good :)

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud