Flagging and highlighting flows automatically from list of regexes

[malwareinfosec]

Hello,

I am the author of EKFiddle (https://github.com/malwareinfosec/EKFiddle), a plugin for Fiddler. The primary use case is to detect malicious web traffic using regular expressions that apply to URI, IP, Response Body, etc... Any detection is shown to the user by changing the UI, i.e. adding colors to the web sessions and a description of the threat.

I've also done something similar for ZAP proxy (https://github.com/malwareinfosec/FiddleZAP).

I am interested to see if this is something that mitmproxy could support?

[Prinzhorn]

You can "mark" flows (https://docs.mitmproxy.org/stable/api/mitmproxy/flow.html#Flow.marked) and you can add flow.comment / flow.metadata, which shows up in the UI. If you would need anything more specific than that, it'd be better to open a feature request.

We'd be more than happy to include such an addon in the contrib folder https://github.com/mitmproxy/mitmproxy/tree/main/examples/contrib, not sure about shipping it. This is a very niche use case for mitmproxy I would assume.

[Prinzhorn]

fiddle.py

def request(flow):
    if flow.request.path == "/evil":
        flow.marked = ":red_circle:"
        flow.comment = "Requested path was /evil"

mitmproxy --scripts fiddle.py
curl --insecure --proxy localhost:8080 https://example.com/
curl --insecure --proxy localhost:8080 https://example.com/evil

[malwareinfosec]

Thank you, this is very straightforward.

I was able to do the same for mitmweb:

However, I do not see a way to add a 'comment' column within the UI. I tried to add it from the web_columns option but that didn't work. If there is a way, how do you make it permanent so that restarting mitmweb brings up the new column automatically?

[Prinzhorn]

I'm really not familiar with mitmweb, but it looks like the data is correctly send to the client

so it might be a matter of adding comments to the list of allowed columns or we should render it in the flow details on the right. Actually both probably, but in the column it would just be a single truncated line with overflow ellipsis.

If you have any interest in submitting a PR it seems we export the different columns here https://github.com/mitmproxy/mitmproxy/blob/main/web/src/js/components/FlowTable/FlowColumns.tsx and comments would be filtered here because of missing implementation

mitmproxy/web/src/js/components/FlowTable/FlowRow.tsx

Line 46 in 590eef0

.filter((x) => x)

[malwareinfosec]

Having comments in both the columns and flow details would be great. I'm not sure how to do this kind of pull request though.

I've created an early version script based on your original code snippet that checks flow.request.path and flow.response.content against a list of regexes:
https://github.com/malwareinfosec/fiddleitm

[Prinzhorn]

I've opened #6708

Great!

[lups2000]

I opened a new PR for this, confirm me that is what you wanted. Otherwise I can change it , thanks :)