ZMap fails to terminate

[dbomma]

Describe the bug
ZMap fails to terminate after reaching 100%. It continues to print logs, but fails to terminate or increment the "done" count.

CLI Arguments

echo '0.0.0.0' | xargs sudo zmap --sender-threads=200 -p 80

Example Target IP
0.0.0.0
I am attempting to perform a full /0 scan.

Expected behavior
I expect ZMap to terminate after printing "100%".

Environment:

• OS: Ubuntu 22.04.4 LTS
• Version: GitHub HEAD

Additional context
Here is a sample of the logs from Zmap:

2:13:46 106% (136 years left); send: 3702258432 done (489 Kp/s avg); recv: 22297 0 p/s (2 p/s avg); drops: 0 p/s (135 p/s avg); hitrate: 0.00%
2:13:47 106% (136 years left); send: 3702258432 done (489 Kp/s avg); recv: 22297 0 p/s (2 p/s avg); drops: 0 p/s (135 p/s avg); hitrate: 0.00%
2:13:48 106% (136 years left); send: 3702258432 done (489 Kp/s avg); recv: 22297 0 p/s (2 p/s avg); drops: 0 p/s (135 p/s avg); hitrate: 0.00%
2:13:49 106% (136 years left); send: 3702258432 done (489 Kp/s avg); recv: 22297 0 p/s (2 p/s avg); drops: 0 p/s (135 p/s avg); hitrate: 0.00%
2:13:50 106% (136 years left); send: 3702258432 done (489 Kp/s avg); recv: 22297 0 p/s (2 p/s avg); drops: 0 p/s (135 p/s avg); hitrate: 0.00%
2:13:51 106% (136 years left); send: 3702258432 done (489 Kp/s avg); recv: 22297 0 p/s (2 p/s avg); drops: 0 p/s (135 p/s avg); hitrate: 0.00%

The percent is over 100%, but the amount of packets sent is also not increasing.

[phillip-stephens]

When I try to reproduce with the latest main on Ubuntu, I'm not getting the same output with your CLI arguments. Can you post a screen recording?

$ echo '0.0.0.0' | xargs sudo ./src/zmap --sender-threads=200 -p 80
Mar 05 22:32:09.563 [INFO] zmap: By default, ZMap will output the unique IP addresses of hosts that respond successfully (e.g., SYN-ACK packet). This is equivalent to running ZMap with the following flags: --output-module=csv --output-fields=saddr --output-filter='success=1 && repeat=0' --no-header-row. If you want all responses, explicitly set an output module or set --output-filter="".
Mar 05 22:32:09.563 [INFO] dedup: Response deduplication method is full
Mar 05 22:32:09.567 [ERROR] blocklist: no addresses are eligible to be scanned in the current configuration. This may be because the blocklist being used by ZMap ((null)) prevents any addresses from receiving probe packets.
Mar 05 22:32:09.567 [FATAL] zmap: unable to initialize blocklist / allowlist

[phillip-stephens]

Besides that however, using --sender-threads=200 is likely going to be problematic.The sending threads can send as fast as possible and that many threads is going to cause resource contention. There's no performance benefit to having # of threads > available cores.

[dbomma]

When I try to reproduce with the latest main on Ubuntu, I'm not getting the same output with your CLI arguments. Can you post a screen recording?

$ echo '0.0.0.0' | xargs sudo ./src/zmap --sender-threads=200 -p 80
Mar 05 22:32:09.563 [INFO] zmap: By default, ZMap will output the unique IP addresses of hosts that respond successfully (e.g., SYN-ACK packet). This is equivalent to running ZMap with the following flags: --output-module=csv --output-fields=saddr --output-filter='success=1 && repeat=0' --no-header-row. If you want all responses, explicitly set an output module or set --output-filter="".
Mar 05 22:32:09.563 [INFO] dedup: Response deduplication method is full
Mar 05 22:32:09.567 [ERROR] blocklist: no addresses are eligible to be scanned in the current configuration. This may be because the blocklist being used by ZMap ((null)) prevents any addresses from receiving probe packets.
Mar 05 22:32:09.567 [FATAL] zmap: unable to initialize blocklist / allowlist

Ah, sorry about that, missed the /0. correct command is

echo '0.0.0.0/0' | xargs sudo zmap --sender-threads=200 -p 80

[dbomma]

Besides that however, using --sender-threads=200 is likely going to be problematic.The sending threads can send as fast as possible and that many threads is going to cause resource contention. There's no performance benefit to having # of threads > available cores.

That makes sense, I'll try toning it down a notch and seeing if that resolves the issue on its own.

[zakird]

@phillip-stephens even though a large number of threads shouldn't be used, it does seem like there's a bug here. ZMap should still terminate in this situation and it shouldn't reach above 100%...

[phillip-stephens]

This was closed automatically because when the PR was merged it closed the associated issue. Re-opening to investigate the non-termination.

My initial guess is that there's such strong resource contention that the threads can't make progress (or at least in a reasonable timeframe). Tho I don't have a guess for the >100% completion 🤔

[zakird]

I’m a bit skeptical that’s the issue. Eventually the threads would get a chance to run, and even if they didn’t, that wouldn’t result in completion being over 100%.

…

On Mar 7, 2024 at 3:19:17 PM, Phillip Stephens ***@***.***> wrote: This was closed automatically because when the PR was merged it closed the associated issue. Re-opening to investigate the non-termination. My initial guess is that there's such strong resource contention that the threads can't make progress (or at least in a reasonable timeframe). Tho I don't have a guess for the >100% completion 🤔 — Reply to this email directly, view it on GitHub <#809 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABREUGKG24JEZUNNGNF2QDYXDYXLAVCNFSM6AAAAABEH5TQ7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBUG43DMMBYGY> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[phillip-stephens]

I'm thinking this has two components:

1. Fails to terminate
   a) Potentially fixed in Do not skip elements >= (1 << 48) w/o checking for sentinel #814
2. Percentage complete is over 100%
   a) Fixed in Capped percent complete in monitor.c #825

I'm going to close this since I was unable to reproduce post #814 and we can reopen if this resurfaces!