Added upper bound on number of threads ZMap can use

[phillip-stephens]

We had an issue opened because ZMap had strange behavior with --sender-threads=200. I mentioned that there isn't any performance benefit, but I think ZMap itself should prevent usage of the tool that is counter-productive or leads to unexpected behavior. I can't think of any drawbacks to capping sender threads to number of cores - 1.

Changes

• Added cap of --sender-threads=number of cores - 1 and warn the user and override their -T if they exceed the cap
• Updated the man pages to document the change

[zakird]

I think this is a good warning to have, but I'm wary of overriding if they're explicitly setting it.

ZMap would certainly be slower but it should still operate correctly when using a large number of threads.

[phillip-stephens]

Well it at least has issues with very high number of threads on Mac.

$ sudo ./src/zmap -p 80 -T 200 1.1.1.1/20
Password:
Mar 06 17:01:49.472 [INFO] zmap: By default, ZMap will output the unique IP addresses of hosts that respond successfully (e.g., SYN-ACK packet). This is equivalent to running ZMap with the following flags: --output-module=csv --output-fields=saddr --output-filter='success=1 && repeat=0' --no-header-row. If you want all responses, explicitly set an output module or set --output-filter="".
Mar 06 17:01:49.473 [WARN] blocklist: ZMap is currently using the default blocklist located at /etc/zmap/blocklist.conf. By default, this blocklist excludes locally scoped networks (e.g. 10.0.0.0/8, 127.0.0.1/8, and 192.168.0.0/16). If you are trying to scan local networks, you can change the default blocklist by editing the default ZMap configuration at /etc/zmap/blocklist.conf. If you have modified the default blocklist, you can ignore this message.
Mar 06 17:01:49.473 [INFO] dedup: Response deduplication method is full
Mar 06 17:01:49.489 [INFO] recv: duplicate responses will be excluded from output
Mar 06 17:01:49.489 [INFO] recv: unsuccessful responses will be excluded from output
1.1.1.158
1.1.1.23
1.1.1.228
1.1.1.233
1.1.1.221
Mar 06 17:01:51.029 [FATAL] socket-bsd: could not get an open packet filter

IIRC, Mac has a much smaller number of packet filters compared to linux. Either way, the warning should be sufficient to inform the user that they shouldn't be doing that 😄 .

[zakird]

Yeah, I think that error explains that pretty well. We could add some more helpful text to that error that says "Try using fewer sender threads"

[phillip-stephens]

Updated!

[zakird]

Logic looks good. That said, most of the changed files in this commit seem to be autogenerated using an older version of ronn than what's currently checked in, and aren't real changes. Probably makes sense to pull those out of the PR.

[phillip-stephens]

@zakird If this looks good to you, I'm good to merge.