Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

+2.9% perf: Avoid copying in send core, further batch memory optimisation #818

Closed
wants to merge 2 commits into from

Conversation

droe
Copy link
Contributor

@droe droe commented Mar 10, 2024

  • Avoid the extra memcpy in the send core by cleanly adapting the probe module interface to the introduction of batching (splitting thread init into thread init and packet preparation, allowing the latter to be called on each buffer in a batch)
  • Remove almost unused ip field from batch->packets[] and align the packet data such that the IP headers start at a 32 bit boundary, speeding up 32 bit header field access

The former results in a ~2% and the latter a further ~0.9% improvement to send rate at 10 GbE on lower-end hardware, while (arguably) improving code structure, or at least not making it worse.

Tested on macOS Sonoma, FreeBSD 14 and Ubuntu 23.10; smoke tested various probe modules.

Introduce new prepare_packet callback for the initialization of send
buffers; contrary to the per-thread initialization callback where this
was done previously, prepare_packet can be called multiple times, once
for each send buffer.

Make use of this to prepare packets to send directly in the batch
buffers instead of copying them over.
The ip field was only still used in send-bsd and only on a failure path
for logging, which does not seem like a strong justification for keeping
it around, especially given that it can always be read from the packet
data itself.

While removing the ip field, align buf such that the IP header is going
to start at a 32 bit aligned address, which again results in 32 bit
header field access to be faster because of ideal alignment.
@droe
Copy link
Contributor Author

droe commented Mar 11, 2024

I broke something, let me come back with this at a later time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant