Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rate Limit Minute (-rlm) behavior does not distribute requests evenly #1620

Open
swdbo opened this issue Mar 7, 2024 · 0 comments
Open

Rate Limit Minute (-rlm) behavior does not distribute requests evenly #1620

swdbo opened this issue Mar 7, 2024 · 0 comments
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.

Comments

@swdbo
Copy link

swdbo commented Mar 7, 2024

httpx version:

v1.6.0

Current Behavior:

When using the -rlm (rate limit per minute) argument, the tool sends the specified amount of requests all at once instead of distributing them evenly across the minute. This behavior is counterintuitive, as it leads to a burst of traffic followed by a period of inactivity, rather than spreading the requests out to avoid overwhelming target servers.

Expected Behavior:

The expected behavior for the -rlm argument is to distribute the specified number of requests evenly across the minute. For instance, if -rlm 10 is specified, one would expect a request to be sent every 6 seconds, thereby evenly pacing the load on the target server(s) and adhering more closely to a "rate limit."

Steps To Reproduce:

  1. Prepare a urls.txt file with multiple target URLs.
  2. Run httpx with verbose logging and the -rlm argument set to 10, like so: cat urls.txt | httpx -v -rlm 10
  3. Observe the output and timing of requests; all 10 requests are made at the same time, not spaced out at 1 request every 6 seconds as expected.

Anything else:

This unexpected behavior could lead to potential flooding of target websites, which is especially concerning in scenarios where careful rate limiting is necessary to comply with target server policies or to avoid unintentional Denial-of-Service conditions. An adjustment to ensure requests are distributed evenly throughout the specified time frame would greatly enhance the utility and reliability of the -rlm feature.

@swdbo swdbo added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

No branches or pull requests

1 participant