[wip]Feature/wb rate limiting 2 #397
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
felliott
force-pushed
the
feature/wb-rate-limiting-2
branch
from
5ffb33c
to
85633fb
Compare
felliott
force-pushed
the
feature/wb-rate-limiting-2
branch
from
85633fb
to
52cb391
Compare
felliott
force-pushed
the
feature/wb-rate-limiting-2
branch
from
52cb391
to
901073d
Compare
felliott
force-pushed
the
feature/wb-rate-limiting-2
branch
from
b2a311d
to
0cab673
Compare
felliott
force-pushed
the
feature/wb-rate-limiting-2
branch
from
03cc7d4
to
d2feb84
Compare
* Add rate-limiting support to WaterButler via redis. This
implementation uses the fixed window algorithm.
* METHOD: Users are distinguished first by their credentials and
then by their IP address. The rate limiter recognizes different
types of auth and rate-limits each type separately. The four
recognized auth types are: OSF cookie, OAuth bearer token, basic
auth with base64-encoded username/password, and un-authed.
OSF cookies, OAuth access tokens, and base64-encoded
usernames/passwords are used as redis keys during
rate-limiting. WB obfuscates them for the same reason that only
password hashes are stored in a database. SHA-256 is used in this
case. A prefix is also added to the digest to identify which type
it is. The "No Auth" case is hashed as well (unnecessarily) so
that the keys all have the same look and length.
Auth by OSF cookie currently bypasses the rate limiter to avoid
throttling web users.
* CONFIGURATION: Rate limiting settings are found in
`waterbutler.server.settings`. By default, WB allows 3600
requests per auth per hour. Rate-limiting is turned OFF by
default; set `ENABLE_RATE_LIMITING` to `True` turn it on. The
relevant envvars are:
SERVER_CONFIG_ENABLE_RATE_LIMITING: Boolean. Defaults to `False`.
SERVER_CONFIG_REDIS_HOST: The host redis is listening on. Default
is '192.168.168.167'.
SERVER_CONFIG_REDIS_PORT: The port redis is listening on. Default
is '6379'.
SERVER_CONFIG_REDIS_PASSWORD: The password for the configured
redis instance. Default is `None`.
SERVER_CONFIG_RATE_LIMITING_FIXED_WINDOW_SIZE: Number of seconds
until the redis key expires. Default is 3600s.
SERVER_CONFIG_RATE_LIMITING_FIXED_WINDOW_LIMIT: Number of reqests
permitted while the redis key is active. Default is 3600.
* BEHAVIOR: Return the Retry-After header in the 429 response if
the limit is hit. This header states when it will be acceptable
to send another request. Other informative headers are included
to provide context, though currently only after the rate
limiting has been enforced.
If rate-limiting is enabled and WB is unable to reach redis, a
503 Service Unavailable error will be thrown. Since redis is not
expected to be available during ci, rate limiting is turned off.
* Grab-bag of related updates:
* Bump redis dep to 3.3.8. No consequential changes.
* Don't throw errors in the error handling. Provide a fallback for
the `resource` attribute if rate-limiting kicks in before that
has been initialized.
* Update some docstrings to clarify return values and process.
* Refactor test rate-limiting auth testing to only extract data as
needed.
* Add docs to settings; use `config.get_bool()` on booleans.
rebase: add password support to conn
felliott
force-pushed
the
feature/wb-rate-limiting-2
branch
from
d2feb84
to
3e23b51
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Followup on #380
Ticket
Purpose
Changes
Side effects
QA Notes
Deployment Notes