Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add in-memory ASF request dispatching for boto clients #9569

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

bentsku
Copy link
Contributor

@bentsku bentsku commented Nov 7, 2023

Motivation

Taken from #7396 from @thrau:
This PR adds code to make calls from boto clients directly to a Gateway without an HTTP server roundtrip. This can make internal calls much faster because they don't require IO.

The mechanism used to achieve this is botocore's before-send event, which can be used to short-circuit the request logic by returning a response from the event handler, before the HTTP request is even made.

I did some basic refactoring to make this easier, including:

  • created localstack.runtime.components as a place for runtime-level singleton components
  • added a new method to Gateway with the signature def handle(self, context: RequestContext, response: Response), which now makes the Gateway a drop-in replacement for a handler chain (and easier to call if you want to invoke it with a pre-existing RequestContext).

I believe we could merge this as is, as it's only opt-in, and we could improve in follow-up iterations. We'd at least get a preview version of it, and we can give it more tries.

Benchmarks:

Results:

  Base In-memory client Everything (#9568) + in-memory
Throughput for 1kb publish (req/s) 20.17 28.98 102.77

In-memory only with cloudwatch SQS metrics + SNS delivery logs: 1000 requests took 34.5011 s = 34.5011 ms/op = 28.98 req/sec
All fixes: 1000 requests took 9.7307 s = 9.7307 ms/op = 102.77 req/sec

Testing

Ext Integration Tests 6506 ✅
Successful run with the flag enabled: https://app.circleci.com/pipelines/github/localstack/localstack/25062/workflows/401a985e-e287-483e-82f3-229daf251654

TODO:

  • basic PoC implementation
  • integration with the client factory and connect_to
  • more benchmarks to see whether/where this is really useful
  • documentation

@bentsku bentsku added area: asf semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases labels Nov 7, 2023
@bentsku bentsku self-assigned this Nov 7, 2023
@bentsku bentsku added this to the Playground milestone Nov 7, 2023
@bentsku bentsku force-pushed the gateway-botocore-integration-ben branch 2 times, most recently from 71efcc8 to 4a28905 Compare November 9, 2023 09:41
@bentsku bentsku force-pushed the gateway-botocore-integration-ben branch from 4a28905 to 4229261 Compare January 10, 2024 16:58
Copy link

github-actions bot commented Jan 10, 2024

S3 Image Test Results (AMD64 / ARM64)

  2 files  ±0    2 suites  ±0   3m 14s ⏱️ +3s
401 tests ±0  351 ✅ ±0   50 💤 ±0  0 ❌ ±0 
802 runs  ±0  702 ✅ ±0  100 💤 ±0  0 ❌ ±0 

Results for commit 430d6fc. ± Comparison against base commit 7295eb3.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Jan 10, 2024

LocalStack Community integration with Pro

    2 files      2 suites   1h 43m 15s ⏱️
3 004 tests 2 695 ✅ 309 💤 0 ❌
3 006 runs  2 695 ✅ 311 💤 0 ❌

Results for commit e8a9390.

♻️ This comment has been updated with latest results.

@bentsku bentsku force-pushed the gateway-botocore-integration-ben branch 3 times, most recently from 6e40664 to a8249ce Compare May 21, 2024 10:53
@bentsku bentsku force-pushed the gateway-botocore-integration-ben branch from 20308e7 to 011b952 Compare May 23, 2024 11:01
@bentsku bentsku marked this pull request as ready for review May 24, 2024 10:06
@bentsku bentsku requested a review from thrau as a code owner May 24, 2024 10:06
@thrau thrau force-pushed the gateway-botocore-integration-ben branch from 011b952 to 45591a0 Compare May 24, 2024 16:59
@bentsku bentsku force-pushed the gateway-botocore-integration-ben branch from 45591a0 to e8a9390 Compare May 29, 2024 12:32
@thrau thrau force-pushed the gateway-botocore-integration-ben branch from e8a9390 to 430d6fc Compare June 1, 2024 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: asf semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants