Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use connection pooling for metadata queries #97

Open
RPCMoritz opened this issue Apr 19, 2021 · 0 comments
Open

Use connection pooling for metadata queries #97

RPCMoritz opened this issue Apr 19, 2021 · 0 comments

Comments

@RPCMoritz
Copy link
Contributor

When listing directories, it looks like connection pooling isn't being used.
Since I am listing a lot of directories, I am cycling through the ephemeral port list around 20 times, before doing anything else.
I would probably significantly reduce time and load on both ends, if I was able to reuse a single connection, instead of spawning 200000.

I believe the issue why connection pooling isn't being used automatically (as advertised by requests/urllib3) is that the connections created are always streaming, even if it's just a PROPFIND, which doesn't really benefit from the streaming.

My proposed solution would be to make the streaming flag in the request conditional upon the method being called. stream=True for actual up/downloads, and stream=False for list

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant