Skip to content
This repository has been archived by the owner on May 10, 2024. It is now read-only.

Fix HostConnectionPool quadratic runtime #3847

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

carsonip
Copy link

@carsonip carsonip commented Mar 8, 2019

list.pop(0) is O(N) such that get() is quadratic. Use deque and
six.moves.range to keep get() always O(N). Add tests to
HostConnectionPool.

list.pop(0) is O(N) such that get() is quadratic. Use deque and
six.moves.range to keep get() always O(N). Add tests to
HostConnectionPool.
@carsonip
Copy link
Author

carsonip commented Mar 8, 2019

Test fails due to the previous SNI commit.

@carsonip
Copy link
Author

carsonip commented Mar 8, 2019

Benchmark script for this commit:

import time
from boto.connection import HostConnectionPool


def f():
    p = HostConnectionPool()
    count = 10000
    for _ in xrange(count):
        p.put(None)
    for _ in xrange(count):
        assert p.size() == count - _
        p.get()


t = time.time()
f()
print(time.time() - t)

Before fix: 0.27458691597
After fix: 0.0182430744171

@jayninja
Copy link

jayninja commented Mar 8, 2019

Test fails due to the previous SNI commit.

@carsonip looks like we are waiting on #3844 to be merged to fix python2.6 due to dependency issues :(

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants