Overcounting pass-through requests

[ysmu]

It appears that pass-through calls are seeing 2x the actual call count. Is this behavior expected?

import httpx
import respx


with respx.mock as mock:
    route = respx.get('https://example.com').pass_through()
    httpx.get('https://example.com')
    print(f'pass_through() calls: {route.call_count}')


with respx.mock as mock:
    route = respx.get('https://example.com')
    httpx.get('https://example.com')
    print(f'mocked calls: {route.call_count}')

pass_through() calls: 2
mocked calls: 1

Environment:

Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import respx
>>> respx.__version__
'0.16.3'
>>> import httpx
>>> httpx.__version__
'0.16.1'
>>>

[lundberg]

Yes, unfortunately this is true.

Simply put, this is due to both the ConnectionPool and the Connection httpcore transports are patched by respx to ensure mocking is always happening regardless what the httpx client is configured to use (httpx default using pool).

In pass through mode, both gets hit and recorded, but without pass through the mocked response will be returned directly by i.e. the connection pool and recorded correctly.

This is noticed and marked as a TODO in this test.

I don't think the solution is to exclude the connection transport from patching.

Any thoughts and ideas welcome 😉

One think you could try though, is to tell respx to patch httpx rather than the current default httpcore, which should result in only one recorded hit for pass through requests. This is a rather new feature, but not yet documented, sorry for that.

with respx.mock(using="httpx") as mock:
    ...

[ysmu]

Thanks! I confirmed the new feature works for me.

Do you think it would be a good idea to add a flag to the request object going through ConnectionPool indicating that it is counted? That way, when Connection in httpcore sees it again, it can safely ignore it.

Anyhow, the workaround works for me and I'll let you decide if you want to keep this issue open.

[lundberg]

Great!

I see three alternatives solving the over-counting issue related to using the httpcore Mocker:

1. Flag the request, like you're proposing, but rather use that flag in the stats logic to decide to be recorded or not.
2. Keep the logic fully in the stats to know/decide if and when to record a call, handling "debounce".
3. Remove the ConnectionPool transport from the patching and always let the "final" Connection transport alone be responsible for the mocking and recording.

Anyhow, great that it could be solved for you. I'll leave the issue open and under consideration a little longer. Thanks.