Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Unable to make simultaneous asynchronous requests #302

Open
ZentixUA opened this issue Apr 18, 2024 · 3 comments
Open

[BUG] Unable to make simultaneous asynchronous requests #302

ZentixUA opened this issue Apr 18, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@ZentixUA
Copy link

Describe the bug
Making multiple simultaneous requests to a single host results in a curl error if a TCP connection to the requested host was established earlier (a request was made previously).

To Reproduce

  1. Install httpx using pip install httpx
  2. Install curl-cffi using pip install curl-cffi
  3. Run this code:
import asyncio
import sys

from curl_cffi.requests import AsyncSession
from httpx import AsyncClient

BASE_URL = 'https://images.pexels.com'
PATHS = ['/photos/842711/pexels-photo-842711.jpeg'] * 2


async def test_httpx():
    print('----- httpx -----')
    async with AsyncClient() as session:
        await session.get(BASE_URL)
        tasks = [session.get(BASE_URL + path) for path in PATHS]
        print(await asyncio.gather(*tasks))


async def test_curl_cffi():
    print('----- curl-cffi -----')
    async with AsyncSession() as session:
        await session.get(BASE_URL)
        tasks = [session.get(BASE_URL + path) for path in PATHS]
        print(await asyncio.gather(*tasks))


if __name__ == '__main__':
    if sys.platform == 'win32':
        asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
    asyncio.run(test_httpx())
    asyncio.run(test_curl_cffi())

Expected behavior
The requests will be successfully completed and the bytes of the requested images will be received with an HTTP status of 200, as occurs when using httpx.

Versions

  • OS: Windows 11
  • curl_cffi version: 0.6.2
  • pip freeze dump:
anyio==4.3.0
certifi==2024.2.2
cffi==1.16.0
curl_cffi==0.6.2
exceptiongroup==1.2.0
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
idna==3.7
pycparser==2.22
sniffio==1.3.1
typing_extensions==4.11.0

Additional context

  • Which session are you using? Async.
  • If using async session, which loop implementation are you using? Asyncio.
@ZentixUA ZentixUA added the bug Something isn't working label Apr 18, 2024
@yifeikong
Copy link
Owner

A temporary fix is to force http/1.1, I'm still investigating a further solution.

- async with AsyncSession() as session:
+ async with AsyncSession(http_version=CurlHttpVersion.V1_1) as session:

@ZentixUA
Copy link
Author

Hello! Are there any new details on the issue?

@ZentixUA
Copy link
Author

Hello again, any updates?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants