Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot GET entities > 2GB using jdk-connector #5470

Closed
nicobrevin opened this issue Nov 17, 2023 · 2 comments · Fixed by #5636
Closed

Cannot GET entities > 2GB using jdk-connector #5470

nicobrevin opened this issue Nov 17, 2023 · 2 comments · Fixed by #5636

Comments

@nicobrevin
Copy link

It's a bit hard for me to produce a minimal failing test case but I can show you exactly where the problem is.

I am forwarding some large files from openstack swift via jetty, jax-rs-client and the jdk-connector (because the performance seems way way better for big files)

In org.glassfish.jersey.jdk.connector.internal.HttpParser there's a method called decideTransferEncoding where it expects the content length to parse as an int. This fails when the response size is greater than Integer.MAX_VALUE. There might be more issues with this further on down the line, but this was the first one I hit. If I get some time I'll try and get you a patch, but for now I'm going to try and work around it.

@jansupol
Copy link
Contributor

So you use a buffered entity, you buffer over 2GB of data to memory and then you send them to the client. Do I understand the issue correctly?

@jansupol jansupol linked a pull request May 4, 2024 that will close this issue
@jansupol
Copy link
Contributor

jansupol commented May 4, 2024

Note that for large files, chunked encoding is far better than a buffered stream. Use client.property(ClientProperties.REQUEST_ENTITY_PROCESSING, RequestEntityProcessing.CHUNKED).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants