Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

put_object_stream reads stream to determine it's size #350

Open
annmuor opened this issue Jun 7, 2023 · 1 comment
Open

put_object_stream reads stream to determine it's size #350

annmuor opened this issue Jun 7, 2023 · 1 comment
Assignees
Labels

Comments

@annmuor
Copy link

annmuor commented Jun 7, 2023

Describe the bug
When using async put_object_stream I assume that the stream will be readed part by part as in async mode, but in reality it reads the whole stream to determine the size first. For large files it leads to OOM errors.

To Reproduce

let (mut rx, mut tx) = tokio::io::duplex(65535); // tokio duplex
tokio::spawn(async move { loop  { tx.write_all(&[1,3,4,5,6]).await; } });
Bucket::put_object_stream(b, &mut rx, "/123.txt").await;

Expected behavior
Memory is not overflowing, data is written as you go.

Environment

  • Rust version: 1.69
  • lib version 0.33.0

Additional context

@robinfriedli
Copy link
Contributor

This seems to be covered by #346

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants