Skip to content

Issues: webdataset/webdataset

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

90 Open 202 Closed
90 Open 202 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

custom batch sampler ?
#359 opened May 23, 2024 by benihime91
pipe:s3cmd or pipe:aws s3 cp ?
#358 opened May 22, 2024 by cgebbe
Memory leak during training with standard DataLoader coupled with WebDataset dataloader
#357 opened May 20, 2024 by vishaal27
1
Update pypi with 0.2.88?
#349 opened Apr 15, 2024 by chrisdxie
how to use wids on local dataset
#330 opened Jan 30, 2024 by WudiJoey
3
Buckets of elements of different sizes
#325 opened Jan 18, 2024 by ethansmith2000
2
AttributeError: module 'webdataset' has no attribute 'Dataset'
#323 opened Jan 8, 2024 by YYDreamzure
1
Please provide a mechanism for detecting when one tarball has been fully consumed enhancement New feature or request faq Suitable for FAQ inclusion.
#314 opened Nov 30, 2023 by pabl0
@tmbdev
1
stream is deleted but not closed
#311 opened Nov 24, 2023 by wangkenpu
1
Problem about webdataset load the same tar file by epoch enhancement New feature or request
#310 opened Nov 16, 2023 by si22bao
@tmbdev
1
Curl in popen doesn't close if there are no tar shard on an AWS. enhancement New feature or request
#308 opened Oct 31, 2023 by dmasny99
@tmbdev
1
Loss of performance
#306 opened Oct 25, 2023 by karamavusibrahim
how to use num_workers in ddp training? faq Suitable for FAQ inclusion.
#303 opened Oct 8, 2023 by zhangvia
1
too many files when using multiprocessing with pytorch dataloader and decode bug Something isn't working
#301 opened Sep 30, 2023 by ekorman
@tmbdev
How can I use cache by "wds.DataPipeline(*pipeline)"
#295 opened Sep 19, 2023 by Yangr116
Problem with parsing filenames
#288 opened Aug 14, 2023 by gau-nernst
Compatibility with JSONL
#287 opened Aug 9, 2023 by zanussbaum
When I have a duplicate entry I get a secondary error
#286 opened Aug 4, 2023 by enjalot
"Issues with 'with_epoch' Configuration Relative to 'batch_size' in Distributed Training Using PyTorch Lightning and WebDataset" faq Suitable for FAQ inclusion.
#282 opened Jul 17, 2023 by ljqnb
1
issue in tariterators.py
#279 opened Jul 12, 2023 by omar-ahmd
1
The intended copy behaviour of compose is not achieved. bug Something isn't working
#272 opened Jun 1, 2023 by ferrine
@tmbdev
2
resume dataloader enhancement New feature or request
#271 opened May 22, 2023 by mactavish91
@tmbdev
1
Arg repeat of class WebDataset is not used bug Something isn't working
#269 opened May 2, 2023 by ychfan
@tmbdev
3
Sharded Dataset Has Long Delay Before First Batch (and caching=False)
#267 opened Apr 30, 2023 by cupdike
8
Stream data from Hugging Face enhancement New feature or request
#265 opened Apr 28, 2023 by lhoestq
@tmbdev
5
ProTip! Type g i on any issue or pull request to go back to the issue listing page.