Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Dataloader for large dataset. #7

Open
RuihongQiu opened this issue Mar 24, 2019 · 2 comments
Open

Test Dataloader for large dataset. #7

RuihongQiu opened this issue Mar 24, 2019 · 2 comments
Labels
help wanted Extra attention is needed

Comments

@RuihongQiu
Copy link
Contributor

Maybe a test dataloader for iterate the test set is more user friendly for large dataset? Just simply treat all test data as a huge batch sometimes will cause OOM error.

image

Sometimes the size could be like this.

@yihong-chen
Copy link
Owner

Couldn't agree more. A test dataloader would be better. I will add it to my todo list.

@RuihongQiu
Copy link
Contributor Author

Hope to see the method. I try to implement it but get stuck in the calculation of HR and NDCG.

@yihong-chen yihong-chen added the help wanted Extra attention is needed label Mar 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants