Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ucf11 evaluation problem #11

Open
ae86208 opened this issue Aug 31, 2016 · 1 comment
Open

ucf11 evaluation problem #11

ae86208 opened this issue Aug 31, 2016 · 1 comment
Assignees
Labels

Comments

@ae86208
Copy link

ae86208 commented Aug 31, 2016

image

After update 100, the model begin to predict, while as shown above, the prediction has been running for over 24 hours without progress to new model update, is that normal? If not, what maybe the main cause of this problem? I'm using your code of issue #6 to do the data preprocessing, with train/validate/test 6:2:2, thanks!

@kracwarlock
Copy link
Owner

Hi @ae86208 the PD time (which is the data loading time) is quite large (~640s it should generally be < 5s) in your experiments. While testing this is going to take a huge amount of time since it will go over all batches. You will need to investigate why this is happening.

Maybe the hard drive you are using for the dataset has a lot of latency? In any case you can either

  • rewrite the data handler code for loading batches
  • modify it to load all the batches at once provided you have enough memory

@kracwarlock kracwarlock self-assigned this Oct 4, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants