Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault (core dumped) when running USE #5

Open
kiaia opened this issue Oct 27, 2019 · 4 comments
Open

Segmentation fault (core dumped) when running USE #5

kiaia opened this issue Oct 27, 2019 · 4 comments

Comments

@kiaia
Copy link

kiaia commented Oct 27, 2019

Thank you for your code
I follow the instructions to try to reproduce the results. After Cos sim import finished! Segmentation fault (core dumped) happened, the previous steps are no problem, I use tensorflow 1.14.0, pytorch 0.41, gpu memory is 12gb. Can you please analyze what is the possible cause of the error?

@jind11
Copy link
Owner

jind11 commented Oct 27, 2019

importing the cos sim matrix takes more than 12 GB RAM, which caused your issue here.

@kiaia
Copy link
Author

kiaia commented Oct 27, 2019

Thx for your suggestion!I'm sorry that I may not have made it clear what problems I have.Let me add the information:
The server's free memory is more than 50GB, and the video card's memory is 12GB.comp_cos_sim_mat.py has been executed in advance, and cos_sim_counte_fitting.npyhas been imported successfully. I found there is a problem when calling self.embed = hub.module (module_URL) in USEand I found the issue of universal sense encoder, which seems to be a problom associated with tensorflow version along with tf-sentencepiece version . So far, I have not been able to solve the problem by trying to follow the above scheme. If there is any progress, I will give feedback.Could you give me some advice if possible?Thx a lot.

@ShuangNYU
Copy link

Thx for your suggestion!I'm sorry that I may not have made it clear what problems I have.Let me add the information:
The server's free memory is more than 50GB, and the video card's memory is 12GB.comp_cos_sim_mat.py has been executed in advance, and cos_sim_counte_fitting.npyhas been imported successfully. I found there is a problem when calling self.embed = hub.module (module_URL) in USEand I found the issue of universal sense encoder, which seems to be a problom associated with tensorflow version along with tf-sentencepiece version . So far, I have not been able to solve the problem by trying to follow the above scheme. If there is any progress, I will give feedback.Could you give me some advice if possible?Thx a lot.

Hi! I also found the problem about tensorflow_hub when loading USE. Have you had any progress on this issue? I tried the example code on website of USE but the same error occurred.

@jind11
Copy link
Owner

jind11 commented Jul 3, 2020

could you first make sure your environment strictly follows the requirements.txt file? I think that could probably solve this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants