Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converting data types takes longer time #15621

Closed
Hellcat1005 opened this issue Feb 21, 2020 · 6 comments
Closed

Converting data types takes longer time #15621

Hellcat1005 opened this issue Feb 21, 2020 · 6 comments

Comments

@Hellcat1005
Copy link

Hellcat1005 commented Feb 21, 2020

Reproducing code example:

import numpy as np
import time

a = np.zeros((400, 400, 400), dtype=np.int32)
a[a == 0] = 255
t1 = time.time()
b = a.astype(np.float32)
print(time.time() - t1)

For numpy 1.17.3, it take 5 seconds. For numpy 1.16.4, only 0.1 seconds.

@charris
Copy link
Member

charris commented Feb 21, 2020

I don't see this on my machine:

In [3]: %timeit b = a.astype(np.float32)                                        
22.4 ms ± 320 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [4]: np.__version__                                                          
Out[4]: '1.19.0.dev0+f1a247f'

I suspect this is another case of #15545.

@charris
Copy link
Member

charris commented Feb 21, 2020

Possibly useful info:

  • gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1)
  • vmlinuz-5.4.8-200.fc31.x86_64

@seberg
Copy link
Member

seberg commented Feb 21, 2020

@Hellcat1005 just to be sure and narrow it down, gh-15545 suggests that this is a kernel speed issue that was fixed (probably in 4.6), could you tell what uname -a gives you (which kernel version you have). To help converge on the issue?
(I also cannot reproduce the slowdown, but my kernel/gcc versions are almost identical to Chucks)

@Hellcat1005
Copy link
Author

@seberg
uname -a gives me: Linux huiying 4.4.0-170-generic #199-Ubuntu SMP Thu Nov 14 01:45:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

@bmerry
Copy link
Contributor

bmerry commented Feb 21, 2020

@Hellcat1005 upgrading to kernel 4.15 fixed similar performance issues for me. Assuming you're using Ubuntu 16.04, try installing linux-image-generic-hwe-16.04.

@rossbar
Copy link
Contributor

rossbar commented Jul 14, 2020

Seems very likely this is another instance of gh-15545, for which a fix was introduced in gh-15769. If the issue persists, please open a new issue with information about the system.

@rossbar rossbar closed this as completed Jul 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants