shift operator does not work with numpy arrays with dtype=uint64

[Martin-Seysen]

I have tested the following python code on 64 bit Windows 7:

import sys
print "python version is:\n", sys.version
import numpy as np
print "numpy version is:\n", np.version.version
a = np.array([1,3], dtype = np.int64)
print "a[1] is:", a[1]
print "type of a[1] is", type(a[1])
print "a[1] << 1 is:", a[1] << 1 # works fine for dtype = int64
u = np.array([1,3], dtype = np.uint64)
print "u[1] is:",u[1]
print "type of u[1] is", type(u[1])
print "u[1] << 1 is:", u[1] << 1 # but for dtype = uint64 there is a problem

This script produces the following output:

python version is:
2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)]
numpy version is:
1.8.1
a[1] is: 3
type of a[1] is <type 'numpy.int64'>
a[1] << 1 is: 6
u[1] is: 3
type of u[1] is <type 'numpy.uint64'>
u[1] << 1 is:
Traceback (most recent call last):
File "test_uint64_array.py", line 12, in
print "u[1] << 1 is:", u[1] << 1
TypeError: ufunc 'left_shift' not supported for the input types, and the inputs
could not be safely coerced to any supported types according to the casting rule ''safe''

I can do my job with int64 instead of uint64. I just want to report this problem.

[jaimefrio]

It seems to be an issue with array scalar, not ndarrays:

>>> import numpy as np
>>> u = np.array([1, 3], dtype=np.uint64)
>>> u << 1
array([2, 6], dtype=uint64)
>>> u[1] << 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: ufunc 'left_shift' not supported for the input types, and the inputs
could not be safely coerced to any supported types according to the casting rule
 ''safe''

[mcara]

@jaimefrio wrote:

It seems to be an issue with array scalar, not ndarrays

I disagree... Here are a couple of counter examples:

>>> u = np.array([1, 3], dtype=np.int64)
>>> u[0]<<1
2
>>> u = np.array([1, 3], dtype=np.uint8)
>>> u[0]<<1
2

Even more interesting:

>>> u = np.array([1, 3], dtype=np.uint64)
>>> u[0]<<np.uint64(1)
2

[mhvk]

@mcara, - I think your examples confirm what @jaimefrio stated: only for uint64 scalars is this a problem, and only if you use as a the shift something that could have a sign. This probably implies that it tries to promote the uint64 number to something signed (which would be float64), and, since this doesn't happen for arrays, that the casting rules for arrays and scalars must have gone out of sync.

[eric-wieser]

and, since this doesn't happen for arrays, that the casting rules for arrays and scalars must have gone out of sync

Sounds like #8809 to me

The killer combination seems to be uint64 (scalar) << int here.

[mcara]

@mhvk I missed that the title already restricted the problem to uint64. Sorry.

[mcara]

I see quite a few issues with uint64 and all of them seem to be "related":
#5745, #7126, #4151, #2955, #2524 (since 2012!), #8002 - all open.

[mcara]

The killer combination seems to be uint64 (scalar) << int here.

The following example shows that the vice versa holds true:

>>> import numpy as np
>>> np.__version__
'1.11.3'
>>> np.int64(1)<<np.uint64(61)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: ufunc 'left_shift' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

[mhvk]

I think the problem in all cases is the promotion to float64 (which looks at both arguments and is independent of the actual function; for other functions, though, the float operation is defined). What I don't understand is why it doesn't happen for arrays...

Actually, this is not quite true:

np.array([1], dtype=np.uint64) << 1
# array([2], dtype=uint64)
np.array([1], dtype=np.uint64) << np.array([1], dtype=np.int8)
# TypeError: ufunc 'left_shift' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

So I suspect the difference is that for the array case, the 1 gets translated into an array with dtype np.uint64, while for the scalar case it is not. Indeed, when for the array case I select a negative integer, which cannot be represented as uint64, a TypeError ensues.

np.array([1], dtype=np.uint64) << -1
# TypeError: ufunc 'left_shift' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Anyway, the conclusion seems to be that for the array scalar case, the cast from python int is done differently than for the array case.

[seberg]

This is related to NEP 50, which is/will fixed in main/NumPy 2.0, so closing.