Inconsistency in assignment to object array.

[dimasad]

Consider the following code:

>>> import numpy as np
>>> x = np.zeros((1,1), dtype=object)
>>> y = np.zeros((1,1), dtype=object)
>>> z = np.zeros((1,1), dtype=object)
>>> a = np.array(None)
>>> x[0] = a
>>> y[0,0,...] = a
>>> z[0,0] = a
>>> x
array([[None]], dtype=object)
>>> y
array([[None]], dtype=object)
>>> z ### This is inconsistent!!!!
array([[array(None, dtype=object)]], dtype=object) 

As it can be seen, assignment of a zero-rank array to an object array causes nesting of array inside an array. I believe that the third case (assignment to z) should behave like the other cases, otherwise one can quickly generate arrays inside arrays inside arrays and necessitate extra code to treat special cases.

This was tested in numpy 1.9.2 under python 3.4.2.

[seberg]

Well, in its own twisted way, it actually is consistent. Unlike the other two, z is a scalar assignment (i.e. the result of the indexing np.array([[None]])[0, 0] gives back None and not np.array(None) as in the other two cases.
We can argue about the scalar vs. 0-d array result, but I am not sure there is a nice way to get around these kind of issues with object arrays, considering there is a difference between scalar and 0-d indexing.

[seberg]

You could look it the other way around. If this was not the case, I do not think there would be any possibility to put an array into another array! ;)
EDIT: OK that may be overstating, but you would probably have to do a dance to achieve it.

[leewz]

Yeah, the third one is a level deeper than the other two. Whatever the proper behavior is, it should NOT be the same as the other two.

See #6070, which suggests that e.g. array(None) shouldn't work without specifying dtype.

[dimasad]

I was also thinking about consistency with respect to numerical arrays, since for them we have the following behaviour:

>>> x = np.zeros((1,1), int)
>>> a = np.ones((), int)
>>> x[0,0] = a
>>> x
array([[1]])

Anyways, I had not thought of putting an ndarray inside an ndarray as a valid use-case. If that is so then this issue might simply be a feature. I think, however, that when ndarrays start to nest like that it is usually unwanted.

With respect to @leewz's comment, I used array(None) intentionally simply as a shorthand for array(None, dtype=object). The same behavior would happen if I had used a = np.ones((), dtype=object).

[seberg]

Yes, though you could argue that the way you get there is at least in principle reversed. It is not so much that the right hand side takes the element from the left hand side in the assignment. It is that the left hand side says that it can be interpreted as an integer (since it is 0-d).
Looking from that side you would have to make the int example an error probably, though that is likely unnecessary hassle.

[ahaldane]

Just to add to comments above: Assigning to object arrays is a special case that behaves differently from non-object arrays, specifically to allow nesting of ndarrays. For example, this special case is explicit in this code.

See also http://stackoverflow.com/questions/3273040/numpy-object-array-of-numerical-arrays

This is one of a couple "special" things about object arrays I think should be documented. (Another is the "special" casting rules that apply to object arrays). I think a little dedicated section on object arrays in the user docs would be good. Probably it should wait until some issues related to object arrays are decided though.

[dimasad]

I think it's safe to close this, then. I guess the simplest workaround to get the behavior I want is to always include the ellipsis for item assignment. Object arrays are tricky to work with but can help a great deal when working with sympy objects, for example.

[argriffing]

Here's a thing that seems inconsistent to me:

>>> a = np.array([[1, 2], [3, 4]])
>>> b = np.array([[1, 2, 3], [4, 5, 6]])
>>> np.array([a, b])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not broadcast input array from shape (2,2) into shape (2)

but

>>> np.array([ [[1, 2], [3, 4]], [[1, 2, 3], [4, 5, 6]] ])
array([[[1, 2], [3, 4]],
       [[1, 2, 3], [4, 5, 6]]], dtype=object)

It seems inconsistent that one creates an object array, but the other one does not (and gives an error instead).

[leewz]

Honestly, I prefer it that way. Numpy is for numbers. If you want something weird, you should have to explicitly say so. It should give you an error if you accidentally ask for something weird, and I can see that being an accident.