Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TensorArray fails when used as boolean mask index #162

Open
BryanCutler opened this issue Jan 14, 2021 · 2 comments
Open

TensorArray fails when used as boolean mask index #162

BryanCutler opened this issue Jan 14, 2021 · 2 comments

Comments

@BryanCutler
Copy link
Member

BryanCutler commented Jan 14, 2021

Pandas can not recognize an extension array that when taken as numpy, it is a 1-D boolean array and use that array as a boolean mask for indexing.

arr = tp.TensorArray(np.arange(20).reshape(10,2))
s = pd.Series(arr)
thresh = s > 8
s[np.all(thresh.array, axis=1)]

results in:
KeyError: "None of [Index([False, False, False, False, False, True, True, True, True, True], dtype='object')] are in the [index]"
or other strange errors because it is not picked up as 1-D boolean array and tries to be a list-like indexer or something else

@BryanCutler
Copy link
Member Author

BryanCutler commented Apr 5, 2021

In the notebook Text_Extenstions_for_Pandas_Overview an example shows a TensorArray used as a boolean mask:

s[np.all(thresh.array, axis=1)]

This is now failing when the Series tries to validate the mask. Need to find a fix or another way to do this.

This has been resolved in the notebook with a workaround. I wanted to leave this open because Pandas should be able to recognize an extension array that converts to a 1-d bool array and use that as a boolean index.

@BryanCutler
Copy link
Member Author

Fixed up the issue to better describe the required functionality from Pandas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant