Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break down the List type #1166

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Break down the List type #1166

wants to merge 5 commits into from

Conversation

remram44
Copy link
Member

Right now there is a single List type that can represent different things. It represents Python lists (potentially nested), but also numpy arrays (which don't always behave like list, for instance + doesn't concatenate, and truth value has a different meaning), and it's also the magic type for streaming and aggregating multiple connections (i.e. it is special to port depth).

This is an attempt to rework that. It introduces Array, which is the base class (basically replaces List), plus two subclasses, List (that now represents a Python list, a module declaring this port type can expect a proper Python list) and NumpyArray (for a numpy.ndarray).

This came up during my work on the interpreter and pandas.

It turned out to be complicated, because of all the code that checks for the List type (for depth magic). I'm also not sure whether List shouldn't completely be eliminated (though NumpyArray is definitely useful).

@rexissimus
Copy link
Member

Would it not be better to only do the list magic on List, not Array, so that it is not possible to accidentally loop a numpy array? Looping a list over a module, or aggregating connections, usually requires a Python list.

We could also get rid of List by converting it to a Variant of depth 1. The looping magic already treats them the same. In most cases it is better to use a typed list (type of depth 1) for a simple list of objects, or a numpy array for bigger data.

@remram44
Copy link
Member Author

Would it not be better to only do the list magic on List, not Array

This is my intention, but Array ports have to accept what List ports accept, which is not the case right now.

* List renamed to Array, represents either an numpy ndarray or a list.
* Introduces List, represents a Python list only.
* Introduces NumpyArray, represents a numpy ndarray only.

The input ports of List and NumpyArray still accept Array, so these can
be used to cast between List and NumpyArray (list and numpy.ndarray)
easily.

The rationale is that ndarray doesn't behave exactly like list, in
particular operators will broadcast on ndarray (+ will concatenate
lists) and truth value is defined differently.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants