On first glance, I wonder whether it wouldn't be cleaner and easier to create IterationNode as a subclass of INode. That way, you can forego all the checks and discard -1 as a sentinel value for no iteration. If it's supposed to iterate, it's gonna be an iteration node.

Another note: Personally, I'm not a big fan of assert in python - the exception system is a great tool, if the exceptions are specific enough and the proper exceptions are raised. As an example, when you check the iteration_count variable, I find it preferable to use something like

from numbers import Integral
if iteration_count is None:
    pass
elif not isinstance(iteration_count, Integral):
    raise TypeError("iteration_count has to be an integer")
elif 0 < iteration_count:
    raise ValueError("iteration_count has to be positive")

This way, the errors raised are more informative.

I thought about making the IterationNode a child of the INode but in that way you need to somehow specify it when creating the node. For example I mainly use the following syntax

@Node(outputs=['out'])  # How to specify that out is an `IterationNode`? 
def foo(baz)
    return {'out': baz}

Also the IterationPlug is nothing more than a list with a specific type, just to trigger the iteration within the evaluation of the Node.

I agree on the assert I use them quite a lot for type checking, but it isn't the best way.

I see two options here: Either the @Node decorator takes an optional argument iteration=False/True to know whether to create a basic INode or an IterationNode, or we set up an @IterNode decorator that creates iteration nodes. I prefer the second option, since it's more explicit.

If the existence of a specific IterationNode class makes the IterationPlug obsolete, that's all the more reason to build it, imho.

To share this a tad more explicitly, because it caused me pain as a user in the past: The recommended way to check for integers is to

from numbers import Integral
if isinstance(unkown_number, Integral):
    do_stuff_with_integer(unknown_number)

The abstact class Integral does a number of checks for whether a value is an integer. This way, it will e.g. also accept numpy's various integer types, which don't inherit from the built-in int.

I see two options here: Either the @Node decorator takes an optional argument iteration=False/True to know whether to create a basic INode or an IterationNode, or we set up an @IterNode decorator that creates iteration nodes. I prefer the second option, since it's more explicit.

Agree but what about Nodes that Iterate depending on their input? I can imagine a case where a Node will output an IterationPlug depending on it's input. So at compile time it's not always clear if an iteration will happen.

But won't you have to set up an IterationPlug at "compile time" to iterate anyways? If I understand the current implementation correctly, anything that's not in an IterationPlug will not get iterated over, and you'll have to define the plugs when setting up the graph anyways.

I understand the miscommunication, it's because I called it an ItertationPlug, which is wrong... The output is just a normal OutputPlug but the value of the OutputPlug (eg. plug.value = ItertationPlug([1, 2, 3])).

The type of the value is just a trigger for subsequent nodes to iterate over and recall itself. Naming it a plug wasn't correct maybe..

Okay, then how does a node know whether to set the output as a regular list or as a list to iterate over? As far as I can tell, this is decided by the iteration_count attribute of a node, which is set when initializing a node. Or is there some other mechanism to trigger the iteration?

Beforehand it doesn't knows.. if you make a Node like this:

@Node(outputs=['out'])
def foo(baz):
    return {'out': baz}

This Node is not specifically made for iteration, but when baz is an IterationPlug (or how we would like to call it), it will recall itself for every value in baz and create an output out which is then becomes also an IterationPlug

I understand. But what would stop you from creating all nodes should potentially iterate over inputs as IternationNode? In case they don't receive an IterationPlug typed input, their behaviour is the same as a regular node.

I feel uncomfortable if the behaviour of my nodes is defined at run-time based on (potentially dynamic) inputs. That's why I would feel more at ease if this feature was part of a separate node type. Flowpipe's graph executing sucessfully right now has a pretty solid guarantee that each (dirty) node gets executed exactly once, which is a very powerful statement for such a modular system.

Sorry for joining the discussion so late.

I agree with @neuneck's requests and the reasons given. To sum it up:

• The IterationNode should be a suclass of INode.
• It will be available via it's own decorator @IteratorNode.

@Onandon11 would these changes interfere in any way with your usage of flowpipe?

Alright, we will manage :) I cannot oversee the impact on our repo currently, but we will find out fast enough.

I will try to implement it this week :)

I was starting to implement the proposal. But it's still a bit unclear for me, do you want and IterationNode which is a subclass of INode and will always iterate over all of it's inputs?

Basically yes, but a mix of iterative and fixed inputs makes a lot of sense, IMHO. My suggested solution would be to have an IterationPlug, a subclass of InputPlug, which will lead to the node iterating over values inside that plug. This makes very clear what will happen at runtime, just form the way nodes are set up. The output can then just be set to a list of the results of the compute() method. If the next node should iterate over those outputs, all you'd have to do is connect the output up to an IterationPlug.

There are some further things you should decide on:

• If there are multiple iteration inputs, should they be ziped?
• If so, what should happen when the inputs have a different length?
• Should the compute() be run on the cartesian product of the input iterables?
• Should that be something the user defines, and if so, how?