Parallelization with more than one port

[mr-j0nes]

Hey Jonathan,
hope your 2018 is going well.

I have a quick question about parallel streams. Say I want to have following configuration:

 m += a <= b ["processOne"]=> c
 m +=      b ["processTwo"]=> d

Then the port processOne is multiplied (e.g. processOne0, processOne1,, etc) as many times as threads in c and the same for processTwo and d.

As I understand the current framework does not support this. Is there any specific reason why it is not supported now? And what do you think would be the efforts for introducing something like this (i.e. do you think I can implement it myself) ?

Thanks a lot for your help..

[jonathan-beard]

Ok. So, the syntax

m += a <= b => c

would replicate b for as many times as there are output ports in _a, assuming that the output port count on _a and the input port count on _b would be equal, otherwise an exception would be thrown.

The syntax

 m += a <= b ["processOne"]=> c
 m +=  b ["processTwo"]=> d

Would in this case would create something that looks like this

The runtime would then have to check to ensure that the output port count on _a matched the input port count on both _c and _d. Then the one instantiated _b kernel would be replicated to match the number of ports as in the next figure.

Is that the intended behavior of what you would like to implement? If so, I can help you out if you're still interested in implementing. If not, let me know and lets think though it a bit. In either case, happy to help. Apologies for the delayed response. It's been a very busy few months.

-Jonathan

[mr-j0nes]

Hi Jonathan,
now I am the one apologizing for the late response, it's been crazy busy here too.

Indeed that was the intended behavior of what I wanted at the time. In between the designed has changed a little bit in my mind, I am thinking now more of a paralellization of every single kernel independently of the paralellization of the next one. So given:

 m += a <= b ["processOne"]=> c >> e
 m +=      b ["processTwo"]=> d

I would like to give b and c different number of threads. I think I can achieve this by adding before and after each of those kernels a split one, sort of what you do with raft::order::out, but keeping the order.

This feature has very low priority in my project right now so I will not need it for now. When I need it I think this can be achieved from outside the library (i.e. making my own split kernels), so we can consider this post closed for now..

Thanks again for a great library..