You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First of all, thanks for this amazing library, really useful. Now going to the issue / request:
It seems to me that ApplyByCols and ApplyToRows could have a parameter to make them run in parallel. Has this ever been considered as a feature? I think it could speed up pipelines quite a lot, esp. useful for big dataframes. WDYT?
---- Edit by @shaypal5 : added in v0.0.67 -----
Also, I see that pipeline.fit and pipeline.transform have the timed bool. Would it be possible to add the same for apply? I know I can do pipeline.transform(...., time=True), but don't see a reason why apply cannot have it
The text was updated successfully, but these errors were encountered:
Hmm. Never considered this. I will definitely accept PRs around parallelization! :)
Definitely. Perhaps it was just an oversight. This should be an easy PR to make. Perhaps you can write it?
Also, would you mind separating this into two distinct issues?
The first one is complex, while the other I can label with the first good issue label. :)
First of all, thanks for this amazing library, really useful. Now going to the issue / request:
It seems to me that ApplyByCols and ApplyToRows could have a parameter to make them run in parallel. Has this ever been considered as a feature? I think it could speed up pipelines quite a lot, esp. useful for big dataframes. WDYT?
---- Edit by @shaypal5 : added in v0.0.67 -----
Also, I see that pipeline.fit and pipeline.transform have the
timed
bool. Would it be possible to add the same for apply? I know I can do pipeline.transform(...., time=True), but don't see a reason why apply cannot have itThe text was updated successfully, but these errors were encountered: